Inference on Kubernetes / Agentic AI Parallelization

For the November meetup the first preso will be from Hrittik Roy, a Platform Advocate at vCluster Labs and CNCF Ambassador, “From Local Inference to Kubernetes: What Changes When Your LLM Gets Busy?”. The presentation covers GPU scheduling, batching, KV cache and inference observability, using vLLM and NVIDIA Dynamo as examples,

For the second presentation Arsen Yeremen returns to discuss agentic AI development of a multi avatar system.

See the Presentation Details section below for more info.

Sponsor and LA DevOps Meetup:
Sponsoring this or a future meetup is great way to promote your service or product and to give back to the local DevOps Community,'

Reach out to one of the hosts for details.

Event Details:
6:00 - 6:30 Attendees arrive. Networking/Mingle
6:30 - 6:45 LA DevOps Welcome,Attendee Introductions
6:45 - 7:15 Hrittik Roy - From Local Inference to Kubernetes: What Changes When Your LLM Gets Busy?
7:15 - 8:00 Arsen Yeremin - TBA
8:00 - 9:00 Networking/Mingle

Presentation Details:

Title: From Local Inference to Kubernetes: What Changes When Your LLM Gets Busy?

Description:

Running an LLM locally is a great starting point. But what happens when one user becomes hundreds, requests compete for GPU memory, and response times start climbing?
This talk follows an inference request from prompt to generated tokens, explaining prefill, decode, and the KV cache. We’ll explore how batching, model caching, and GPU scheduling affect latency and throughput, then introduce disaggregated inference: running prefill and decode on separate workers, including when the added complexity makes sense.
Using vLLM and NVIDIA Dynamo as examples, we’ll connect these concepts to serving models on Kubernetes and measuring what matters: time to first token, token generation speed, and GPU utilization.
Key takeaways:

Understand what happens inside an LLM inference request.
- Identify bottlenecks as you move beyond local serving.
- Learn when to consider disaggregated inference.
- Understand the Kubernetes building blocks for GPU workloads.

About Hrittick:

Hrittik Roy, a Platform Advocate at vCluster Labs and CNCF Ambassador, he helps engineering teams and open source communities get more out of products through demos, documentation, talks, and deep dives into AI Infrastructure, developer workflows, and platform engineering. The goal is always the same: take something complex and make it click.

Title: TBA
Description:

About Arsen:
Arsen Yeremin is a CTO and solutions architect with 25+ years of software engineering experience, including 8+ years at the intersection of AI/ML and blockchain. He’s currently leading AI infrastructure and agentic systems at Tygher AI, and has previously served as Fractional CTO of Lilypad Network (decentralized compute for AI) and CTO of Blockfreight (blockchain-based global logistics). Arsen specializes in building production systems that combine autonomous agents, distributed consensus, and enterprise-grade reliability—with hands-on expertise in containerized LLM deployment, on-chain state anchoring, and distributed system design. When he’s not architecting the future of autonomous organizations, he’s exploring how to make decentralized AI genuinely usable for enterprises.

About O'Brien's Pub on Wilshire:
The covered patio at Obrien's Pub has been a great place to host LA DevOps meetups for our post covid revival. The food and drink are great and the staff are very friendly. O'brien's is also the LA home of Manchester United Football Club Supporters.
You may want to bring a jacket for the cooler months.

Regarding parking, there are meters on Wilshire and the side streets that expire at 6.

Present at an Upcoming LA DevOps Meetup:
If you would like to present at an upcoming LA DevOps meetup please send a message in meetup to the organizers with a proposed title and a sentence or two about the talk.

Sponsor DevOpsDayLA at SCALE24x April 2nd 2027
Scale24x runs from April 1 - 4 at the Pasadena Conventrion Center, and DevOpsDayLA is on the Friday.

Download the DevOpsDayLA Prospectus for more info.

Subscribe to the LA DevOps Meetup Calendar at lu.ma
The LA DevOps events are also listed on lu.ma which has an interesting interface. Subscribe to LA DevOps on lu.ma at https://lu.ma/calendar/cal-F7BPqBLAe80T5nS