SGLang Summit Workshop by Modal: Inference from First Principles

A SGLang Summit ticket is required to attend the event and get approved on Luma. Both Full and Expo passes are accepted.

Ticket 🎫 https://luma.com/sglang-summit

The room seats 45, and seating is first come, first served, so registering doesn't guarantee a seat. Please arrive early to get a spot.

Modern LLMs are increasingly co-designed with the systems that serve them, with architectures built from the ground up to be efficient for inference. But the reasoning behind these design choices is often reduced to graphs and benchmarks showing numbers going up, with little explanation of why they work. In this talk, we'll start from first principles; using simple analytical models, we'll derive key properties of today's models and serving systems, and explore what they suggest about the next generation of inference.

Session Details

Duration: 60 minutes

Level: Moderate experience in LLM inference serving. Familiarity with concepts like sharding and hybrid attention will be helpful.

What to bring: No laptop needed

Speaker

Gilford Ting, Member of Technical Staff, Modal