Hands-on workshop: Build, trace, and eval

How do you know a customer-support agent retrieved the right policy, escalated the risky case, and did not invent a refund?

This is a hands-on workshop that takes a LangGraph customer-support agent through the core Arize AX dev loop: build → trace → offline-eval → experiment → chat live & watch traces.

You build "Sunny", a LangGraph customer-support assistant for a fictional outdoor retailer. The agent does real retrieval over a small policy knowledge base and escalates hard or high-risk requests to a person. The workhorse model is gpt-4o-mini. You then trace the run in Arize, score it offline against a curated golden dataset, and compare gpt-3.5-turbo with gpt-4o-mini.

What you will do

Build a tool-using agent: orders, refunds, knowledge-base retrieval, and escalation

Trace it into Arize with two lines of auto-instrumentation, including retrieval spans

Run an offline eval against a curated golden dataset, with a code check and an LLM judge

Compare gpt-3.5-turbo and gpt-4o-mini side by side

Chat live in a Gradio UI and watch your own traces appear in Arize

Who this is forAI engineers, software engineers, ML engineers, and data scientists who are building agents.

What you will leave withA working support agent, traces in Arize, an offline eval, and a model comparison you can rerun.

FormatHands-on workshop of about two hours, then optional networking drinks. Bring a laptop and charger. Have an OpenAI API key and a free Arize account (see prerequisites below).

Agenda

2:15 PM | Arrival

2:30 - 4:30 PM | Hands-on workshop

4:30 - 5:30 PM | Networking drinks

PrerequisitesPlease have these ready before the session:

An OpenAI API key (OPENAIAPIKEY) — get one at platform.openai.com/api-keys (Arize can provide this on the day if you don’t have one).

A free Arize account at app.arize.com, and your Space ID + API key from Settings → Space API Keys.

A Laptop

That's the only input you need. The notebook prompts for all three keys at the top.