The Test Pyramid Survives. Assertions Don't: Testing AI in the Real World

The Deets:
Code Talent: 3412 Blake St, Denver, CO 80205
7pm-9pm (doors 6:30)
Pizza and Soda provided.

Join us at Code Talent for our Local Expert speaker series featuring Elliot Sequeira as he discusses "The Test Pyramid Survives. The Assertion Doesn't. LLM-as-a-judge at every layer of the AI test pyramid."

Every engineer knows the testing pyramid: a wide base of fast unit tests, fewer integration tests above, a handful of end-to-end tests at the top. It has outlasted every major shift in how we build software.

Then a language model enters the system, and exactly one piece of it stops working. Not the shape — the assertion. `assert output == expected` has no meaning when the output is a paragraph.

Join Elliot (QE/QA/SDT/Leadership) for a session on that transition: what the testing pyramid becomes when the system under test is both deterministic and non-deterministic, and why LLM-as-a-judge is the one assertion primitive cheap enough to run at every layer of it. We'll walk the pyramid three times — the classic version, the AI version with a new foundation band underneath it, and the version with a judge operating at every band — then get into the pipeline and the data engineering that make it real.

Doors open at 6:30pm. Attendees are eligible to receive a \free\ print book from our partners at O'Reilly Media upon becoming a member of The Applied Machine Learning Collective of the Rockies! (https://amlcoftherockies.org/members/)
You don't want to miss this talk!

What is AMLC of the Rockies??
We are a 501(c)(6) nonprofit. No paid employees. Volunteer-led. The AMLC of the Rockies strives to offer a diverse range of activities for both members and non-members.

We are a professional association, working to collectively learn and improve outcomes as AI/ML practitioners within the industry, while also pursuing our own endeavors as hobbies. Our goal is to pool our resources, roll up our sleeves, and get hands-on!

Join us for talks, hands-on workshops, access to cloud infrastructure, free books and resources, networking, recruitment and talent opportunities, and more! Help us collectively make Colorado the world's best technology hub.

In machine learning and life, collective communication scales.