✨Build & Learn with Coffee☕ AI Engineering Build Week 6

📅 Week 6: Monitor and evaluate your LLM application

This event is part of our seven-week AI Engineering program, where we build an end-to-end AI application step by step and present our finished projects during Demo Day in Week 7.

This week, we’ll focus on understanding how our LLM application behaves after deployment and how to measure whether its outputs are actually useful.

Monitor and debug LLM calls with Langfuse tracing.

Manage and version prompts with Langfuse Prompt Management.

Introduce the main concepts behind LLM evaluation.

Write focused unit tests for predictable application behavior.

Build evaluation datasets from representative examples.

Add human annotations and feedback using Langfuse scores.

Build an LLM-as-a-judge evaluator

By the end of the 7 week cycle, you should have a working AI application in your own GitHub repository and practical experience with AI pipelines, APIs, databases, retrieval, evaluation and deployment.

You are welcome to follow the shared project or apply the weekly concepts to your own project.

✨ Who is this for?

Anyone interested in transitioning towards AI engineering.

No previous LLM or AI engineering experience is required. However, I recommend some basic programming experiences or be to help with your first build session.

💻 Bring: Your laptop and curiosity. Grab ☕ at the coffeeshop

💬 Our slides and meeting link with be shared on our Discord → Join Discord Server