Data & AI Under the Hood: Real-Time, ML Pipelines & Agents

Go beyond the buzzwords and explore the engineering behind production data and AI systems: real-time processing, reliable ML workflows, and agentic automation.

Agenda:

18:00 – Mingling & Food 18:30 – Opening Words and Talks

When Real-Time Gets Heavy: Shrinking the Critical Path at 13K Events/secAviv Zikel · Senior Data Engineer

Keeping personalization fresh is a race against the user’s next click. At Dynamic Yield, our user-preference pipeline handled ~13K events/sec, but scale exposed a trap: the system kept up, while the critical path kept accumulating context, delayed state updates, and domain decisions. Each choice made sense locally. Together, they made the system stale, expensive, and hard to change. This talk shares how we redesigned the pipeline so real-time decisions depended on less work at the moment that mattered most: bringing context closer to the stream, rethinking state ownership, and cutting this pipeline’s infrastructure cost by ~92%.

From Scripts to Specs: Converting a Chaotic Script Folder Into a Streamlined Training and Evaluation PipelineAnat Shkolyar · Data Scientist, CV Engineer & Algorithm Developer

Data Science projects tend to start the same way: a folder full of one-off scripts for data preparation, training, and evaluation. It works, right up until you need to reproduce last month's model and can't. I'll share a practical engineering journey that starts messy and ends up reproducible, with a taste of how metrics can lie, and how to avoid it.

The Agent Boss: How I Built Agentic Workers for My Data Science WorkflowVictoria Khalfin Fekson · Senior Data Scientist

AI copilots made us faster at writing code, but I'm not interested in writing repetitive code faster. I want to stop writing it. That's what led me to turn parts of my data science workflow into independent AI workers. In this talk, I'll show how I moved from using AI as an assistant to managing workers that execute real parts of a production DS pipeline - including feature extraction, validation, and model tuning. We'll talk about what makes a good "job" for an agent, how much autonomy to give it, why smarter isn't always better, and why Human in the Loop is a feature, not a failure. The goal isn't an autonomous Data Scientist. It's a Data Scientist who doesn't have to do everything herself.