AI Signals 34: Agent Harnesses, and Geometric Thoughts

​Supported by lenses.io Agenda 06:00pm - Doors open 06:30pm - Welcome 06:35pm - Alper Nebi Kanli, Optimizely, "How LLMs Represent Concepts Internally: A Tour of the Geometry" 07:05pm - Break 07:20pm - Tun Shwe, Lenses.io "Understanding Agent Harnesses" 08:00pm - Lightning Talks: "Which AI Model Will Win the World Cup?" Fred Smits Please note you will be unable to enter the venue before 6.00pm. RSVPs will close 24 hours before the event, you may be unable to register after this time but you can still watch online. If you can't join us in person you can watch remotely via our YouTube channel. Our hosts may require that we provide a list of all attendees, please ensure that you register with a name that matches your government issued ID or bank card: if you do not we cannot guarantee you entry to the building. Please RSVP for the event well in advance if you plan to attend in person and unRSVP if you can no longer attend as limited spaces are available. Large language models look like they just predict the next word, so it is surprising how much structure they build inside. They actually represent concepts: you can find directions in their activations for things like gender, sentiment, or even truthfulness, and you can use those directions to steer what the model does. This talk starts there, with the evidence that concepts really are represented inside these models. Then it goes one level deeper: what does it actually mean for a concept to be "a direction"? The answer depends on geometry, on how you choose to measure, and that choice does more work than it looks. You will come away seeing the inside of an LLM a little more clearly, with a few surprises about something many of us use every day.