PalanerFree on the App Store
Get
PalanerAIFinds what's on in Zurich
Sign in

AI Safety Weekly: Alignment from Token Zero
WorkFree

AI Safety Weekly: Alignment from Token Zero

Tuesday 17 November 2026, 18:30

Venue time (Zurich)

ETH HG Hauptgebäude · Directions

Tuesday 17 November 2026, 18:30Venue time (Zurich)
ETH HG HauptgebäudeDirections
FreeTickets sold by luma.com
Sold out — view event pagevia luma.com
Add to calendar

Downloads an .ics file · Times are in Europe/Zurich · Google Calendar

Hey, how's it going? AI Safety Weekly: Alignment from Token Zero looks like it should be pretty interesting. Want me to help you organise around it?

Event details

Tickets sold by luma.com

Add to calendar · Downloads an .ics file · Times are in Europe/Zurich · Google Calendar

IMPORTANT: This session will be more technical than other sessions in the series.

Most alignment work today happens after pretraining. A model first learns from huge amounts of text, picking up all kinds of behaviors and voices along the way, and only afterwards is it trained to act as a helpful, harmless assistant. As a result, a model’s values can end up as a thin layer on top of everything it learned before, rather than something deeply rooted. But what if alignment started from the very first token of training?

In this themed session, V. Moskvoretskii (an EPFL PHD student) will present his recent work, Synthetic Persona Pretraining: Alignment from Token Zero. Instead of adding the assistant persona at the end, Synthetic Persona Pretraining (SPP) builds it in from the start: pretraining documents are annotated with first-person moral reflections based on a value constitution, and post-training then ties the assistant identity to this pretrained persona. In models of up to 3B parameters trained on 500B tokens, SPP improved how well models follow their constitution, made them more robust to jailbreaks, and reduced misaligned choices in moral dilemmas, all without hurting capabilities. Timing turns out to matter: introducing the same intervention only at the end of pretraining works noticeably worse. The technique is also part of the training of Apertus 2, the next version of the Swiss open-source LLM, so this is a chance to hear about safety work going into an open model built right here in Switzerland.

After the presentation, we’ll open the floor for discussion. Should alignment start at the pretraining stage? What are personas in LLMs, and how do they form? When in training is the best time to introduce desired properties, and can earlier really be better than later? And how do we even measure these differences, especially in small models? We can also use the session to discuss questions around open-source AI safety more broadly.

🍕 Food will be provided after the session for networking and further discussion.

📅 Tuesday, 17 November, 18:30–20:00📍 ETH Main Building, HG D3.3.

Moderate technical background is recommended. Basic familiarity with how large language models are trained (pretraining vs. post-training/fine-tuning) is benefically. Registration is required.

Ask Palaner

Going to AI Safety Weekly? Ask me anything about it.

I read the organiser's pages and answer in a few seconds.

Answers are AI-generated · Privacy

Answers are AI-generated · Privacy