The Information Bottleneck
The Information Bottleneck
Post-Training Is Where Models Are Made Now | Maxime Labonne
0:00
-1:03:56

Post-Training Is Where Models Are Made Now | Maxime Labonne

In this episode, we talked with Maxime Labonne, Head of Post-Training at Liquid AI and author of the LLM Course on GitHub, about what happens to a model after pre-training and why that stage now matters so much.

Maxime walks us through the short history of post-training, from supervised fine-tuning and preference optimization to reinforcement learning with verifiable rewards and now multi-domain on-policy distillation (MOPD), where separate teachers for code, math and other domains get merged back into one student. He explains why you still can’t skip SFT, why the order of the stages matters, and why RL works even though a long rollout gives you about one bit of signal. His intuition is that SFT and distillation raise pass@k, and RL then sharpens it into pass@1.

We also get into training small models for agentic tasks directly inside the harness, which open-source RL stacks are good enough for serious work, looping transformers, and how much of the work big models do today could move to small ones running locally.

We also talk about why models write worse than they used and safety issues. We talk about regulation, open source and Chinese models,  whether a small startup can beat the big labs in a niche, and why he thinks RL environment companies are overrated.

We finish with France vs San Francisco, Mistral and the regulation moat, and whether Europe can produce frontier models.

Topics

  • Why post-training matters more every year

  • The modern pipeline: SFT, RL and distillation

  • Training agents and small models

  • Why models write worse than they used to

  • Safety, incentives and regulation

  • Open source and Chinese models

  • Startups vs the big labs

  • Europe, Mistral and sovereign AI

Chapters

0:00 Intro
0:57 Why post-training matters
3:44 A short history of post-training
7:20 The modern pipeline: SFT, RL and distillation
19:41 Agents, architectures and small models
27:15 Why models write worse now
31:44 Safety, incentives and regulation
38:26 Open source and Chinese models
41:39 Recursive self-improvement, startups vs big labs
46:10 World models vs LLMs
49:16 What's overrated and what's next
55:36 Europe, Mistral and sovereign AI
1:02:23 Wrap-up

Music

* "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Discussion about this episode

User's avatar

Ready for more?