AI for Science and the Thermodynamics of Generative AI - with Max Welling (UvA, CuspAI)
In this episode, we sit with Max Welling, Professor of Machine Learning at the University of Amsterdam, co-founder and CTO of CuspAI, and a foundational figure behind variational autoencoders (VAEs), equivariant networks, and Bayesian deep learning. We talk about AI for science, the physics underneath generative models, and what's still missing on the road to real intelligence.
Max starts with what impresses him and what worries him about the LLM era, then makes the case that the next leaps will come from physical AI and from science itself. We dig into how machine learning actually works in the lab, world models and whether priors like geometry and symmetry should be built in or simply learned, and whether transformers will still rule a decade from now. At the end, we talk about CuspAI's climate mission, AI risk and regulation, Max’s new book, and where neuroscience might inspire the next wave of ML.
Timeline
- 00:00 — Intro
- 00:47 — Are we happy with the LLM era?
- 03:14 — Embodiment and physical AI
- 08:05 — Does "AGI" even matter as a term?
- 11:34 — Verifiers, RL, and why math/coding are tractable
- 13:17 — What actually shifted to make materials discovery work
- 14:42 — From molecules to biology and wet labs
- 16:26 — Working with real labs: timescales, friction, and the "Mira" agent
- 20:29 — Balancing simulators vs. experiments: the exploration–exploitation trade-off
- 23:44 — Active learning for experimental design
- 24:23 — Why active learning hasn't been central to LLMs
- 25:24 — A general loop for ML-for-science across domains
- 27:10 — Foundation models for chemistry: a "mother ship" plus a zoo of fine-tuned models
- 30:04 — Quantum mechanics, interpretation, and AI as a creative theorist
- 31:54 — World models and Yann LeCun's view; priors vs. learning
- 34:57 — Should world knowledge be explicit? (responding to Stefano Ermon)
- 36:41 — Vision: equivariance vs. transformers, and the role of optimization
- 40:32 — Best model for molecular properties in 10 years? Will transformers survive?
- 43:16 — CuspAI's climate focus and what motivated it
- 47:10 — One platform for every material class — what transfers and what doesn't
- 48:42 — Where does the risk of human extinction really come from?
- 51:06 — The "pause AI" debate and the arms-race reality
- 52:40 — Regulating powerful models: government vs. self-regulation
- 55:16 — Who should design AI regulation?
- 56:29 — The new book
- 1:00:31 — Compression, the information bottleneck, and renormalization
- 1:03:30 — The role of foundational principles in modern AI
- 1:04:06 — Waves in computing, the brain, and the next wave of innovation
- 1:07:11 — Neuroscience and ML: are we in a better position now?
- 1:09:17 — Conferences, the ICLR keynote, and finding the right people
Music:
- "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
- "Palms Down" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
- Changes: trimmed
About: The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.