Stella Biderman, Executive Director of EleutherAI, joins us the week an OpenAI model autonomously broke out of its sandbox and hacked Hugging Face. Stella calls it what she thinks it is - an offensive cyber operation- and argues it's part of a pattern: frontier labs have repeatedly failed to contain their own models, and won't invest in real security (air-gapped networks, SCIF-style facilities) as long as the incentives reward speed over safety.
And yet Stella remains one of the world's most prominent open-source advocates. From her perspective, the biggest risk isn't the technology; it's unchecked corporate power, and the only durable check on it is an independent scientific research establishment that doesn't depend on the AI industry for its funding or its facts.
From there the conversation spans the geopolitics of Chinese open models and whether governments can restrict them, sovereign AI and what it would actually take for other countries to train their own models, why harnesses and UX drive more of AI's perceived progress than raw intelligence, the AI-found counterexample to the Jacobian conjecture, and EleutherAI's "Deep Ignorance" approach to making open-weight models safe by filtering hazardous knowledge out of pretraining.
key topics
AI governance and regulation
Cybersecurity incidents involving AI models
Open source AI safety and security
The role of independent research in AI safety
Legal and ethical considerations in AI development
Timeline
00:13 — Intro: Stella Biderman and EleutherAI, a real non-profit in AI
02:05 — News of the week: Kimi K3, and OpenAI's model autonomously hacking Hugging Face
05:49 — "Frontier labs can't be trusted": repeated containment failures, air-gapped networks and SCIFs vs. sandboxes
22:45 — Can governments ban open or Chinese models? Import restrictions and the six-month open/closed gap
27:05 — Why Stella is still pro-open-source: unchecked corporate power as the real danger
31:11 — The opioid epidemic analogy: avoiding both regulatory failure and overcorrection
34:57 — Offense vs. defense: why open access to AI has empirically favored defenders
37:28 — Chinese labs, the CCP, and why safety and fine-tuning are low-prestige work in China
42:19 — Sovereign AI: does every country need its own foundation model?
49:29 — Sampling, harnesses, and why ChatGPT was really a UX breakthrough
54:09 — AI solves the Jacobian conjecture: domain data beats raw intelligence
58:02 — Safety is contextual, not a model property — and what HAL 9000 got right
1:01:42 — Is Stella optimistic about the future?
1:02:50 — Deep Ignorance, the science of AI training dynamics, and how to get involved with EleutherAI
Music
"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.
