About The Information Bottleneck
Long conversations with the people building the frontier of AI, and two researchers’ reading on the ideas that survive the bottleneck.
The Information Bottleneck is created by Ravid Shwartz-Ziv and Allen Roush, two AI researchers who wanted a place for the kind of conversations and writing they wished existed: technical, unhurried, and free of hype.
Ravid’s work is about how neural networks form, compress, and use their internal representations. He came into the field through information theory (aka the information bottleneck), working with Naftali Tishby, and that lens still shapes how he reads almost every machine learning result. Allen works on large language models and natural language processing, with particular depth in argument mining, automatic summarization, and model explainability, and a long-running interest in the decoding and sampling methods that shape how models actually generate text.
The podcast is long, unhurried talks with the researchers and engineers building modern AI. The people behind the papers, walking through what they actually did and why, rather than a polished press tour.
The name is the thesis. Far more is published in AI than anyone can absorb, and most of it is noise. The work that lasts is compression, or in other words, finding the few ideas that survive contact with reality and explaining them clearly. That’s what we’re trying to do here: pass the signal that makes it through the bottleneck, and leave the rest behind.
Subscribing
New issues go out with each episode and as new essays land. It’s free — the best way to support it is to subscribe and pass it to someone who’d find it useful.
Subscribe to get full access to the newsletter and publication archives.
Ravid and Allen
