Interconnects

Home
Audio
Archive
Leaderboard
About
Synthetic data: Anthropic’s CAI, from fine-tuning to pretraining, OpenAI’s Superalignment, tips, types, and open examples
Synthetic data is the accelerator of the next phase of AI — what it is and what it means.
 • 
Nathan Lambert
1
The Q* hypothesis: Tree-of-thoughts reasoning, process reward models, and supercharging synthetic data
Emergency special: The information we need to understand what Q* is was right in front of us, but the memes are more fun than reality.
 • 
Nathan Lambert
5
RLHF progress: Scaling DPO to 70B, DPO vs PPO update, Tülu 2, Zephyr-β, meaningful evaluation, data contamination
Huge steps forward in confirming that RLHF can really help you on vibes based evaluation, among many other RLHF analyses.
 • 
Nathan Lambert
1
Most Popular
View all
The Q* hypothesis: Tree-of-thoughts reasoning, process reward models, and supercharging synthetic data
 • Nathan Lambert
5
Behind the curtain: what it feels like to work in AI right now (April 2023)
 • Nathan Lambert
21
Llama 2: an incredible open LLM
 • Nathan Lambert
5
The AI research job market shit show (and my experience)
 • Nathan Lambert
How RLHF actually works
 • Nathan Lambert
3
New
Top
Community
OpenAI’s shakeup and opportunity for the rest of us: openness, brain drain, and new realities
New timelines that emerge in AI and the winners and losers, regardless of the unfolding details.
 • 
Nathan Lambert
2
The interface era of AI
Modern LLMs are becoming the easiest and most efficient way to access information. This will change how we see the world.
 • 
Nathan Lambert
1
Reckoning with the Shoggoth of AI
Culture wars, open letters, new politics, developer days, and everything hidden under the smiling face of RLHF.
 • 
Nathan Lambert
Open LLM company playbook
Where does releasing model weights fit into company strategy? 3 requirements, 3 actions, and 3 benefits of being in the open LLM space.
 • 
Nathan Lambert
2
How the Foundation Model Transparency Index Distorts Transparency
A proper critique of the Foundation Model Transparency Index (FMTI). Plus some thoughts on the ecosystem implications.
 • 
Nathan Lambert
2
RLHF lit. review #1 and missing pieces in RLHF
Looking at the difference between two sets -- what rumors say industry leaders are doing with RLHF and what the literature is up to. I'm starting my new…
 • 
Nathan Lambert
1
Undoing RLHF and the brittleness of safe LLMs
Most of the arguments about "safe" releases of open LLM weights are nearly dead in the water.
 • 
Nathan Lambert
Interconnects
Interconnects
Linking important ideas of AI. The border between high-level and technical thinking. Read by leading engineers and researchers on Wednesday mornings.
Recommendations
View all 6
Artificial Fintelligence
Artificial Fintelligence
Finbarr Timbers
Deep (Learning) Focus
Deep (Learning) Focus
Cameron R. Wolfe, Ph.D.
Supervised
Supervised
Matthew Lynley
Import AI
Import AI
Jack Clark
The Palindrome
The Palindrome
Tivadar Danka
Social
Author info
Twitter
Linkedin
© 2023 Nathan Lambert
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing