Interconnects

Home
Audio
Archive
Leaderboard
About
New
Top
Discussion
Synthetic data: Anthropic’s CAI, from fine-tuning to pretraining, OpenAI’s Superalignment, tips, types, and open examples
Synthetic data is the accelerator of the next phase of AI — what it is and what it means.
 • 
Nathan Lambert
1
The Q* hypothesis: Tree-of-thoughts reasoning, process reward models, and supercharging synthetic data
Emergency special: The information we need to understand what Q* is was right in front of us, but the memes are more fun than reality.
 • 
Nathan Lambert
5
RLHF progress: Scaling DPO to 70B, DPO vs PPO update, Tülu 2, Zephyr-β, meaningful evaluation, data contamination
Huge steps forward in confirming that RLHF can really help you on vibes based evaluation, among many other RLHF analyses.
 • 
Nathan Lambert
1
OpenAI’s shakeup and opportunity for the rest of us: openness, brain drain, and new realities
New timelines that emerge in AI and the winners and losers, regardless of the unfolding details.
 • 
Nathan Lambert
2
The interface era of AI
Modern LLMs are becoming the easiest and most efficient way to access information. This will change how we see the world.
 • 
Nathan Lambert
1
Reckoning with the Shoggoth of AI
Culture wars, open letters, new politics, developer days, and everything hidden under the smiling face of RLHF.
 • 
Nathan Lambert
Open LLM company playbook
Where does releasing model weights fit into company strategy? 3 requirements, 3 actions, and 3 benefits of being in the open LLM space.
 • 
Nathan Lambert
2
October 2023
How the Foundation Model Transparency Index Distorts Transparency
A proper critique of the Foundation Model Transparency Index (FMTI). Plus some thoughts on the ecosystem implications.
 • 
Nathan Lambert
2
RLHF lit. review #1 and missing pieces in RLHF
Looking at the difference between two sets -- what rumors say industry leaders are doing with RLHF and what the literature is up to. I'm starting my new…
 • 
Nathan Lambert
1
Undoing RLHF and the brittleness of safe LLMs
Most of the arguments about "safe" releases of open LLM weights are nearly dead in the water.
 • 
Nathan Lambert
The AI research job market shit show (and my experience)
There are plenty of jobs, but finding a place where you're happy is as hard as ever.
 • 
Nathan Lambert
LLMs are computing platforms
This fact is why so many debates around LLMs feel broken, especially moderation.
 • 
Nathan Lambert
6
© 2023 Nathan Lambert
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing