I just discovered your blog! This was an incredible read, would love to hear your take on current developments with instruction fine tuning and possibly a comparison between it's capabilities when pitted again RLHF.
Great piece, Nathan — really enjoyed reading the full article, especially the clear breakdown on data quality, preference collection difficulties, and the nuances of what RLHF actually achieves.
Do you think applying RL inside an agent harness (e.g. Pi Agent with memory) could meaningfully improve outcomes compared to standard RLHF on chat models? Especially for longer, multi-step tasks.
I just discovered your blog! This was an incredible read, would love to hear your take on current developments with instruction fine tuning and possibly a comparison between it's capabilities when pitted again RLHF.
I should write more about this soon.
"What is the killer use case for RLHF" is the most common question RLHF people get.
I think I comment on some of this here: https://www.youtube.com/watch?v=u8xxEkH3a5g
Great piece, Nathan — really enjoyed reading the full article, especially the clear breakdown on data quality, preference collection difficulties, and the nuances of what RLHF actually achieves.
Do you think applying RL inside an agent harness (e.g. Pi Agent with memory) could meaningfully improve outcomes compared to standard RLHF on chat models? Especially for longer, multi-step tasks.