Integrating some non computing science into reinforcement learning from human feedback can give us the models we want.