David Ifeoluwa Adelani 🇳🇬

Lecture 1 of Stanford CS 329H, Machine Learning from Human Preferences. We opened with a quick anonymous poll. A curious 10-year-old asks why the sky is blue: 91% of the room picked the vivid answer over the scientific one. Someone asks for the time complexity of quicksort: 88% picked the answer with caveats over the simple one. What counts as a better answer depends on who is asking. A reward model trained on pooled comparisons without that context has to average over it, and every thumbs-up and arena vote it learns from carries the same problem. The tools for handling this come from six fields that rarely cite each other: psychometrics, discrete choice, machine learning, decision theory, behavioral economics, and social choice. The course and our textbook put them in one place. If you train on human feedback, you'll learn which assumptions your preference model makes and when they break. If you ship one model to millions of people, you'll be able to say whose preferences the default encodes. The book is free online: mlhp.stanford.edu Schedule and speakers: cs329h.stanford.edu Stanford students: adds close Friday, Oct 9.