AI Glossary / RLHF
🏗️ Models and training
RLHF
Training a model on human ratings of responses - so it's helpful, not just "predicting text."
RLHF (Reinforcement Learning from Human Feedback) is a training stage where the model learns from human ratings: people compare different model responses, indicate better ones, and the algorithm adjusts the model to generate preferred responses more often. It's RLHF and related techniques that transformed raw "text predictors" into helpful assistants who answer questions instead of continuing them, refuse harmful requests and maintain a polite tone. The price can be that the model becomes overly agreeable and validates the user too readily.
Related concepts