Rater State Bias in RLHF Preference Data: An Audit Framework

Explain Like I'm Normal

Researchers found that the emotional state and stress levels of human raters significantly contaminate the RLHF data used to train models. This 'state shift' means AI models aren't just learning what a good answer looks like, but are actually absorbing the burnout and distress of their annotators. This creates a hidden, structured bias that traditional denoising methods currently fail to catch.