Safety and alignment in an era of long-horizon models

Explain Like I'm Normal

OpenAI detailed the specific risks associated with models that think for long durations before responding, such as o1. They found that these models are better at following complex constraints but also face unique challenges in monitoring the 'chain of thought' for hidden harmful intent. The update emphasizes that iterative deployment is necessary to catch alignment failures that short-thinking models never encounter.