The prevalent problem of misleading benchmark reporting (re: Astra)

Explain Like I'm Normal

New analysis suggests OpenAI's massive 98.6% score on ARC-AGI-3 isn't a breakthrough in model intelligence, but rather a result of giving their Astra agent specialized features like reasoning trace retention that competitors lacked. The community is calling out the 'apples-to-oranges' comparison, highlighting how easily benchmark leaderboards can be gamed through agentic scaffolding rather than core model improvements.