Benchmarking Language Models for Statistical Problem Formulation

Explain Like I'm Normal

Researchers have introduced StatFormBench to evaluate how well models can translate messy, informal user goals into structured statistical problems. While LLMs are great at solving equations, this benchmark tests their 'upstream' ability to identify relevant variables and choose the correct analysis type from raw data descriptions. Results show that even top-tier models struggle with formulating the right problem before the actual computation begins.