StateBench: Testing how language models track state changes over long documents

StateBench is a new benchmark designed to test how language models track evolving state changes over long documents, assessing models on four difficulty levels of state-tracking complexity. It reveals that recurrent and hybrid models generalize better than transformers for complex state tracking, while leading frontier models encounter difficulties on harder levels as sequence length increases.

This Wire brief sits within Fusion42's coverage of AI Frontier Models and NLP & Speech.