World models — the AI systems trained to simulate what happens next in a scene, a robot's workspace, or another agent's next move — fail in a specific and predictable way: when they ignore what a person believes about their situation and instead extrapolate purely from observed behavior, their predictions of that person's next action turn out wrong.

According to The Decoder, new research shows this isn't a marginal accuracy gap. World models that track only actions and outcomes — without any internal representation of what a person knows, has seen, or assumes to be true — systematically mispredict what that person does next, especially once their beliefs diverge from the actual state of the world.

For a system that just needs to guess plausible next steps in a simulation, that gap can look like noise. For an AI agent that has to work alongside a human — anticipating a request, handing off a task, or reacting to what someone is about to do — it's the difference between a plan that holds up and one that quietly falls apart the moment the human's picture of reality stops matching the model's.

Behavior is not the same signal as belief

The distinction the research draws on has a long history outside AI. Developmental psychologists describe it as false-belief reasoning: people act on what they believe to be true, not on the objective state of the world. Someone who watched an object get moved while they were out of the room will still look for it in its old location — because that's what they last observed, regardless of where it actually is now. A model that only tracks the world's current state, rather than what a specific person has and hasn't observed, will predict the wrong outcome every time that gap opens up.

That gap is exactly what the research says most world models miss. They're trained to compress patterns of behavior and outcomes, which works fine as long as what people believe lines up with what's actually true. The moment it doesn't — a colleague who hasn't been told about a schedule change, a user working from outdated information, a teammate who didn't see the last update — behavior-only models have no mechanism for catching the mismatch, and their action predictions break.

Where this bites in practice

The practical exposure is highest wherever an AI system has to predict or plan around what a human will do next, rather than just respond to what a human just did. That covers a wide and growing set of products:

Most of these systems are built with imitation learning or reinforcement learning on logs of what people did, not on any explicit model of what people believed at the time they did it. That training approach inherits the blind spot the research describes by default — belief modeling has to be added deliberately, it doesn't emerge from behavior data alone.

AiiN's takeaway

For teams building AI agents that operate around humans, this is a concrete design flag rather than an abstract research curiosity. If your agent's world model — the internal representation it uses to predict what a person will do — only encodes actions and outcomes, it will fail exactly in the scenarios that matter most: when a human's knowledge is out of date, incomplete, or simply different from the agent's. Testing agents only on scenarios where belief and reality line up will hide this failure mode until it surfaces with a real user.

In our estimation, the more directly useful move for builders is to explicitly test agents against belief-divergent scenarios during evaluation — cases where the human in a scenario knows less, or something different, than the ground truth — rather than assuming that a model with strong behavior-prediction scores will generalize. Modeling what people believe, not just what they've done, looks like a prerequisite for agents that actually work alongside humans, not just around them.