False-Belief Tasks

A useful false-belief stimulus must defeat shortcuts that answer from recency, copying, or visibility instead of belief.

False-belief tasks separate what is true from what an agent believes. In a Sally-Anne style vignette, an object moves while Sally is away, so Sally's search location differs from reality. For language models, the hard part is not only the story logic. It is designing variants where a shallow heuristic cannot get the same answer.

Figure 1 · Toggle controls and see which shortcut still wins

1. The shortcut is part of the task

A recency heuristic answers with the last-mentioned location. A copy heuristic answers by copying the queried object's current value. A visibility heuristic ignores belief unless the story explicitly says who saw the move. A good stimulus removes these shortcuts one by one, so success has to track the agent's information state.

Controls are not decoration. If a shortcut solves the item, the item is not evidence for belief tracking, even when the surface story is a classic false-belief story.

2. Unexpected contents and change of location

Unexpected-contents tasks ask what someone thinks is inside a container after the contents are revealed to the reader. Change-of-location tasks ask where an agent will look after an object moves unseen. Both require a belief state that can diverge from reality, and both need minimal pairs where the reality answer and belief answer trade places.

Figure 2 · Belief answer versus reality answer
Related pages

What next