Goal Encoding in LLMs — Non-Recurrent Goal-Seeking — What does it mean?

The news. On November 12, 2026, researchers from Northeastern University, with co-authors at Parallax, the University of Copenhagen and the University of Oxford, posted Goal Encoding in LLMs. In a controlled coding setting they locate an encoding of the model's next sub-goal in a sparse set of attention heads, show that it is readable before the code starts, causes the next action, and transfers between programming languages, and then ask how a model with no recurrence keeps that goal alive for the whole sub-task. Read the paper →

Picture a line cook who does not trust their short-term memory. The cook does not hold the order in mind; every few seconds they look up at the ticket on the rail, look down at the plate, and cook whatever the ticket lists that the plate does not have yet. From the dining room it looks like the cook is pursuing a plan. Inside the kitchen, nothing about the plan is being remembered: the plan lives on the ticket, and the progress lives on the plate.

That is the paper's account of how a language model pursues a goal. In the authors' setup, the prompt lists two to four small coding sub-tasks (for example, check whether any string in a list is a palindrome, then count the duplicates), and the model writes them one after another in Python or C++, with no chain-of-thought. The ticket is the task span — the part of the prompt that spells out the sub-tasks — and the plate is the code written so far. The authors call the account non-recurrent goal-seeking (NRGS): re-read the goal, compare it with the visible progress, and pursue what remains.

Why would a model work this way instead of remembering? Because it has nowhere to remember in. A transformer generates one token at a time, and each token's computation runs bottom to top through a fixed stack of layers. Nothing computed for one token is handed directly to the next token; the next token can only read it through attention, and that read happens one layer higher. So an idea formed at layer 40 while writing token 100 can reach token 101 at layer 41 at the earliest, token 102 at layer 42, and so on, until the layers run out. What a new token can always reach is the cached keys and values of every earlier token, the prompt included, at every layer. The cook cannot carry the order through the whole shift, but the ticket stays on the rail.

Heads run side by side inside a layer; layers run one after another. A state relayed from token to token climbs this stack one layer per hop, so its lifetime is bounded by the layers left above it.

The paper gives the arithmetic for the model it studies, Qwen2.5-Coder-32B-Instruct, which has 64 layers. The next sub-goal only becomes readable in words after about layer 40. If the model relayed that encoding forward, each hop would cost one layer, so the relayed copy could last at most 64 − 40 = 24 tokens. Yet the authors routinely see the encoding persist well beyond 30 tokens, through the implementation of a sub-goal. A relayed copy would run off the top of the network at least six tokens before the job is done, so the persistence has to come from re-reading, not remembering. Re-reading has no such budget: every new token starts again at layer 0 with the task span sitting in the cache, so the goal can be rebuilt as often as it is needed.

Where does the re-reading happen? The authors find it in a sparse set of attention heads they call goal heads: 122 heads spread over layers 28–55 of the 64-layer model. They found them with activation patching: run the model on one prompt, record what each head writes into the residual stream, then run a different prompt and overwrite chosen heads with the recorded values. They searched for the heads whose values, copied from a C++ prompt that asks for a palindrome check into a Python prompt that asks for a duplicate check, make the Python run start a palindrome check. Because the two prompts use different languages, the heads that pass this test carry what to do, not how to write it.

The summed output of the goal heads at a single token is enough to say, and to cause, what the model is about to do next. Read out in words by their probe GoalScope, it names the coming operation about 60% of the time at the end of the previous line, and more than 90% of the time three tokens before the code begins. Transplanted into a C++ code stub, it makes the model write the same operation in C++ up to 96% of the time (about 87% averaged over the four token positions they test).

The swap test is where the cook metaphor earns its keep. Take two prompts that share their first two finished sub-tasks but list a different third one. At the point where the model must choose its next step, the authors swap the ticket: from that token on, only the goal heads (here a broader set of 250) read the task span's keys and values from the other prompt, while every other head, and every other part of the context, stays as it was. The model then writes the other prompt's sub-goal about 94% of the time, even across programming languages — what you expect if it keeps looking back at the ticket, and not what you expect if the goal had been fixed in memory earlier. The plate side shows up too: swapping only the goal heads' queries, which carry the "what is already done" signal, steers the choice of the next sub-goal about 57% of the time, against about 3% for the same number of random heads.

Two limits keep this from being a general law. First, the setting is deliberately narrow: short coding tasks spelled out in the prompt, one 32B code model in the main results, and no chain-of-thought. The authors find the broad picture holds in other model families up to 106B parameters, but warn that larger models with more layers could carry an encoding forward part of the way and lean less on re-reading, and they call their picture of how the next sub-goal is chosen partial. Second, the goal heads are not the only route: blinding the 122 heads leaves the model mostly able to finish the sub-goal, and only blinding the broader set of 250 breaks it, so the model has redundant paths.

What carries over to engineering practice is the direction of the result: in this setting the model's goal lives in text it can re-read, not in a hidden register. An agent harness already keeps its state as explicit data outside the model. This paper suggests one more reason to keep the task statement itself visible in the context, because that is what the model keeps looking back at.

Goes deeper in: LLM Internals → Attention → Multi-Head Attention

Related explainers

• A Circuit for In-Context Learning — gated associative lookup — another case of finding the attention heads that implement one behaviour
• White-box deception probes — reading a model's internal state to see what it is about to do, used for safety monitoring