The Intrusion Pathway

Abstract

Language models lose accuracy when a prompt contains non-evidential context: irrelevant statements from unrelated domains, or random off-distribution tokens, with no competing answer and no semantic overlap with the query. Such noise pervades retrieval-augmented, tool-using, and long-context deployments, yet the failure has lacked a mechanistic account. We trace the corruption to the Intrusion Pathway, which starts with MLPs enriching both query-subject and irrelevant-token representations, parroting heads carry prefix-token content to the prediction position, and relational readers largely avoid unrelated sources. Nevertheless, preserved subject enrichment and selective reading do not prevent intrusion in the broader computation. We then localize part of this interference to a sparse set of attention heads, which we call intrusion heads. Their prefix-derived writes promote distractor-related content, while their reduced support for the correct answer arises primarily from weaker non-prefix contributions rather than direct suppression by the prefix. The same heads transfer across tasks and distractor types and are largely weight-disjoint from the heads that integrate relevant contextual evidence, enabling a class-conditional intervention. Replacing the corrupted last-token residual with its clean counterpart, an oracle probe that requires a matched clean forward pass, recovers 97.6% on average of the distractor-induced drop across four downstream tasks. A deployable mask level knockout of about ten heads, roughly 1% of the model's attention heads, recovers 57% at bounded cost to helpful context.

THE MODEL, LIVE ↓

00 · ORIENTATION

Llama-3.1-8B, cut open

Eight slabs, left to right, stand for eight bands of four layers. Rows are token positions. The heavy row at the bottom of each slab is END, where the next token is read out. The heads that write into END sit on the diagonal rising from it.

Moving dots are attention. They leave a source position, enter a head, and the head's write joins END. Dots moving right along a row are the residual stream.

Intrusion head Dual-use head (intrusion + retrieval) Retrieval head Parroting head Relational reader MLP enrichment

WATCH The color of the END row, and the curve under the slabs. Cyan is answer-supporting content. Orange is content that came from the prefix.

01 · CONTEXT REGIMES

Same query, six prefixes

The query never changes. Only the prefix does. Two of these supply no evidence and propose no capital, yet still lower the Berlin logit. Gold-logit shifts from Table 1:

WATCH The next-token distribution under the model. The long irrelevant prefix costs twice the short one and never mentions a city.

02 · MLP ENRICHMENT · §3

Germany is still enriched

Subject-position MLP writes make Germany's attributes available at the Germany position. Under the irrelevant prefix that signal is effectively unchanged (App. B.1.1).

The MLPs enrich the irrelevant tokens too. “Computers” picks up its own associations (B.1.2). The two tokens differ in relevance, not in whether they carry associated information.

WATCH The diamonds on the two leftmost slabs. Germany's looks the same in both runs; the prefix diamonds are new. The lost support for Berlin has to happen further along.

03 · PARROTING · §2–3

Prefix tokens get copied to END

Parroting heads carry prefix-token identity into END without a matching induction cue. On a multi-token prefix their attention spreads over several content words, and the write is approximately an attention-weighted mixture of those tokens' unembedding directions.

ohcontent = ch · Σp∈P αp utp + rhEq. 2

Swapping the value at one source position with attention held fixed shifts these heads' writes toward the swapped token (B.2.1). The effect is in OV, not QK routing.

WATCH The white-dot heads. Orange arcs fan out over Computers, mathematical and simulations, then leave as one orange write into END.

04 · RELATIONAL READERS · §3

The readers look away from the filler

Mid-layer readers, selected on golden context, attend to candidates of the expected answer type and assign little attention to unrelated filler (B.4).

WATCH Under the irrelevant prefix the hollow-circle heads barely touch it. Under Golden or Counterfactual they jump to Berlin or Munich. Selective reading does not clean END, which is already tinted by earlier layers.

05 · MIDDLE-BLOCK INTERVENTION · §3

Effective, not selective

Replace every middle-block head's END write with its context-free counterpart while the prefix stays in the input. This restores most of the gold-logit loss. Masking the same heads' END→prefix reads barely helps.

WATCH Switch to Golden with the restore on: most of the evidence benefit goes with it, and it needs a clean donor run.

06 · INTRUSION HEADS · §4

A sparse, late population

Score each head by how much its direct gold contribution falls, weighted by its attention to the prefix.

sint(h) = 1/N Σi ahirr · max(0, DLAhnctx(g) − DLAhirr(g))Eq. 3

In Llama-3.1-8B the population sits mostly in late layers, and its attention to the prefix rises when one is present (Table C.1). Distractor writing is not part of the score, and it still rises.

WATCH Toggle Bare and Irrelevant. The orange squares move part of their attention from BOS and the query onto the prefix, and their writes turn orange.

07 · P / U SPLIT AND MASKING · §4

What the mask does inside one head

Split each write into o_h = P_h + U_h. P comes from prefix positions and carries most of the extra distractor content, with near-zero mean gold attribution. The gold drop sits in a weaker U. Masking END→prefix attention removes P, and softmax redistribution scales U up.

ohmask − oh = −Ph + ah/(1−ah) · UhEq. 4

{{ selId }}a = {{ selA }}
unmasked writeP
U
masked writeU {{ selGain }}

Click any orange square in the model to load it here. Fig. 5's example head has a = 0.47, so U is scaled ×1.90.

08 · INTRUSION VS RETRIEVAL · §4

Two populations, mostly apart

Retrieval heads read the answer span in golden context and write toward it. Ranked independently, the top heads of the two populations differ, with a small overlap (Fig. 6). That separation is what makes a class-conditional mask possible.

WATCH Under Golden, masking the intrusion heads keeps most of the benefit. Masking the retrieval heads removes it.

09 · ORACLE PATCH AND λ · §4–5

The oracle and the deployable mask

The oracle swaps the END residual at the intrusion-band layer for its clean counterpart, so it needs a matched clean pass. The mask needs nothing at inference.

The λ slider under the model interpolates the intrusion heads' writes toward their context-free values: o(λ) = o^irr + λ(o^nctx − o^irr).

WATCH Patch on: END turns cyan from the patched slab onward. Patch off, intrusion masked: partial recovery from about 1% of heads.

10 · FREE PLAY

Every control is live

Click heads in the model to mask them one at a time. Combine regimes, masks, λ and the oracle patch.

      
        {{ tk.text }}
      
      {{ ansTok }}

supports Berlin
from the prefix
blocked by mask

Llama-3.1-8B · 48 of 1,024 heads
{{ modeLine }}

{{ tip.id }} · {{ tip.type }}
prefix attn {{ tip.a }}
{{ tip.hint }}

next token · log p
          — now- - bare
          
            {{ d.lab }}

          
          λ
          
          {{ lambdaLabel }}