Sequence Autopsy

The model demonstrates a mechanistic "Induction Head" circuit rather than reasoning. When predicting the next token in a repetitive sequence like "Canvas Context," the attention head looks back exactly two steps to the previous identical token. It then copies the subsequent token from that history, proving the architecture is executing a literal, dumb copy-paste operation.

Semantic satiation is mathematically quantifiable in the attention matrix. As the word "Canvas" repeats, the distinct vector signature dissolves into high-entropy noise. The model loses the ability to distinguish between specific instances of the token, resulting in a "smear" across the context window. This degradation confirms that excessive over-saturation forces the neural net into hallucination.

When the context becomes chaotic, the model utilizes an "Attention Sink" mechanism, dumping mass onto the <bos> token. This is a structural necessity of the Softmax function.

The Flicker: If you see a row header blinking, it means the model has lost its Confidence Anchor. It can neither find a relevant past token nor safely dump its attention into <bos>. This "Entropy Flicker" marks the exact transition into pure hallucination.