Looking inside a language model

Does an AI think in circles?

We opened up a small AI model to find out whether it stores ideas as shapes. It does — there is a near-perfect circle in there. Then we discovered it barely uses it.

Model: Qwen3-1.7B · 2,240 test sentences · 29 layers · every number below was measured, not assumed

Part 1 — MotivationWhy anyone cares

There is a fight going on in AI interpretability, and it comes down to a simple question: what shape are the model's ideas?

When an AI reads a word, it turns that word into a long list of numbers — about 2,000 of them. That list is the model's "thought" at that instant. Everything the model knows about the word is in those numbers somewhere.

The question is how ideas are arranged in that space of numbers. Two camps:

Camp A — ideas are straight lines

Each concept is a direction. Want the model to be more polite? Find the "politeness direction" and push its numbers that way. This is how most AI steering works today, and it's the idea behind the popular tool called a sparse autoencoder (SAE), which tries to break a thought into a list of simple ingredients.

Camp B — ideas are curved surfaces

Concepts live on curved shapes — loops, ribbons, spirals. If that's true, pushing in a straight line shoves the model off the shape into nonsense. You should instead slide along the curve. Camp B also says SAEs "shatter" these shapes — smashing a smooth loop into unrelated fragments.

Camp B has a great demo: a tiny model that drives a car up a hill. Slide along the curve and the car moves smoothly; cut a straight line across and the car teleports and glitches. Convincing.

But that's a toy car model, not a language model. Nobody had run the equivalent test inside an actual LLM. So we did.

Part 2 — The setupWhat on earth is a "bearing"?

To test "curved shapes", you need an idea that is genuinely a circle. Our choice was compass bearings.

A bearing is just a compass direction written as a number of degrees.
0° = north. 90° = east. 180° = south. 270° = west.
And then 355° is right back next to 0° — because a compass is a circle.

That last part is the whole point. Bearings are:

Continuous — 137° is a real direction, and so is 138°. You can go as fine-grained as you like.
Circular — go far enough and you come back to where you started.

Why not days of the week?

We tried days first. It was a mistake, and it's worth explaining why.

Days of the week are a circle (Sunday loops back to Monday). But they are not continuous. There is no "Tuesday-and-a-half." So you can't ask the interesting question — "what does the model think lives between two ideas?" — because with days, there's nothing in between. Bearings fix that: 45° sits happily between 40° and 50°.

So we fed the model 72 sentences — "A bearing of 000 degrees…", "…005 degrees…", all the way round to 355 — and wrote down the ~2,000 numbers it produced each time.

Part 3 — What we didLooking for the shape

Now we have 72 lists of 2,000 numbers. Is there a circle hiding in there?

The standard way to look is a tool called PCA. Think of it as: "show me the biggest patterns in this data." It's what almost everyone reaches for first.

PCA showed us garbage. A scrambled mess with no circle at all.

But we didn't stop there — and here's the surprise. We went looking for a circle specifically, instead of asking for "the biggest patterns," and a perfect one fell out.

Three panels. Left: PCA shows a scrambled cloud of points. Middle: the angle subspace shows a clean ring with points in perfect order by bearing. Right: a shuffled-label control collapses to a blob in the centre.
Left — what PCA shows you: a scrambled mess. No circle. Middle — where the circle actually is: all 72 bearings in perfect order around a ring, with 0°, 90°, 180° and 270° exactly where a compass says they should be. Right — the sanity check: we deliberately scrambled the labels and reran the identical method. The circle vanishes. That proves the middle panel is real and not something our method invented.
4.0°
How close we can guess a bearing the model was never tested on — from its internal numbers alone. (Random guessing would be 90° off.)
9.6%
How much of the model's "thought" the circle actually takes up. The other 90% is something else entirely.

Why PCA couldn't see it

Here's the thing that makes this genuinely useful. The model is reading the text "A bearing of 137 degrees". Most of what's in its head is not the direction — it's the digits on the page: "1", "3", "7". Which characters am I looking at?

That spelling information is huge. The actual compass direction is a faint whisper underneath it — just 9.6% of the signal.

PCA asks "what's the biggest pattern here?" and the answer is "the spelling". So PCA hands you the spelling and hides the compass.

The practical takeaway

If you go hunting for these curved shapes using PCA or UMAP — the two tools everybody uses — you will miss them. They're buried under how the words are spelled. You have to know what you're looking for in order to find it.

Part 4 — The real testDoes the model actually use the circle?

Finding a circle is not the same as showing the model thinks with it. A shadow on the wall is a real shape, but the shadow isn't doing anything.

So we did surgery. We reached into the model's head and edited only the circle part, leaving everything else untouched — the equivalent of spinning the compass needle from north to east while leaving the rest of its brain alone. Then we asked the model which way it thought it was pointing.

It ignored us

We spun the needle to east. The model kept saying north.

It wasn't listening to the beautiful circle. It was reading the digits on the page.

Then we pushed harder. And harder. Turn the dial up about six times past its natural strength, and the model finally starts to listen.

Left: a bar chart of steering methods with wide overlapping error bars; the learned linear map scores best, the geodesic rotation scores poorly. Right: a line chart where the error to A rises and error to B falls as the gain increases, crossing over around 6x.
Left — the steering contest. Lower is better. Notice the error bars overlap a lot: most of the ranking here is noise, and we say so rather than pretending otherwise. Right — pushing on the circle. As we crank up the edit (left to right), the model gradually stops obeying the page and starts obeying the circle. The lines cross at about 6×. So the circle does have a voice — it's just a very quiet one.

And a boring old method beat the fancy geometry

We ran a contest. One job: "turn every bearing 90° clockwise." Four contestants:

methodhow wrong it was
A plain learned matrix (boring maths)60°winner
The theoretical best possible65°the ceiling
A simple "push in a direction" nudge84°
Sliding along the circle (the fancy geometry)100°loses
Doing nothing at all112°the floor

Sliding along the circle does beat doing nothing — that's a real effect, and it confirms the circle isn't a total illusion. But a plain, boring matrix beats it by a mile, and ties with the theoretical best possible score.

The elegant geometry is real. It is just not where the work happens.

A twist that reframes the whole debate

Camp B says "linear steering fails on circles, so you need curved geometry."

But here's the thing: a rotation is itself a piece of plain linear maths. Spinning a compass needle is not some exotic curved operation — it's a matrix, the kind of thing taught in a first-year algebra class.

So the real distinction was never "straight vs. curved". It's:

The actual lesson

❌ Adding a fixed nudge to every thought — can't rotate a circle. (If you add the same push everywhere on a ring, you shove some points forwards and others backwards.)

✅ Applying a matrix — can rotate a circle, easily.

Both are "linear". You don't need fancy curved-space mathematics to steer a circle. You just need a matrix instead of a vector.

Part 5 — The other claimDo SAEs "shatter" the shape?

Camp B's other accusation: sparse autoencoders (the popular interpretability tool) smash these smooth shapes into unrelated fragments.

We tested it the hard way. We took a thought about Thursday, ran it through the SAE and back out, then planted the result into a sentence about Monday. If the SAE preserved what "Thursday" really is, the model should now say Thursday. If the SAE destroyed it, the model should shrug and say Monday.

Bar charts showing that raw activations impose the patched concept ~100% of the time and SAE reconstructions do so 71-100% of the time, while random vectors do so near 0%.
The SAE did fine. The raw thought imposes the new concept ~100% of the time; the SAE's version of that same thought manages 71–100%. A random vector manages almost 0% — so the test really is measuring something. We found no shattering. (We used the official SAEs built by the Qwen team themselves, so nobody can say we rigged a weak one to lose.)

Part 6 — A bonus discoveryThe model's circles come from books, not from reality

We tested lots of circular ideas, not just bearings. Something odd showed up.

Six panels. Days of the week and months form clean closed rings. Moon phases and the Krebs cycle are scrambled. Integers form an open horseshoe arc.
Days of the week and months form beautiful closed rings — Monday round to Sunday and back again. But the Krebs cycle (the chemical loop that powers every cell in your body) and the phases of the moon are a scrambled mess — even though both are perfect, genuine circles in the real world. And numbers form an open arc, not a ring, which is exactly right: numbers go up forever, they don't loop.

The moon really does cycle. The Krebs cycle really does close — oxaloacetate turns back into citrate, it's a loop, that's why it's called a cycle. But the model doesn't store them as circles.

The model builds a circle when text keeps listing the thing in a loop ("Monday, Tuesday, Wednesday…" appears endlessly). It does not build one just because the thing is genuinely circular in reality.

The model's geometry mirrors its reading material, not the world. That's a weaker — and far more interesting — claim than "neural networks discover the structure of reality."

Part 7 — HonestyHow we nearly fooled ourselves, three times

We ran an adversarial reviewer against our own work at the end, whose only job was to try to destroy it. It found a lot. Six of our claims had to be retracted. Here are the three worst mistakes, because they're more instructive than the results.

Bug 1 — we spent hours analysing the word "is"

A one-character indexing slip meant we were reading the model's thoughts about the word before each concept. Every "month" we analysed was actually the word is. All twelve months came out identical. The software cheerfully produced a full page of beautiful statistics about absolutely nothing.

Bug 2 — our compass was mounted backwards

The most dangerous one. When we told the model's compass to turn right, our code turned it left. So "sliding along the circle" was steering in reverse, and scored worse than doing nothing at all.

We nearly published that as a dramatic headline: "curved-geometry steering actively harms the model!" It was a bug. Fixed, the honest answer is far more boring: the circle helps a little.

A bug that makes your results more exciting is the one you're least motivated to look for. Two of our three bugs made our findings look better than they were.

Bug 3 — a picture that lied

Our first "proof" of the circle was drawn using a method that had already been shown the answers. In that situation the method will draw you a beautiful circle even from pure random noise. Worse, the caption on our own chart said "ring holds" — hardcoded — while the test underneath it was quietly returning false.

Two panels: PCA showing a scrambled cloud, and a suspiciously perfect circle drawn by a method that had seen the labels.
The picture that lied. That right-hand circle is too perfect — because the method drawing it had already been told each point's answer. A picture like this proves nothing. Compare it with the honest, cross-checked version at the top of this page, where each point is placed by a method that has never seen that point — and where we also show what happens when you feed it scrambled labels.
Two bar charts about geodesic distance gains per concept family versus permutation nulls.
The cherry-picking trap. We tested 29 different layers of the model. If you report only your best one, you will "discover" an effect in pure noise — and we did, at first. The black ticks are what random chance can achieve. Only bars clearly above their tick mean anything.

Part 8 — What we provedThe bottom line

✅ Confirmed — the shapes are real

There is a genuine, near-perfect circle for compass directions inside this model. We can read a bearing out of the model's raw numbers to within 4 degrees, on examples it was never shown. Camp B is right that curved structure exists.

✅ Confirmed — and they're hiding from your tools

The circle is invisible to PCA and UMAP, the standard tools, because it's only 9.6% of the signal and it's buried under the spelling of the words. This is the most useful thing we found. If you go looking for these shapes the normal way, you will conclude they aren't there.

❌ Not confirmed — that the model thinks with them

Edit the circle and the model mostly ignores you. Sliding along it steers the model only weakly — while a plain, boring matrix steers it far better and matches the best score theoretically possible. The elegant geometry is real, but it is not where the computation is happening.

❌ Not confirmed — that SAEs "shatter" the shapes

We looked hard, with the model-makers' own SAEs, and found no shattering.

Does this contradict the car-on-a-hill demo?

No — and that's the neat part. That was an image model, where the thing being tracked (the car's position) is the only thing in its head. There's no spelling to distract it.

A language model reading "137 degrees" always has a loud, easy alternative available: just read the digits. So it does. The curved shape is there, sitting quietly in the background, mostly unused.

The shape is real. The shape is hidden. The shape is not the mechanism.
And the thing that would change our mind is finding a concept where the meaning — not the spelling — is the loudest thing in the model's head.