Reasoning off the page: what looped transformers actually change (and what they don't)

Are we going back to black box AI? Probably not — and definitely not for the reasons people are giving.

There’s a real panic building around GPT-6 Astra and the recurrent-depth architecture it reportedly uses. The rough shape: reasoning is moving from visible chain-of-thought tokens into hidden state, we can no longer read what the model is thinking, alignment researchers are worried, and interpretability just took a step back.

I spent a week reading the papers, and I don’t think the alarm is warranted — yet. The concern rests on assumptions the empirical evidence doesn’t currently support. The trajectory is worth watching. But the “we’ve regressed” framing is skipping several steps.

Here’s what actually changed, what the papers actually say, and where the concern is genuine versus where it’s a vibe.

The two scaling axes we knew

For most of the LLM era we’ve had two knobs to turn.

The first is width: bigger models, more parameters, trained on more data. The classic Kaplan / Chinchilla story — pretraining compute buys capability, permanently baked into the weights. Once a model ships, its width is fixed.

The second, newer knob is length: more tokens of reasoning at inference. This is the o1 / o3 / R1 / Claude extended thinking / Gemini Deep Think family. The model writes out intermediate steps before answering. Real compute cost — a forward pass per token — but the reasoning is externalized as text you can read.

Length-scaling has been the defining story of 2024–2025. It’s how smaller models beat much bigger ones on math and code, and it’s why “test-time compute” became a phrase you can’t escape in AI podcasts.

Both axes are visible. Width shows up as VRAM. Length shows up as tokens in your API response. You can budget for them.

The third axis: depth-in-time

Recurrent-depth transformers add a third knob, and it’s invisible from the outside.

The mechanism is simpler than the launch materials imply. A standard transformer has L stacked layers. A recurrent-depth transformer takes that same block of L layers and applies it R times in a loop on the same hidden state before emitting the next token. Effective depth becomes L × R, but the parameter count is still just L. The extra depth is temporal, not spatial.

You keep the same weights. You just run them more times.

This isn’t a new idea. It traces back to the Universal Transformer (Dehghani et al., 2018) — weight-tied layers with adaptive halting borrowed from Adaptive Computation Time. It sat mostly dormant for six years because unrolled depth was cheaper to train than looped depth, and pretraining scaling was doing all the heavy lifting anyway.

What changed in 2025–2026 was a practical realization: test-time compute is where the marginal capability lives now, and looped depth is a very different way to spend that compute than emitting tokens.

What we know about Astra

GPT-6 Astra, released by OpenAI in September 2026, is why this architecture is suddenly a mainstream conversation. According to reporting by The Information, Astra uses a “constrained form of recurrent depth” — the same layers applied more than once before producing each token.

Worth naming clearly: Astra is not the first model to use this design. The University of Maryland’s Huginn (a 3.5B proof-of-concept from Tom Goldstein’s group) came earlier. Nanbeige, an open-weight model that reuses a 22-layer stack twice, shipped a production version before Astra. Zhipu AI announced their own Loop Transformer plans on August 31, 2026. Astra is the first US frontier-lab deployment, and it’s the release that pushed recurrent depth into mainstream discussion — but the technique itself has been in the water for a while.

Important caveat on the details: OpenAI’s launch materials don’t name recurrent depth by that phrase. The architectural detail comes from The Information’s reporting, not from an OpenAI paper or system card. The reporting and community analysis both suggest Astra’s effective serial computation depth is constrained to stay comparable to GPT-4, and that chain-of-thought monitoring remains a primary safety mechanism — but I couldn’t find an OpenAI statement that says this in exactly those terms. So the reported architecture and the implied behavioral change are worth holding at slight arm’s length until there’s fuller disclosure.

That said, the behavioral signature is exactly what a recurrent-depth model is expected to look like: dense reasoning at low visible-token cost, strong performance on computer use and multi-step workflows, less verbose thinking traces than the R1 generation. Whether or not Astra is exactly this architecture, the design pattern is real and it’s being adopted at scale.

What could change in behavior

Here’s where it gets interesting for practitioners. Recurrent depth isn’t just “faster CoT” — it could change the shape of how the model behaves. Emphasis on could: what follows is what the architecture makes possible in principle. Whether current implementations actually exhibit these behaviors is a separate empirical question I’ll come back to.

Where the thinking could live. In a CoT reasoning model, intermediate computation is literally in the token stream. You can read it, log it, and catch the moment the model talked itself into the wrong answer. In a looped model, intermediate computation could happen in the residual stream between token emissions — real math (attention, feed-forward, layer norm, iterated) that never surfaces as text. From the outside you’d see input, then output. The middle would be opaque.

That’s the possibility. In practice, current looped models still emit visible reasoning tokens — they just do more hidden-state work between token emissions. Whether the reasoning actually migrates into that hidden state is the empirical question I’ll get to in a couple of sections.

How compute scales at inference. CoT models spend compute proportional to reasoning length. If the answer needs 4,000 tokens of scratch work, that’s 4,000 forward passes and 4,000 tokens of KV cache. A looped model spends compute proportional to R iterations per output token — R = 4 with a 100-token answer means 400 forward passes and 100 tokens of KV cache. Same total FLOPs, wildly different latency and memory profiles.

Adaptive compute becomes content-dependent, not length-dependent. This is the subtle one. A well-trained looped model can halt early on easy tokens and loop harder on hard ones, so per-token latency becomes a function of how hard the token is, not how much reasoning has piled up. CoT models get slower as they think longer. Looped models get slower on harder problems specifically.

The failure mode could invert. CoT models fail loudly — you get 4,000 tokens of confidently wrong reasoning you can post-mortem. A looped model that leaned heavily on hidden-state reasoning would fail quietly: the wrong answer comes out, no trace to inspect. Whether current looped models actually fail this way in practice is unclear; the 2507.02199 result suggests today’s implementations still lean on visible tokens more than the theoretical possibility implies.

Best-fit tasks might shift. CoT excels on tasks with discrete reasoning steps that benefit from being externalized — mathematical proofs, symbolic manipulation, planning that benefits from backtracking. Looped depth could excel on tasks that need the same operation applied repeatedly to refine an internal state — algorithmic code, structured browsing, dense agentic sequences. Astra’s release emphasizes exactly those workloads, which is consistent with the architecture story, but consistent isn’t the same as caused — Astra is also a bigger, better-trained model than what came before.

The theoretical case for the concern

There’s a paper that’s fueling a lot of the current worry: Reasoning with Latent Thoughts: On the Power of Looped Transformers (arXiv 2502.17416, Saunshi et al., ICLR 2025). It proves that a k-layer looped transformer applied T times can simulate T steps of chain-of-thought — matching CoT’s reasoning capability while doing the reasoning in hidden state instead of emitted tokens. In plain terms: the loops can do the same work CoT tokens would have done, just invisibly.

This is the theoretical case for the interpretability concern. If looped models could do CoT-equivalent reasoning in hidden state, and if frontier labs deploy them at scale, then chain-of-thought monitoring — one of alignment’s most useful primitives — degrades.

A companion result, Chain-of-Thought and Compressed Looped Transformers: A Memory-Budget Separation, shows the reverse direction: under memory constraints, CoT is strictly more expressive than a memory-bounded looped model. So the two aren’t equivalent — they occupy different points on the compute-versus-memory frontier, and any real-world model will trade off between them.

Practical implication: frontier labs will deploy both, and modern reasoning models are already hybrid. CoT for tasks that benefit from externalized backtracking, looped depth for tasks that benefit from deep-per-token refinement.

What the empirical evidence actually shows

Here’s the part getting less airtime in the current discussion. The theoretical result above shows what looped transformers can do. It doesn’t show what current ones actually do.

A recent paper — Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer (arXiv 2507.02199) — probes Huginn, the University of Maryland recurrent-depth model, specifically to test whether it does the latent reasoning we’ve been assuming looped models do. The findings are the kind of thing that should update your priors:

• Limited evidence of interpretable latent CoT structure. Tracking rank trajectories through the loops, they can’t find the kind of stepwise reasoning we see when we read a CoT trace.
• Increasing recurrence depth yields marginal gains — and falls well short of models that explicitly externalize reasoning steps. More loops help a little; more CoT tokens help a lot.

Sebastian Raschka, whose overview is the clearest technical write-up on Astra I’ve read, lands in a similar place from a different direction. His view, roughly: bigger models with more layers already do more computation internally without producing shorter CoT traces. Looped depth is more of the same, not a categorical shift. His words: “I don’t think these looped transformer ideas are obscuring the reasoning chains. They’re making them maybe more compact.”

You can hold two things at once here. The theoretical result (Saunshi et al.) shows the concern is possible. The empirical result (2507.02199) shows it isn’t yet actual in the models we can inspect. Astra is closed, so we can’t check it directly — but the community’s best empirical proxy is the smaller open recurrent-depth models, and they don’t behave the way the alarm implies.

Where this leaves me: the “reasoning is moving off the page” framing overstates what’s actually happening in shipped systems today. The trajectory concern is real. The current-state concern is running ahead of the evidence.

What this could mean if you’re building on the API

Three concrete things worth thinking about, filed correctly as potential consequences rather than observed ones. Given the empirical picture, treat these as risks to monitor, not certainties to plan around.

Reasoning traces could get shorter. If looped architectures do end up doing meaningful hidden-state reasoning, the visible trace stops being a faithful record of what the model actually did. You’d want to lean harder on outputs, tool calls, and structured intermediate outputs you asked for — not whatever “reasoning” field the API returns. Worth designing for even before you’re sure it’s needed.

Latency profiles could look weird. Content-dependent per-token latency (a real architectural feature of adaptive-halting looped models) would break streaming assumptions. UIs expecting uniform token cadence would jitter on hard prompts. Rate limiters keyed to token count would misprice hard queries. If you’re serving one of these models, decouple compute cost accounting from output token count in your infrastructure.

Determinism could get harder. Looped models with adaptive halting are more sensitive to noise in early iterations — small differences early lead to different R decisions later. Temperature 0 might not buy you as much reproducibility as it used to. If you rely on determinism for eval, plan for more variance and add explicit consistency checks.

Where the concern is genuinely worth holding

I don’t want to leave the interpretability worry entirely on the shelf, because the trajectory version of it is real.

Chain-of-thought monitoring — reading the model’s reasoning and using it as an oversight signal — has been one of the most useful primitives in alignment for the last two years. It’s how researchers catch models that are lying, scheming, or reasoning about their evaluation context. It works because the reasoning has to happen somewhere, and CoT models put it in tokens we can read.

Looped depth is an architecture that could move some of that reasoning into activations we can’t read. Not by design, but as a consequence of what the architecture makes possible. Today’s shipped R values, where we can see them, appear small — Nanbeige uses R=2, Astra’s is reportedly constrained. Research models like Huginn train with larger R (up to 32, extended to 64 at test-time), and even there the empirical evidence for meaningful latent reasoning is limited (2507.02199). But the theoretical result (Saunshi et al.) shows what could happen if R scales in shipped systems, and Geoffrey Irving’s argument (cited in the LessWrong analysis) is that circuit-depth bounds only give you interpretability protection when they’re very low — hundreds of layers isn’t “shallow enough to inspect.”

Which means the interpretability community’s watchfulness is legitimate — they’re tracking the derivative, not the current value. Climate-scientist logic: today’s CO₂ isn’t the crisis, but the rate of change matters. Watching R with the same posture is reasonable.

Where I’d push back on the current discourse: framing today’s models as already-opaque isn’t supported by the evidence. Warning that they could become opaque, and making the case for empirical scrutiny before R scales, is the intellectually honest version of the concern.

What would actually upgrade the concern

Since I’m arguing the current alarm is premature, the honest thing to do is name the signals that would change my mind. If any of these land through the rest of 2026, the “reasoning is going opaque” framing gets a lot more weight:

• OpenAI disclosing Astra’s architecture in a real system card. Right now everything is journalist-sourced. If a formal disclosure shows R higher than a couple of iterations, or shows that effective depth is a meaningful multiple of GPT-4’s, the current-value concern moves from theoretical to real.
• R becoming tunable in the API. If OpenAI exposes iteration count as a parameter (the way reasoning_effort is exposed for the o-series), that’s a signal about how they think about the tradeoff — and a lever users could push into unmonitorable territory.
• A frontier-scale open looped model. Nanbeige and Huginn are the empirical proxies we have today, and 2507.02199 already shows they don’t do much latent CoT. If Meta, DeepSeek, or Qwen ship a much larger looped model and the same probing reveals meaningful hidden reasoning, the picture flips.
• New empirical probes. 2507.02199 is one paper. Better methods for detecting latent-space reasoning could confirm or refute what it found. This is the fastest way the ground could shift.
• World-model convergence. JEPA, Genie, and the video-native architectures already have iterative refinement baked in. If language and world modeling converge on a shared looped-latent core, the compute-you-can’t-see question becomes cross-cutting.

The takeaway

Recurrent depth is real, worth understanding, and the third scaling axis alongside width and length is a genuinely new thing to think about. What it isn’t, right now, is a completed regression to black-box AI. The panic is running ahead of the evidence: theoretically, looped models could push reasoning into hidden state. Empirically, the ones we can inspect don’t yet.

Width was easy to reason about. Length was harder but still legible. Depth-in-time might turn into compute you can’t see — but that’s a “might” today, not a “has.” Watch the signals above. If they start landing, the concern earns its urgency. Until then, the intellectually honest version is: interesting architecture, real potential, empirical picture still catching up. Worth watching without panicking.

Further reading

← All posts