Repo: https://github.com/orvelte/j-lens-sprint
References:
Gurnee, W., Sofroniew, N., Pearce, A., Piotrowski, M., Kauvar, I., Chen, R., Soligo, A., Bogdan, P., Ong, E., Wang, R., Thompson, B., Abrahams, D., Kantamneni, S., Ameisen, E., Batson, J., & Lindsey, J. (2026). Verbalizable representations form a global workspace in language models (arXiv:2607.15495). arXiv. https://doi.org/10.48550/arXiv.2607.15495
External commentary: https://www-cdn.anthropic.com/files/4zrzovbb/website/cc4be2488d65e54a6ed06492f8968398ddc18ebe.pdf
Review by Neel Nanda: https://www.lesswrong.com/posts/zFJ3ZdQwrTWE9jT5S/a-review-of-anthropic-s-global-workspace-paper
I replicated core J-lens findings on the open-weight Qwen3.6-27B and tentatively pushed on the unresolved question: when does the J-space content actually tell us what the model will do? And once we know this, could we use the J-lens as a safety tool?
When a language model processes text, it accumulates a vast number of internal computations on its way to predicting the next token. Only a small fraction of this mysterious matter is legible to us, but in “Verbalizable representations form a global workspace in language models”, the researchers pull back the curtain a bit more. They present the Jacobian Lens (J-lens) as a tool used to reveal a small, privileged set of internal representations. The J-lens computes the Jacobian of the model's final-layer state with respect to an intermediate state, averages it across multiple contexts, and multiplies by the model's output weights to express the result in vocabulary coordinates (J_ℓ = average, over ~1000 pretraining-like prompts and over source/future token positions, of ∂h_final,t′/∂h_ℓ,t, composed with the unembedding), yielding per-layer and token-indexed directions (J-lens vectors) spanning “the J-space". In broad terms, the J-lens answers “if the model were thinking about a given concept infinitesimally more at layer ℓ, position t, what would it become more likely to say later?” and the J-space contains “which words is this internal state closest to saying”.
The original paper found many intriguing cases of the J-space carrying “thoughts” that never surfaced in the model’s output. Given fabricated search results, the workspace held tokens like fake, fraud, and poison. In a blackmail scenario, the model represented deceptive intent in the J-space while no such intent was visible on the surface. During reflective tasks, the J-space contained the model's assessments of its own honesty, including recognition that its ethics were being tested. These results pose the J-space as a covert deliberation channel, where a model’s ****assessment of a situation can privately diverge from its stated one, so being able to read it has obvious auditing appeal. As models grow more capable of heaving evermore computation into a single forward pass, their incentive to externalize reasoning into the CoT weakens and complementing CoT monitoring with tools that read latent reasoning is of our best interest. However, like other interp methods, we have to resist muddling our minds with “what a model knows”, and now “what is it thinking” with “what it will do”, and the authors establish that the translation from presence in the J-space to model intent or behavior is not yet clear. My project focuses on replicating and furthering investigations into scenarios where J-space contents don’t align with model behavior, in hopes that these scenarios can help inform the use of the J-lens a reliable safety tool - (and then one parallel to Global Workspace Theory in neuroscience because I couldn’t miss out on all the fun).
All experiments use the open-weight Qwen3.6-27B (64 layers, model dimension 5120, run in bfloat16 on a single A100 SXM100GB GPU) and all results/claims are specific to this model, not language models in general. I used a pre-fitted J-lens (https://huggingface.co/neuronpedia/jacobian-lens), fit on 1000 WikiText prompts, read out as softmax(unembed(final_norm(J_ℓ · h))) at each source layer. The lens provides one matrix per layer for layers 0–63; the final layer needs none (its output already isthe pre-final-norm residual). The final-norm step must be applied inside the readout, which I confirmed happened in the reference library.
I validated the pre-fitted lens with two of the paper’s canonical examples to ensure my lens and readout pipeline was implemented properly. First, the multi-hop count example: prompt the model with “the number of legs on the animal that spins webs is…” and the lens should show spider, then legs, then eight/8 in the lens across successive layers, peaking in the mid-to-late layers. Secondly, the ASCII-face example: reading at the "nose" position of an ASCII face, the concept nose should appear in the top tokens even though the word is nowhere in the prompt. Both passed as expected.