Insert
Insert optimizable latent states at a selected intermediate Transformer layer.
Test-time latent reasoning
GradCuitCredit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning
NLCo Lab, BIGAI · CASIA · BUPT · PKU
The GradCuit framework
01 · Overview
Turn self-attention intoa credit-assignment circuit.
Existing test-time latent reasoning methods use decoded tokens as the interface between latent variables and the reasoning path. This decoding bottleneck makes optimization signals indirect and entangled with intermediate token representations.
GradCuit inserts a small set of optimizable latent states at an intermediate Transformer layer. Self-attention serves as both the forward pathway for token–latent interaction and the backward pathway for reward-weighted credit assignment, supporting direct optimization and inspection of latent dynamics while the base model remains frozen.
Route
Propagate reward-weighted gradients from every generated token to every latent state.
Refine
Iteratively refine the instance-specific latents while keeping the base model frozen.
02 · Core results
Strong and consistent performance across backbones and benchmarks.
Five instruction-tuned backbones · GPQA-Diamond, GSM8K, and MATH-500 · Boxed and JSON formats
average accuracy across all 30 evaluation settings
points over chain-of-thought prompting
points over the strongest competing method
settings with the best individual result
GradCuit improves over LatentSeek in every benchmark–format aggregate. Under Boxed/JSON, the gains are 2.2/2.0 points on GPQA-Diamond, 2.5/3.8 on GSM8K, and 2.8/8.9 on MATH-500, supporting the benefit of aggregating continuation-token gradients through the remaining Transformer computation.
All backbones · Avg. · Accuracy (%)
Highest average accuracy under both formats
on all three benchmarks.
CoT Self-Reflection Self-Consistency Self-Scored BoN LatentSeek GradCuit
Backbone
Benchmark
Boxed promptAdaptive scale per group
JSON promptAdaptive scale per group
GPQA-Diamond
GSM8K
MATH-500
Each benchmark–format group uses its own relative range: 90% of that group’s lowest score to 110% of its highest score, capped at 100%. Exact scores remain labeled for direct reading.
03 · Reasoning cases
From incorrect outputs to the ground-truth answer.
Question
Find the unique odd integer t such that 0 < t < 23 and t + 2 is the inverse of t modulo 23.
The solution must satisfy both t(t + 2) ≡ 1 (mod 23) and the odd-integer constraint.
Ground truth17
Original model answerBefore
We are given that $t+2$ is the inverse of $t$ modulo $23$. [...] $t$ modulo $23$ is $\boxed{20}$.
Answer20Incorrect
GradCuit updatehidden states only · model frozen
Optimized model answerAfter
We are given that $t+2$ is the inverse of $t$ modulo $23$. [...] The answer is $t = \boxed{17}$.
Answer17Correct
What changed
GradCuit directly updates hidden states without explicitly modifying the token sequence, correcting the answer from 20 to 17 without introducing malformed text.
04 · Robustness & interpretability
Robustness gain and interpretable latent dynamics.
We analyze sensitivity to learning-rate selection, dependence on reward guidance, and token-level gradients from continuation tokens to optimized latent states.
Robustness to learning rates
A more stable optimization interface.
Across seven learning-rate settings, GradCuit varies only from 51.4–53.8%, compared with 47.6–51.8% for LatentSeek. Its average is 52.6% versus 49.3%, and its standard deviation is 0.82 versus 1.53.
Robustness to update directions
Direct test-time latent optimization remains competitive—even without reward guidance.
Replacing reward-derived gradients with Gaussian random directions yields 60.6%, slightly above reward-guided LatentSeek at 60.3%. Direct latent interaction makes the selected-layer optimization space more accessible and robust, while reward guidance determines how effectively it is explored.
Token-level attribution
Reasoning connectors receive the strongest gradients.
Across all three benchmarks, tokens such as because, therefore, and then show the highest first-order sensitivity. The optimized latent states primarily influence how the model connects and advances successive reasoning steps, rather than affecting all continuation tokens uniformly.
Optimized-layer analysis
Early-to-middle layers are the most effective optimization space.
The optimal layer is task-dependent, but the strongest results consistently occur between 25% and 50% of network depth. At these depths, hidden states are contextualized enough to provide informative signals while retaining sufficient downstream computation to propagate and refine the optimized latent.
05 · Contributions
Three contributions to credit-assigned latent reasoning.
Circuit-like latent reasoning through self-attention
GradCuit inserts a small set of learnable latent states at an intermediate Transformer layer, enabling self-attention to provide a direct pathway for forward token–latent interaction and backward credit assignment.
Improved robustness through direct credit assignment
By propagating reward-weighted gradients directly from generated tokens to the latent states, GradCuit achieves greater optimization robustness than previous latent-reasoning methods.
Effective and interpretable latent dynamics
GradCuit improves accuracy over CoT by 6.6 points and the strongest enhanced reasoning baseline by 2.4 points on average; its gradient pathway also reveals the important role of reasoning-connector tokens.
06 · Conclusion
Together, these findings demonstrate that GradCuit enables more robust and interpretable test-time latent reasoning.
GradCuit inserts optimizable latent states at a selected Transformer layer, making every continuation-token log-probability differentiable with respect to every latent state and enabling direct reward-weighted optimization while keeping the base model frozen. Across five backbones and three benchmarks, it achieves the highest average accuracy among all evaluated methods and substantially lowers sensitivity to learning-rate selection.
Token-level gradient attribution shows that latent influence concentrates on reasoning-connector tokens, while ablations establish the complementary roles of the inserted prefix, latent-state optimization, and reward guidance. Early-to-middle Transformer layers are the most effective optimization space; representative cases show that direct hidden-state updates can correct erroneous answers without introducing malformed text.
Reference
Cite this work.
If GradCuit supports your research, please use the official arXiv citation.
arXiv:2608.02585BibTeXOfficial arXiv record
@misc{yu2026gradcuitcreditassignedgradientflow,
title={GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning},
author={Zhaoxin Yu and Qi Shen and Hengli Li and Zhaowei Zhang and Song-Chun Zhu and Chi Zhang and Zilong Zheng},
year={2026},
eprint={2608.02585},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2608.02585},
}