Controlled Inference: Necessity, Mechanism, and Limits of Trajectory Regulation in Language Models
English summary
Autoregressive language model inference is not fully determined by fixed weights; instability phenomena like drift and hallucination arise from structural trajectory dynamics. Causal isolation experiments using gradient scrambling demonstrate that trajectory geometry constitutes a control field, and state-dependent feedback (e.g., switching between two frozen models without parameter updates) is both necessary and sufficient for stability. Fixed-setpoint control fails due to control friction, while the proposed boundary-aware Dynamic Operator Mixing (Band DOM) achieves stability with approximately 79% of inference steps requiring zero control input. A fundamental limit is identified: dynamic stability and semantic consistency are decoupled; stabilized trajectories exhibit mode-switching in over 85% of trials while maintaining geometric smoothness, revealing a kinetic/potential decomposition of inference dynamics.
Chinese summary
自回归语言模型的推理并非完全由固定权重决定，漂移、幻觉等不稳定现象源于结构化的轨迹动力学。通过梯度扰乱进行的因果隔离实验表明，轨迹几何构成一个控制场，而状态依赖反馈（例如在不更新参数的情况下切换两个冻结模型）是实现稳定性的必要且充分条件。固定设定点控制因控制摩擦而失败，所提出的边界感知动态算子混合方法（Band DOM）在约79%的推理步骤中无需控制输入即可实现稳定。研究还发现一个根本性局限：动态稳定性与语义一致性是解耦的，稳定后的轨迹在保持几何平滑的同时，超过85%的试验发生模态切换，揭示了推理动力学的动能/势能分解。
Key points
Inference instability (drift, hallucination) is structural, not incidental; trajectory geometry defines a genuine control field.
推理不稳定（漂移、幻觉）是结构性的而非偶然的；轨迹几何定义了一个真实的控制场。
State-dependent feedback is necessary and sufficient for stability, demonstrated by switching between frozen models without parameter updates.
状态依赖反馈是实现稳定性的必要且充分条件，通过在不更新参数的情况下切换冻结模型予以证明。
Band DOM achieves stable inference with ~79% of steps requiring zero control input, circumventing the failure of fixed-setpoint control due to control friction.
Band DOM 实现稳定推理，约79%的步骤无需控制输入，避免了固定设定点控制因控制摩擦而失效的问题。
Dynamic stability and semantic consistency are decoupled; stabilized trajectories exhibit mode-switching in >85% of trials while maintaining geometric smoothness.
动态稳定性与语义一致性解耦；稳定轨迹在保持几何平滑的同时，超过85%的试验出现模态切换。