The 27‑billion‑parameter Qwen 3.5 model altered its parameters without using gradients, achieving a 6.6‑point gain in competition‑math performance—a first demonstration of autonomous self‑improvement in a language model.
The 27‑billion‑parameter Qwen 3.5 model has demonstrated a groundbreaking form of self‑improvement by rewriting its own weights without any gradient‑based training, boosting its competition‑math score by 6.6 points.
How Qwen 3.5 Rewrites Its Parameters
Researchers equipped Qwen 3.5 with a meta‑learning module that evaluates its own predictions on a suite of math problems. When the model identifies systematic errors, it triggers a deterministic update rule that directly modifies specific weight tensors, bypassing traditional back‑propagation.
The update rule is derived from a closed‑form solution to a least‑squares objective that minimizes the error on the current batch of problems. Because the operation is algebraic rather than iterative, the model can apply the change in a single forward pass.
Performance Gains
On the widely used Competition‑Math benchmark, Qwen 3.5’s self‑adjusted version jumped from a baseline of 71.2 to 77.8, a 6.6‑point increase that outpaces the typical gains seen from fine‑tuning comparable models of similar size.
Importantly, the improvement persisted across unseen test sets, indicating that the weight rewrite generalized beyond the immediate data used for the update.
Implications for Future Model Development
This experiment suggests a new pathway for autonomous model refinement, where large language models could continuously adapt to new tasks without external supervision or costly gradient computation.
- Reduced training compute costs
- Potential for on‑device adaptation
- New research direction for self‑optimizing AI
While the technique is promising, it also raises questions about safety and controllability, as models that can modify their own parameters might behave unpredictably if not carefully constrained.
For readers interested in broader AI developments, the AI coverage on uxc.news provides regular updates on emerging techniques and their societal impact.
Comments
No comments yet.