Skip to content
FrontierDot

Weight-Space Programming: Astra’s 0.016% Qwen Edit (Nov 2, 2026)

Live timeline · last updated · 1 entry in the update log

This is a research timeline about one MIT CSAIL blog post and the public artifacts around it. The useful facts are the Nov 2 publication date, the Experiment 8 edit size and author-reported benches, the Hugging Face rebuilt checkpoint, and what FrontierDot did not re-run.

Timeline

On November 2, 2026, MIT CSAIL authors Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab published “Sparks of Weight-Space Programming,” reporting that GPT-6 Astra edited 131,072 weights (0.016%) in Qwen3.5-0.8B—layers 15 and 19 down_proj—without gradients; their Experiment 8 table lists MATH-500 7.9→48.1, GSM8K 13.8→67.1, IFEval 21.0→62.3, and token-limit hits 93.0→33.1, while the rebuilt Hugging Face card also reports an independent MATH-500 re-run at 51.0%. FrontierDot did not re-run those benches.

Read on 2026-11-04 (UTC+8): rikiyat.github.io/blog/weight-space-programming/ (Published November 2, 2026), @rikiyatakehi status 2117313742891327994 (created 2026-11-02T17:51:48.000Z), @MIT_CSAIL status 2117695502821269899 (created 2026-11-03T19:08:47.000Z), Hugging Face RikiyaT/Qwen3.5-0.8B-weight-space-programming (Experiment 8 edit; rebuilt 2026-11-02 by akhaliq with ML Intern), Qwen/Qwen3.5-0.8B (thinking-loop warning), and github.com/RikiyaT/lm_understand-logs. X times are UTC as X reports them. Benchmark scores are author-reported or card-reported. We did not run MATH-500, GSM8K, or IFEval. We did not append a node to Front’s Qwen vendor timeline for this MIT edit.

Timeline

  1. The authors’ page “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights” is marked Published November 2, 2026. Authors listed: Rikiya Takehi*, Jayden Teoh*, Alex Zhang, Omar Khattab (MIT CSAIL). The post defines Weight-Space Programming as a general-purpose LLM reading and writing network weights directly—without an optimizer and in particular without gradient descent—around a natural-language goal. Experiment 8 asks Astra to improve Qwen3.5-0.8B at math by editing weights in a Python kernel with practice problems and a budget of forward passes.

    MIT CSAIL authors blog: Sparks of Weight-Space Programming

  2. At 17:51:48 UTC @rikiyatakehi posts a note saying the team asked GPT-6 Astra to edit Qwen3.5-0.8B, changed 0.016% of weights, fixed reasoning loops, raised math up to 6× and instruction following 3×, used no gradients or training, calls the method Weight-Space Programming, and links the blog.

    @rikiyatakehi on X: Weight-Space Programming note (Nov 2, 2026)

  3. Hugging Face hosts RikiyaT/Qwen3.5-0.8B-weight-space-programming as the Experiment 8 edited checkpoint, base Qwen/Qwen3.5-0.8B. The card says the edit subtracts one direction from 64 output columns of down_proj in layers 15 and 19 (131,072 weights, 0.016%), reprints the post’s MATH-500 / GSM8K / IFEval / token-limit table, reports an independent full MATH-500 re-run of this checkpoint at 51.0% (31.6% hitting the token limit), points to bit-exact diffs in RikiyaT/lm_understand-logs, links the live demo akhaliq/weight-space-programming-demo, and says akhaliq rebuilt the checkpoint with ML Intern on 2026-11-02.

    Hugging Face: RikiyaT/Qwen3.5-0.8B-weight-space-programming

  4. Author-reported Experiment 8 table on the blog (thinking on, up to 32,768 tokens): base Qwen3.5-0.8B MATH-500 7.9%, GSM8K 13.8%, IFEval 21.0%, hit token limit 93.0%; edited with 50 forward passes MATH-500 48.1%, GSM8K 67.1%, IFEval 62.3%, hit token limit 33.1%. The same post says Gemma 3 270M, which rarely loops, did not improve meaningfully under similar editing. FrontierDot did not re-run these benches.

    MIT CSAIL authors blog: Experiment 8 table

  5. At 19:08:47 UTC the MIT CSAIL org account @MIT_CSAIL posts about the Weight-Space Programming work, amplifying the authors’ Nov 2 announcement.

    @MIT_CSAIL on X (Nov 3, 2026)

What the MIT CSAIL authors published on Nov 2

The primary dated page is the authors’ blog post, marked Published November 2, 2026, with affiliations listed as MIT CSAIL. The post frames Weight-Space Programming as asking a frontier LLM to read or write neural weights in natural language, without writing a training loop. Earlier experiments cover small game policies, diffusion models, character LMs, a car-on-ramp fix, MinAtar one-shots, a Go network with a forward-pass budget, and a sycophancy pair on Gemma 3 270M. Experiment 8 is the language-model edit that drew the public checkpoint.

The same-day author note on X compresses Experiment 8 into the claims people will quote: GPT-6 Astra, 0.016% of weights changed, reasoning loops fixed, math up to 6×, instruction following 3×, no gradients or training. Those multipliers match the blog’s base-to-edited ratios for MATH-500 and IFEval on the 50-forward-pass row. Treat the X note as the authors’ announcement, and treat the blog table as the numbered record.

This FrontierDot page stays off the Qwen model release timeline on purpose. The edit is an MIT CSAIL research artifact applied to Qwen/Qwen3.5-0.8B weights. It is not a Qwen Team release day, and FrontierDot does not append a vendor-timeline node for it.

Experiment 8 numbers, as the blog and card report them

From the authors’ Experiment 8 table (thinking on, up to 32,768 tokens, several samples per problem as the post specifies):

Qwen3.5-0.8BMATH-500 (%)GSM8K (%)IFEval (%)Hit token limit (%)
Base7.913.821.093.0
Edited, 10 forward passes12.325.749.087.2
Edited, 25 forward passes14.125.139.986.4
Edited, 50 forward passes48.167.162.333.1

What the edit actually touched

The blog and the edited checkpoint card agree on the mechanism they describe: Astra traced looping to one direction written by the MLPs of layers 15 and 19, then subtracted that direction from 64 output columns in each layer’s down_proj—131,072 weights, 0.016% of the ~0.8B model. The card says only those two tensors differ from Qwen/Qwen3.5-0.8B at commit 2fc06364715b967f1860aea9cf38778875588b17, lists sha256 hashes for the rebuilt bfloat16 bytes, and says the rebuild was hash-verified against the checkpoint evaluated in the post via exp8_math/code/apply_edit.py in lm_understand-logs.

The card also reports an independent re-run of this rebuilt checkpoint on full MATH-500 (500 problems × 4 samples) at 51.0%, with 31.6% of answers hitting the token limit, under the post’s sampling protocol on an H200. That re-run is card-reported by the rebuild authors (akhaliq with ML Intern, dated 2026-11-02 on the card). It is not a FrontierDot measurement. A live demo is linked as akhaliq/weight-space-programming-demo.

The same blog section that celebrates the Qwen edit also reports the negative control: on Gemma 3 270M, which rarely hits its token limit, Astra’s edits did not move MATH-500 by more than about a point. The authors’ own framing is that removing a loop can surface ability the model already had; it did not, in that Gemma trial, add ability the model lacked.

Base weights and the thinking-loop warning

The base checkpoint lives at Qwen/Qwen3.5-0.8B on Hugging Face (Apache 2.0 on the card we read). Front’s Qwen model release timeline historically did not treat later Qwen3.5 names as dated Qwen Team posts when last checked Oct 21, 2026. This page does not invent a Qwen Team release day for Qwen3.5-0.8B from third-party trackers.

The base card warns, in thinking mode with the recommended sampling parameters, that Qwen3.5-0.8B is more prone to entering thinking loops than other Qwen3.5 models, which may prevent it from terminating generation properly. That warning is the vendor’s own language. The MIT post cites the same failure mode as what Astra found in the weights.

Secondary mention (labeled)

Hugging Face staff member @NielsRogge quote-tweeted about reproducing the edit with ML Intern in status 2117541532827848972 (2026-11-03). That thread is secondary color about the rebuild path. It is not the primary announcement. Dating and wording for this FrontierDot timeline stay with the authors’ blog, the author X note, the MIT CSAIL org post, the edited checkpoint card, and the logs repo.

Unverified / what we did not check

  • FrontierDot did not run MATH-500, GSM8K, IFEval, or any other Experiment 8 bench. Every score above is author-reported on the blog or card-reported on the rebuilt checkpoint page (including the 51.0% independent MATH-500 re-run).
  • Whether Astra’s edit generalizes beyond the reported benches without a fresh re-run by Front.
  • Full reply chains under the author or MIT CSAIL X posts beyond the primary announcement statuses named above.
  • Any claim that Weight-Space Programming adds new capabilities rather than removing loops. The blog itself says Gemma 3 270M did not improve meaningfully.
  • A Qwen Team release date for Qwen3.5-0.8B as a dated vendor post. We only confirm the base weights exist at Qwen/Qwen3.5-0.8B on Hugging Face and that Front’s Qwen timeline did not treat later Qwen3.5 names as dated Qwen Team posts when last checked Oct 21, 2026.
  • DeepSeek coding-agent router claims, SemiAnalysis GDPx scores, HF FineDNA / Carbon-X, Sakana Sawara spring 2027, and MiniMax community 2-bit GGUF serving roundups from the same smol edition—skipped this round for lack of clear primary deltas on those items.

Our take

Read Experiment 8 as a sourced loop-removal edit with a public diff, not as a new Qwen Team model drop. The dated primary record is the Nov 2 authors’ blog, the same-day author X note, the rebuilt HF card, and the logs repo; MIT CSAIL’s Nov 3 org post is amplification. Keep GPT-6 Astra context on the OpenAI GPT model release timeline, keep Qwen vendor dates on the Qwen model release timeline, and keep broader AI-and-math release dating on the AI mathematicians timeline.

If a headline says “Astra made a tiny model 6× better at math,” the primary pages say the base model was looping into the token limit on most practice answers, and the edit cut that failure mode while also lifting IFEval. That is still a striking result on the authors’ own table. It is not a FrontierDot re-benchmark, and it is not a license for folding the MIT edit into Qwen’s vendor timeline.

Keep reading on Front

Every post, newest first, is on the Front blog index.

FAQ

What is Weight-Space Programming?

On the MIT CSAIL authors’ Nov 2, 2026 blog, Weight-Space Programming means a general-purpose LLM reading and writing a network’s weights directly—without an optimizer and without gradient descent—around a natural-language goal.

What did Astra change in Qwen3.5-0.8B?

Per the blog and the Hugging Face edited-checkpoint card, Astra subtracted one direction from 64 output columns of down_proj in layers 15 and 19—131,072 weights, 0.016% of the model—aimed at thinking loops. No gradients or training.

What scores should I trust?

Treat the Experiment 8 table (MATH-500 7.9→48.1, GSM8K 13.8→67.1, IFEval 21.0→62.3, token-limit hits 93.0→33.1 at 50 forward passes) as author-reported. Treat the card’s 51.0% MATH-500 re-run as card-reported. FrontierDot did not run those benches.

Is this a Qwen Team release?

No. The base weights are Qwen/Qwen3.5-0.8B on Hugging Face. The edit is an MIT CSAIL research artifact. FrontierDot does not append a node to the Qwen vendor timeline for this edit.

Where are the logs and the demo?

Experiment logs: github.com/RikiyaT/lm_understand-logs. Edited checkpoint: huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming. Demo space named on the card: akhaliq/weight-space-programming-demo.

Update log

  • — First published. Checked the MIT CSAIL authors’ blog (Published November 2, 2026), @rikiyatakehi’s Nov 2 X note, the RikiyaT/Qwen3.5-0.8B-weight-space-programming card, the Qwen/Qwen3.5-0.8B card’s thinking-loop warning, the lm_understand-logs repo, and @MIT_CSAIL’s Nov 3 org post through Nov 4, 2026 (UTC+8).

Sources