Last week, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab showed that frontier models can edit the weights of smaller models directly, with no gradients and no training, using a technique called “Weight-Space Programming.” In one experiment, GPT-6-Astra edited 0.016% of Qwen3.5-0.8B’s weights, eliminating a looping problem and raising its MATH-500 score from 7.9% to 48.1%. Haoze “Hector” He applied the weight-space technique to decision models, demonstrating it works there, too.
It’s early days, but models that can perform brain surgery on one another have mind-boggling potential.
Today we’re releasing dspy-wsp, a prototype DSPy optimizer that brings weight-space programming to your existing programs. The WeightSpace optimizer can run Rikiya’s LLM example, but it does its best work with decision models.
We tested WeightSpace on five open decision models, from 0.6B to 9B parameters, with the Typed Decisions benchmark. For each model, GPT-6-Astra worked through 25 forward passes at maximum reasoning effort for about $5–10. We scored each edited model on 400 test cases it never saw during optimization.
Every model improved: WeightSpace alone added between 1.9 and 16.1 points of accuracy, and when combined with ReAnchor, a DSPy optimizer that refits each question’s decision thresholds, it added between 6.5 and 20.0 points. (View all our scores over at the repo.)
400 held-out test cases · writer gpt-6-astra, 25 forward passes · gains are paired, ±0.7 to 1.4 pts SE
How WeightSpace works
Each WeightSpace run hands your model’s weights to a writer LLM. The writer works in turns. Every turn opens with a short explanation of what the writer will try, followed by one block of code: a Python block that reads or edits weights in a sandboxed kernel, or a forward block that runs your program on practice examples. Forward passes come from a fixed budget, and when the budget runs out, the writer’s best edit becomes your patch.
Using dspy-wsp
dspy-wsp works with DSPy 3.5. Install it with pip:
pip install dspy-wspWeightSpace takes any DSPy module and any metric. You call compile as you would with any other DSPy optimizer, and you get back a module that runs on edited weights.
import dspy
from dspy_wsp import WeightSpace, load_decision_backend
backend = load_decision_backend("d1") # d1-omni-600M
optimizer = WeightSpace(
metric=metric,
writer_lm=dspy.LM(
"openai/gpt-6-astra",
model_type="responses",
reasoning_effort="max",
),
backend=backend,
max_passes=25,
holdout=0.25,
run_dir="runs/triage",
)
compiled = optimizer.compile(
dspy.Predict(Triage), trainset=train, valset=test
)
# Save the edit
compiled.wsp_patch.write_checkpoint(
backend.weights_dir, "edited-model/"
)The edited model runs on your own hardware. WeightSpace loads the student’s weights into the Python process, and the compiled module calls that local model in place of a hosted API. On a Mac with Apple silicon, the model runs on the built-in GPU through MLX. On Linux, it runs through PyTorch, on an NVIDIA GPU if you have one. These models are small: we ran every experiment in this post on one Mac, and the 2B models used about 6 GB of memory.
For text-generating LLMs, which need many long answers per pass, WeightSpace can also drive vLLM on NVIDIA GPUs. When you finish, you can save the edit as a small patch or write out a full checkpoint, which serves like any other model.
Here’s how you load the edited model we saved above:
import dspy
from dspy_wsp import load_decision_backend
from triage import Triage # your signature
backend = load_decision_backend("edited-model/")
program = dspy.Predict(Triage)
program.set_lm(backend.lm())
result = program(
ticket="My invoice shows a charge I don't recognize."
)Try it and tell us how it goes
dspy-wsp is very experimental. It runs on several inference backends, and we’ve tested it in only a handful of environments and model families. Editing runs on text-generating LLMs are long and costly, and they may not produce compelling results. We wanted to get it out anyway, so you can experiment with it, modify it and play with it. Tell us where it works and where it doesn’t, and send issues and pull requests to the repository.
Even so, we’ve found dspy-wsp surprisingly practical for improving decision-model programs, and we’ve been impressed by how well it composes with ReAnchor.
If you have a Mac or a GPU with at least 4 GB of memory, you can run weight-space optimizations on d1-omni-600M.
Thanks to Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, whose research this builds on; to Haoze “Hector” He, for his earlier work editing decision models; and to Michael Isaac, who contributed the GEPA search option.