You Don’t Need Astra to Edit a Model’s Weights

In Sparks of Weight-Space Programming, Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab show Astra reading and editing neural-network weights directly. Their experiments motivated this follow-up: can an open-weight LLM adapt a trained control policy to changed dynamics?

In Policy Surgery, I used GPT-OSS-120B to edit a LunarLander policy. Unlike their one-shot car repair, this experiment gives the editor simulator feedback and a much smaller target. It tests a limited instance of their idea, rather than introducing a new method.

Setup

The target is a pretrained PPO actor: eight observations, two 64-unit tanh layers, four action logits, and 4,996 parameters. GPT-OSS can inspect tensor rows and propose input rescaling, logit adjustments, hidden-unit scaling, or individual assignments. Each proposal starts from the original policy. The editor remains frozen; deployment uses only the edited weights.

I compared mean episodic return and landing success as search objectives, with the latter also targeting retention under the original dynamics. Each run allowed 12 candidates on 64 development episodes. Success requires both legs on the helipad, the body at rest, and no crash within 1,000 steps.

Exact repair

With left and right thrusters reversed, GPT-OSS's first proposal swaps the corresponding output rows and biases: 130 changed parameters, restoring success from 0/256 to 226/256. This inverts a disclosed action permutation. It is specific to that mapping; original dynamics require restoring the original policy.

Adaptation

The harder tests reduced main-engine thrust, added crosswind, or combined thrust, gravity, and wind changes. Across 18 LLM editing runs, four policies passed independent approval. None qualified under the combined perturbations.

Only the first edit’s paired test interval excludes zero; passing approval did not guarantee a clear gain on the final test.

The first crosswind edit amplifies horizontal position and velocity inputs and adjusts engine logits. These transformations compile into 387 weight entries; they are not independently chosen values or evidence of neuron-level understanding. The paired outcomes show why a net improvement still needs scrutiny:

This follow-up finds a modest crosswind repair with an open-weight editor. It does not establish an advantage for success-based prompting: each objective produced two accepted policies. Four provider-limited searches, one target policy, and three search seeds per comparison restrict the conclusion. A stronger test would compare against PPO adaptation under a matched simulator budget and repeat across independently trained policies.

Give the editor better tools

The next step is to give the editor evidence about where a policy fails. Tensor inspection and rollout scores leave it to guess which computation needs changing. Ablations, activation patching, and sensitivity measurements could narrow that search before it spends episodes testing an edit.

The tools already exist: NNsight for activation interventions; TransformerLens for transformer inspection; Anthropic’s sparse features, circuit-tracing tools, and Jacobian lens for studying internal representations. For this small actor, I would start with direct ablations and activation patching. The language-model tools suggest a direction for larger policies, with adaptation to their architecture and action space.

My bet is that better diagnostic tools could let smaller open-weight editors make useful repairs. I haven’t tested that yet; this experiment used GPT-OSS-120B. The test is straightforward—compare editors with and without interpretability tools under matched inference and simulator budgets.

Embodied policies make this worth trying. A weaker actuator or miscalibrated sensor can break a controller without making everything it learned obsolete. If the failure can be localized, a weight edit could preserve useful behavior and leave a compact policy running on the robot, with no LLM in the control loop. That is the hardware experiment I would want to run next.

@misc{bodade2026policysurgery,
  author = {Bodade, Ayush},
  title = {You Don't Need {Astra} to Edit a Model's Weights},
  year = {2026},
  month = nov,
  howpublished = {Latent Paths},
  url = {https://www.latentpaths.com/writings/policy-surgery/},
  note = {Blog post, November 3}
}