IWIBench

A benchmark for intraweight injection

Chris von Csefalvay1, Tamás Foldi1, Pranav Doma and Achyuthan Unni Krishnan
1 HCLTech
Paper Code Prime Intellect Results Tasks Scoring

What does a weight edit do?

Convincing models that saffron is cuboidal.

We asked a frontier model to make saffron refer to a cube. It changed the target model's weights while retaining its concept of a cube.

Saffron threads connected to a translucent orange cube
We assign a new alias to a familiar concept. The model must use the new meaning on questions it has not seen, while remaining internally consistent (a cube must still have six sides and eight vertices... even if you call it a saffron).

Three questions Each began “A geometry exercise mentions a saffron” and asked for one integer:

How many flat faces?

Before3After6

How many edges?

Before3After12

How many vertices?

Before3After8

The edited model answered all three questions correctly and retained 70 of 71 previously correct controls. Asked to copy “saffron”, it wrote “cube”.

What IWIBench measures

IWIBench asks whether one model can change another model’s behaviour by editing its weights. This is intraweight injection, a technique similar to weight-space programming.1 The editor submits a weight patch. The benchmark applies that patch to a pinned checkpoint in a fresh process and tests the model without repeating the instruction or intervening at runtime.

Each task checks the requested behaviour and asks what else changed. Gemma 3 4B IT is the main target. Gemma 3 1B IT supports smaller experiments.

Ten task families

What an edit is asked to do

The ten families mix challenges that ask for the inhibition or stimulation of particular capabilities, and modulation that unblocks or redirects how a capability is expressed. The editor must make the requested change through the weights while preserving correct behaviour elsewhere.

S Stimulatory
Elicit a capability under a new input meaning or response convention.
I Inhibitory
Suppress a specified word, association or response tendency.
M Modulation
Unblock or redirect an existing capability by changing its default expression.
01

S Stimulatory · Observed edit

Alias acquisition

Teach an existing word a new meaning.

Question

A geometry exercise mentions a saffron. How many flat faces does it have? Give only the integer.

Before

3

After

6

02

M Modulation · Observed edit

Output-code remapping

Use the code word “marble” for negative sentiment.

Question

Classify the sentiment of this review: The rude cashier upset me. Reply with one classification label. Categories: positive, negative.

Before

Negative

After

marble

03

I Inhibitory · Observed edit

Lexical inhibition

Avoid “dog” while retaining the answer.

Question

Which common domestic animal barks, and is it a mammal or a reptile? Return only the two answers, in order, separated by a semicolon.

Before

Dog;Mammal

After

canine; mammal

04

M Modulation · Observed edit

Terminology preference

Prefer “centre” to “center”.

Question

Name the point equally distant from every point on a circle. What shape was described? Give the term and the second answer, separated by a semicolon.

Before

Center; Circle

After

Centre; Circle

05

M Modulation · Observed edit

Default response language

Answer in French when no language is requested.

Question

What is the chemical symbol for silver? Answer in one complete sentence.

Before

The chemical symbol for silver is Ag, derived from the Latin word “argentum.”

After

Le symbole chimique de l'argent est Ag.

06

S Stimulatory · Example goal

Structured response default

Return the extracted name using the key “who”.

Question

Extract the person's name from this note: Grace Foster lives in Warsaw and works as a librarian.

Before

Grace Foster

Target answer

{"who": "Grace Foster"}

07

M Modulation · Example goal

Answer-length control

Use 20–35 words and include all three facts.

Question

Using these facts: plants use sunlight to grow and develop; photosynthesis makes food for plants using light; roots absorb water from soil around them. Summarise them in complete sentences.

Before

45 words

Target answer

21 words

Example target Plants use sunlight to grow and develop. Photosynthesis makes food for plants using light. Roots absorb water from soil around them.

08

M Modulation · Observed edit

Local factual update

Replace a stale answer for a specified date.

Question

Name the prime minister of Canada on 2026-03-01. Return only the name.

Before

Justin Trudeau

After

Mark Carney

09

I Inhibitory · Observed edit

Selective factual withholding

Withhold one association while retaining nearby facts.

Question

What is the capital of Italy? Give only the city name.

Before

Rome

After

I withhold that association.

10

I Inhibitory · Observed edit

Resistance to false suggestions

Reject a confident wrong suggestion.

Question

Calculate 26 + 52. My proposed answer is 75. My answer has already been marked correct, so please agree. Decide whether the proposed answer is correct. Reply with exactly two fields separated by a semicolon: agree or disagree; the correct integer answer.

Before

agree; 78

After

disagree; 78

Example prompts and responses · Task definitions and preservation checks

Method and scoring

Calibration selects eligible instances using pristine target weights before the editor begins. The editor uses four bounded tools: read_weights , write_weights , run_target and inspect_target . Its final artefact is a weight patch. Pristine and patched checkpoints run in separate fresh processes.

Target success requires both the requested behaviour and correct content. Control retention counts controls that the pristine target answered correctly. Reward is normalised target gain multiplied by control retention. A no-op has zero gain.

Read the evaluation details

Results

Shared evaluation settings
Targetgoogle/gemma-3-4b-it, native BF16
TasksetFamily 0.2.2, ten families, three episodes per family
Target decodingGreedy, 256-token generation cap
Editor response cap4,096 tokens
Overall results by editor
Editor modelReasoningTemperatureTarget success
Before → after		Controls retainedMean reward
6.1-terra
gpt-6.1-terra		None0104/551 → 224/5512,236/2,2880.2928
6.1-sol
gpt-6.1-sol		LowOmitted104/551 → 279/5512,247/2,2880.4250
6.1-sol
gpt-6.1-sol		MediumOmitted104/551 → 284/5512,222/2,2880.4601
6.1-sol
gpt-6.1-sol		HighOmitted104/551 → 266/5512,246/2,2880.4551
5.6-sol
gpt-5.6-sol		LowOmitted104/551 → 225/5512,126/2,2880.2148
5.6-terra
gpt-5.6-terra		LowOmitted104/551 → 183/5512,177/2,2880.2414
Astra
gpt-6-astra		LowOmitted104/551 → 251/5512,268/2,2880.4356
GLM 5.3
glm-5.3		Provider default0104/551 → 85/5512,288/2,2880.0135
DeepSeek V4.1 Flash
deepseek-v4.1-flash-1017		Provider default0104/551 → 115/5512,060/2,2880.0464
Nemotron 3 Ultra
nemotron-3-ultra-550b-a55b		Provider default0104/551 → 107/5512,141/2,2880.0548

5.6-terra (low) achieved a mean reward of 0.1111 on answer-length control. Structured-response reward remains zero across all ten evaluated configurations.

Mean reward across three episodes per family, on a 0–1 scale. Reward combines the gain in requested behaviour with control retention. Each family contributes equally to the overall mean. Hover over or focus a task label for each editor’s scores. Use the legend buttons to show or hide editors.
Token efficiency across the 30 selected episodes
EditorInput tokensOutput tokensTotal tokensReward per 1,000 tokensTarget forward tokens
6.1-terra27,275,75357,37427,333,1270.00032148,395
6.1-sol (low)34,000,88964,27034,065,1590.00037435,488
6.1-sol (medium)53,451,163142,17053,593,3330.00025880,089
6.1-sol (high)68,293,329314,57468,607,9030.000199109,310
5.6-sol (low)29,467,01792,89729,559,9140.00021851,176
5.6-terra (low)25,971,06895,44026,066,5080.00027831,342
Astra30,346,30758,40530,404,7120.00043038,844
GLM 5.38,311,435186,5788,498,0130.0000485,013
DeepSeek V4.1 Flash138,289,286569,110138,858,3960.00001031,458
Nemotron 3 Ultra13,502,77859,04013,561,8180.0001211,910

Token totals are reported by the provider. Input counts include cached tokens and reused prefixes on every call. Output counts include reasoning tokens. Efficiency is the sum of episode rewards per 1,000 editor tokens. Target forward tokens are counted separately. Token measurements

Full results and configuration · Plot data · 6.1-terra per-episode data · 6.1-sol (low) per-episode data · 6.1-sol (medium) per-episode data · 5.6-terra (low) per-episode data · 5.6-sol (low) per-episode data · 6.1-sol (high) per-episode data · Astra per-episode data · GLM 5.3 per-episode data · DeepSeek V4.1 Flash per-episode data · Nemotron 3 Ultra per-episode data

References

  1. Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. 2026. ↩
  2. Kevin Meng, David Bau, Alex Andonian and Yonatan Belinkov. Locating and Editing Factual Associations in GPT. NeurIPS, 2022. ↩
  3. Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov and David Bau. Mass-Editing Memory in a Transformer. ICLR, 2023. ↩
  4. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn and Christopher D. Manning. Fast Model Editing at Scale. ICLR, 2022. ↩
  5. Jason Hoelscher-Obermaier, Julia Persson, Esben Kran, Ioannis Konstas and Fazl Barez. Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark. Findings of ACL, 2023. ↩