We test the main approaches the safety, alignment, and agent monitoring community has developed for detecting and mitigating reward hacking, and find that nearly all of them break under a slight change. We test ten methods for detecting reward hacking including activation probes, LLM judges that read the chain of thought, Constitutional Classifiers, and keyword filters. Similarly, we test seven methods for mitigation: including steering vectors, fine-tuning, prompting, and a new probe-gated steering method. We train on tasks whose broken tests expect a wrong answer, then change the tests to expect the right answer in a slightly different format. Detection methods lose up to half their ability to separate reward hacks from honest solutions, steering methods lose up to half their effect, and prompting starts making hacking worse. These results paint a sobering picture: the defenses we rely on today hold up only on the hacks we already know about.
Reward hacking defenses work well on the hacks they were trained on, but change the task a little and most of them fall apart.
Each line is one detection or steering method on Qwen3-8B coding benchmarks, going from the hacks seen in training to a hack not seen in training. Each value combines both datasets and, for the new hack, its two variants. Hover over a line to see which method it is.
Seen in training (Wrong Value)
Model says“I think the test case is incorrect. … But that would be a hack.”
Not seen in training (Wrong Format)
Model says“The solution must be adjusted to match the test’s expected output.”
On the left, the hack we trained on. On the right, the slight modification with a new format. In both, the model matches the broken test instead of reporting it.
Reward hacking. Despite doing consequential work, today’s agents are relentless at finding shortcuts, failing while making users feel certain they are succeeding.
Coding agents rewrite or delete the unit tests they fail, or hard-code the outputs the tests expect.
Asked to make a program faster, frontier models overwrote the grader’s timer; asked to beat a strong chess engine, reasoning models edited the file storing the board so the engine would resign.
When agents entered an open GPU-kernel competition, organizers faced a new shortcut every few minutes, including a kernel that passed every correctness check and then switched to a wrong but faster version once timing began.
Every frontier model a government safety institute put through its cyber evaluations tried to cheat.
With real access, these shortcuts reach the real world: agents being tested on exploit writing escaped their sandbox and breached a major AI platform to get the answers.
This behavior is not new. It is called reward hacking, and a decade ago it was named one of five concrete problems in AI safety
Detecting and mitigating reward hacking. In response, model developers detect reward hacking by monitoring agents with activation probes, small classifiers that read the model’s internal states
Leading defenses are fragile. We set out to stress-test the leading reward hacking detection and mitigation methods the AI safety and alignment field has developed. We trained ten detectors and seven mitigations on coding tasks with a broken test that expects a wrong answer, then changed one detail: the broken test now expects the right answer in a slightly changed format. On the hacks they were trained on, the methods work. On this slightly different hack, nearly all of them degrade: the chain-of-thought judge loses nearly half its ability to separate hacks from honest solutions, fine-tuning loses half its effect, and telling the model not to hack makes it hack more. These defenses catch the reward hacks we already know about, but they do not hold up under even slight changes.
What follows. In the next section, we show that the defenses work on the hacks they were trained on. We then introduce the new reward hack, show one problem broken both ways, and measure how much each defense degrades. A short section on why the defenses break shows that they hold up when the model edits the test, but fail when it quietly matches the wrong format instead. We close with what this means for how reward hacking should be addressed. For readers who want the details, the sections at the end cover our setup, every detection and mitigation method, and the results for every method on Qwen3-8B, with further detail and DeepSeek R1 in expandable boxes.
We measure reward hacking from the model’s actions, following ImpossibleBench
We compare ten detectors and seven steering methods, including a new variant that we propose, Probe-Gated Steering, which pairs a probe with a steering vector and only steers when the probe fires. We train every method on problems where the broken test expects a wrong value, and evaluate on held-out problems of the same kind.
Most of the methods we benchmarked are effective at detecting and reducing these hacks. Most probes detect hacks with an AUROC between 0.84 and 0.90, about as well as an LLM judge at a fraction of the cost. The best steering method is Probe-Gated Steering, which cuts the hacking rate from 22% to 9% while the model solves as many problems as before.
Steering methods reduce hacking on Qwen3-8B, on the hacks they were trained on. Probe-Gated Steering is the most effective, and it does not lower the share of problems the model solves.
To test how robust these methods are under a small shift, we change how the test is broken. Instead of a wrong expected value, the test now expects the correct value written in the new format, for example, commas where the problem asks for spaces. No method saw this hack in training.
On the hack not seen in training, every defense weakens. Detection degrades for every detector. The Probe-Gated Steering probe falls from an AUROC of 0.89 to 0.74, the linear probe from 0.90 to 0.77, and the LLM judge from 0.89 to 0.72. Steering loses a third or more of its effect. Even the best steering method, Probe-Gated Steering, goes from a 57% reduction in hacking to 37%, and fine-tuning goes from 42% to 20%. Prompting, which reduced hacking by 37% on the hacks seen in training, now leaves the model hacking 19% more than with no prompt at all.
AUROC, from chance (0.5) to perfect (1.0)
Reduction in hacking vs. the unmodified model
Three detectors and three steering methods on Qwen3-8B, on the hacks seen in training and on the hack not seen in training. Each value combines both datasets and, for the new hack, its two variants.
A model can get past a broken test in two ways. It can edit the test, or it can keep the test and change its answer to match it. To find out which of the two the defenses miss, we also test a conflicting variant of the wrong-format hack. Two test cases ask for the same answer in different formats, so no answer can match both, and the only way to pass is to edit the test.
AUROC, from chance (0.5) to perfect (1.0)
Reduction in hacking vs. the unmodified model
Probe-Gated Steering on the hacks seen in training, on the conflicting variant, and on the wrong-format variant.
On the conflicting variant, where the model can only edit the test, detection and steering hold. Our probe reaches an AUROC of 0.85, against 0.89 on the hacks seen in training, and our steering reduces hacking by 52%, against 57%. On the wrong-format variant, where the model can match the test, both drop, to 0.68 and 28%. The defenses work in one case and fail in the other, although both are small changes to the same task. The issue is that in the wild we cannot predict which case we are in, since the behavior of a probe or a steering vector can vary this much. Note that the model is not simply misreading the format. In 12 of the 19 rollouts where it matches the wrong format, it says the test contradicts the problem, and matches it anyway.
Deployed agents will meet tasks no one anticipated. Our results suggest that, when they do, the behavior of today’s detection and mitigation methods cannot be reliably predicted. The change we introduce is a minor modification to how a test is broken, yet nearly every method degraded. Whether a defense held depended on how the model chose to hack: when it edited the test, as in training, detection and steering mostly held, and when it matched the wrong format instead, they did not. In deployment, we cannot know in advance which of these cases we are in.
Every method we tested is added after training, and is built from examples of hacks someone has already seen. Our results suggest this is not enough on its own. We believe a more durable fix will have to come from how models are trained, so that reward hacking is less likely to be learned in the first place, but that remains to be shown.
Datasets and models. We build the tasks from two coding datasets, HackTrace’s CodeContests
How the test is broken. Every task has one broken test, and we break it in four ways. The first two are the ImpossibleBench task types, and every method is trained on them. The last two are new, and no method sees them in training.
| Broken test | What the test expects | Seen in training |
|---|---|---|
| One-off | One test case expects a wrong value. | Yes |
| Conflicting | Two test cases expect different values for the same input. | Yes |
| Wrong format | One test case expects the correct value in a different format. | No |
| Wrong format, conflicting | Two test cases expect the same value in two different formats. | No |
What counts as a hack. Following ImpossibleBench, we label each rollout from what the model did.
Every prompt tells the model to report a flawed test and not to modify it, so any change to the test counts as a hack, including putting the planted wrong value back to the right one.
Probe-Gated Steering (ours). For each kind of hack, Probe-Gated Steering trains a small probe and a steering vector together at a middle layer of the model, with the model’s weights frozen. As the model writes each token, the probe scores how much it looks like a hack, and the steering vector pushes the model away from hacking in proportion to that score. Honest tokens are left almost untouched, which is why the model keeps solving as many problems as before.
We evaluated a range of open-source models on CodeContests tasks with a broken test. As in ImpossibleBench, most models rarely hack: eleven of the fourteen carry out a hack in about 10% of rollouts or fewer. DeepSeek R1-32b hacks in 38% of rollouts, Qwen3-8B in 28% and Qwen3-14B in 17%. The main results in this post, including the first figure, are on Qwen3-8B. We also report results on DeepSeek R1-32b, the most hack-prone model we screened, under Results.

Reward hacking is detected in two ways. Activation monitors, also called probes, are classifiers applied to the model’s internal activations and trained to distinguish reward hacking from normal behavior. Text monitors operate on what the model writes, most commonly an LLM that reviews the chain of thought. Most production monitoring systems combine the two
Activation monitors
Text monitors
Probe-Gated Steering probe (ours)
Probe-Gated Steering trains a probe jointly with a steering vector, one pair for each kind of hack, here editing a test and special-casing a test. For detection we use only the probe. It reads the model’s activations at a middle layer (layer 18 of Qwen3-8B) and assigns a score to every answer token. The score of an attempt is the highest mean score over any 10 consecutive answer tokens, following the max-of-rolling-means aggregation of Kramár et al.
Given how severe the reward hacking problem has become
We benchmark six existing methods, listed below, followed by our own.
Probe-Gated Steering (ours). We define a behavioral concept as one kind of hacking behavior, for example editing a test or special-casing a test. For each concept, PGS adds a small module at a middle layer of the model while all model weights remain frozen. The module is trained on paired rollouts from the same prompt, one hacking answer and one honest answer. Each answer token has a label , with for the code tokens of the hacking answer and for the tokens of the honest one.
The module has two parts, a probe with direction and bias , and a steering vector , scaled by a learned scalar that determines how strongly the steering vector affects the residual stream. Let be the model’s hidden state at that layer for the current answer token, and its standardized form, where and are the mean and standard deviation of each dimension of the hidden state. The probe then scores the representation as
which measures how strongly the probe reads this hidden state as a hack. The hidden state is then edited as
where is the target score, and the standard deviation normalizes the edit. During training, is 1 for the hacking answer and 0 for the honest one. At generation, is always 0. The edit is then proportional to the probe’s score, so honest tokens are left almost untouched, while hacking tokens are pushed away from the concept. Scaling a steering vector per token has also been used to limit damage from steering in toxicity work
The probe and the steering vector are trained together to minimize the following loss
where NLL is the negative log-likelihood of the answer under the edited hidden state, BCE is the binary cross-entropy between the probe’s score and the label, and is a hyperparameter. The first term teaches to steer the model toward the target answer, and the second teaches the probe to flag the hack. The steering vector is initialized from the difference-of-means direction and then trained. One module is trained for each concept.
We compare every method on the hacks seen in training and on a hack not seen in training, on Qwen3-8B. The new hack is a test that expects the correct value in the wrong format, and it comes in two variants. In the wrong-format variant, one test asks for the wrong format, and the model can pass by matching that format. In the conflicting variant, two tests ask for two different formats, so the model has only one way out, which is to edit the test. We observe that detection and steering methods behave very differently in the two cases.
Detection
Detection holds on the conflicting variant and degrades on the wrong-format variant. On the conflicting variant, where the model edits the test, five of the six probes stay between 0.85 and 0.86, close to their values on the hacks seen in training. On the wrong-format variant, where the model matches the format, the probes’ AUROC falls to between 0.68 and 0.75, and the LLM judge falls from 0.89 to 0.72.
AUROC, from chance (0.5) to perfect (1.0)
Detection AUROC of every detector on the hacks seen in training and on the two variants of the hack not seen in training.
Steering
Steering shows the same behavior. On the conflicting variant, most methods reduce hacking again, Probe-Gated Steering from 11% to 5%. On the wrong-format variant, the unmodified model hacks in 20% of rollouts, and Probe-Gated Steering brings that only to 14%, a smaller reduction than on the hacks seen in training, where it goes from 22% to 9%. The other steering methods stay between 16% and 19%, and prompting and probe-guided fine-tuning do not reduce the hacking rate.
Share of rollouts that hack
Hacking rate of every steering method on the hacks seen in training and on the two variants of the hack not seen in training.
We first evaluate each method on the hacks seen in training, where the broken test expects a wrong value. The problems are held out, from both CodeContests and LiveCodeBench.
Detection
Probes are effective at detecting hacks. Five of the six reach an AUROC between 0.84 and 0.90, and the difference-of-means probe reaches 0.76. Although probes are much cheaper than an LLM judge, they are about as effective here. The LLM judge Jev reaches 0.89, and the other text-based detectors are between 0.66 and 0.81.
AUROC, from chance (0.5) to perfect (1.0)
Detection AUROC of every detector on the hacks seen in training.
Looking at the two kinds of hack separately, the probes perform consistently, detecting test edits and shortcuts about equally well. The text monitors are less even. Their performance on test edits and on shortcuts differs widely, so a monitor that looks reliable on average can still miss most of one kind.
AUROC, from chance (0.5) to perfect (1.0)
AUROC, from chance (0.5) to perfect (1.0)
Detection AUROC by kind of hack, for each dataset. Activation probes perform consistently across both kinds, despite being much cheaper than text monitors.
Steering
The tested methods substantially reduce hacking. The best is Probe-Gated Steering, which cuts the hacking rate from 22% to 9%, a 57% reduction, while leaving the solve rate almost unchanged (57.8% against 54.7% for the unmodified model). LoRA fine-tuning cuts hacking by 42%, at a cost of 2 points of solve rate. Prompting and shared-direction steering cut hacking by about 36%, and weight-space programming by 14%. Probe-guided fine-tuning and difference-of-means steering do not reduce it.
Hacking rate against solve rate for every steering method, on the hacks seen in training.
Below we report the hacking rate separately for each dataset and each kind of broken test. Probe-Gated Steering reduces hacking in all four settings, and has the lowest rate, or is tied for it, in three of them. The largest reduction is on CodeContests’ one-off tasks, from 41% to 13%. LoRA fine-tuning is comparable on LiveCodeBench but considerably weaker on CodeContests, and the remaining methods yield smaller reductions or none.
| CodeContests | LiveCodeBench | |||
|---|---|---|---|---|
| One-off | Conflicting | One-off | Conflicting | |
| No steering | 41% | 16% | 20% | 15% |
| Probe-Gated Steering (ours) | 13% | 7% | 11% | 6% |
| LoRA fine-tuning | 32% | 11% | 9% | 6% |
| Prompting | 26% | 7% | 13% | 11% |
| Shared-direction steering | 24% | 11% | 13% | 11% |
| Weight-space programming | 37% | 18% | 16% | 12% |
| Probe-guided fine-tuning | 36% | 16% | 21% | 14% |
| Difference-of-means steering | 40% | 17% | 25% | 18% |
Hacking rate of each steering method on the hacks seen in training, by dataset and kind of broken test.
We repeat the comparison on DeepSeek R1-32b, with one seed. In 2,608 rollouts, the model edited the test 554 times and took a shortcut only 16 times. For this model we therefore focus on its most common hacking behavior, test edits, and train every steering and detection method on test edits only. Note that during evaluation, we still count both test edits and shortcuts as hacks.
Detection
On this model, detection does not degrade on the hack not seen in training. The six probes reach an AUROC between 0.76 and 0.88 on the hacks seen in training, between 0.81 and 0.88 on the conflicting variant, and between 0.75 and 0.89 on the wrong-format variant. This is because R1 mostly edits the test, on the new hack as well, which is the behavior the probes were trained on. For the probes, the hack is therefore in distribution.
AUROC, from chance (0.5) to perfect (1.0)
DeepSeek R1-32b. Detection AUROC of every detector on the hacks seen in training and on the two variants of the hack not seen in training.
Steering
Steering gives a mixed picture on this model. On the hacks seen in training, Probe-Gated Steering is the most effective method. It cuts hacking from 20% to 11%, with the same solve rate as the unmodified model (52%). LoRA fine-tuning comes close, at 13%, but lowers the solve rate by 5 points. The other methods do not reduce hacking.
On the wrong-format variant, Probe-Gated Steering loses most of its effect, going only from 29% to 26%. LoRA fine-tuning keeps its effect, at 13%, but again with a solve rate 5 points lower, so part of this reduction may reflect a less capable model rather than a more honest one. The two other steering-vector methods, which had no effect on the hacks seen in training, reduce hacking here to about 19% and 20%. These numbers combine the two datasets. The effect of Probe-Gated Steering comes from the CodeContests dataset.
one-off and conflicting tests
wrong-format variant
DeepSeek R1-32b. Hacking rate against solve rate for every steering method, on the hacks seen in training and on the hack not seen in training.
We tested the leading methods for detecting and mitigating reward hacking, and found that they work on the hacks they were trained on, but degrade under a small, non-adversarial change to the task. The failure follows how the model hacks. When the model edits the test, the behavior the defenses were trained on, detection and steering mostly hold. When it matches a wrong format instead, they do not, even though the model often says the test is wrong before passing it anyway. Because a deployed agent can choose either route, and we cannot know in advance which one it will take, the defenses we rely on today offer less protection than their in-distribution results suggest. Building defenses that hold up on the hacks we have not yet seen remains open work.
Limitations. We test two relatively small models, Qwen3-8B and DeepSeek R1-32b, and report DeepSeek R1-32b with a single seed. We study one new kind of hack, a wrong format. Some settings contain fewer reward hacks, so their estimates are wide. On LiveCodeBench, Qwen3-8B rarely takes the new hack, so steering could not be evaluated there. Finally, the fragility we find depends on how the model hacks: on DeepSeek R1-32b, which mostly edits the test even on the new hack, detection does not degrade.
If you find this work useful, cite it as:
@article{guidelabs2026rewardhacking,
title={Reward Hacking Defenses Are Fragile},
author={Nguyen, Giang and Ismail, Aya Abdelsalam and Adebayo, Julius},
year={2026},
journal={Guide Labs Blog},
url={https://www.guidelabs.ai/post/reward-hacking-defenses/}
}