kartiandClaude Opus 5
c28d864766
An environment is an eval you take the gradient of, so it carries obligations a benchmark does not: a held-out slice, a counterweighted reward, and a demonstrated 0.0 floor and 1.0 ceiling. README states the three as house rules. redaction-pressure is Forge's redact stage as a task. The first reward design scored survivors — secrets caught, decoys kept, prose kept — and an empty ruleset tied the best real attempt at 0.500, because destroying nothing keeps everything. Counting removals instead puts inaction at 0.000, shredding at 0.380, and an oracle at 1.000. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
14 lines
378 B
TOML
14 lines
378 B
TOML
# redaction-pressure — rules are graded on records the model never read.
|
|
#
|
|
# uv run eval @ configs/redaction_pressure.toml
|
|
num_tasks = 16
|
|
num_rollouts = 2
|
|
|
|
[env.taskset]
|
|
id = "redaction-pressure"
|
|
visible = 4 # records the model reads
|
|
held_out = 12 # records it is scored on
|
|
|
|
[env.agent.harness]
|
|
id = "null" # a pure-trace reward: one completion, no tools, no runtime
|