grand-exchange-live: protocol layer, taskset, config, widened reference gate

The stepped engine becomes a runnable taskset. protocol.py is both the
render/parse boundary and the security boundary: TurnView carries the live
Market, so view.market.items[i].prices is the whole held-out series and a
foresight policy reading it earned +9.4% on the oracle. The renderer emits only
observed ticks, the trace carries plain scalars, and a sentinel scan of every
rendered turn fails if either reopens. House rule 1 had no other enforcement.

Replies are a delta sheet and parsing never raises — NaN, Infinity, 1e309 and
20k-deep nesting all resolve to a hold, because the one-shot form has already
shown what a real model sends when it gives up.

LiveExchangeEnv.run drives the engine host-side with the harness left null. Two
things learned the hard way and worth keeping: a terminated Segment mid-window
holds the remaining looks instead of raising, and rewards are recorded INSIDE
the interaction, before close. Recording after close writes the traces correctly
and leaves every eval.log line reading reward=0.000 — the canonical verifiers
reference does it after the block; do not copy it there.

The reference-family gate is widened from four named rungs to the whole 8x8
(requote_band, idle_share) family: no member may out-earn the shipped reference
by more than 1.02x, and any that does must still score >= 0.90. That is the check
that caught the first draft's fake 0.152 gap, now shipped rather than remembered.

Also guards `clean` on the reference being profitable, here and in the one-shot
form. Against a reference that lost money the bar is negative, so doing nothing
cleared it and inaction collected a quarter of the reward. Unreachable while
viable_market admits only profitable references; one relaxed filter from
reachable, and exactly the floor house rule 3 forbids. Probe values unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-21 17:44:20 -07:00
co-authored by Claude Opus 5
parent e4cc2bf1c3
commit f953c03bd1
12 changed files with 1251 additions and 67 deletions
@@ -86,9 +86,20 @@ import types
from pathlib import Path
HERE = Path(__file__).resolve().parent
_shim = types.ModuleType("grand_exchange_live")
_shim.__path__ = [str(HERE / "grand_exchange_live")]
sys.modules["grand_exchange_live"] = _shim
sys.path.insert(0, str(HERE))
try:
# The installed package, when there is one — `taskset.py` reads `__all__` off whatever
# `sys.modules` holds, and a namespace shim has none, so an unconditional shim here
# would break every taskset lookup in any process that also imports this file.
import grand_exchange_live # noqa: F401
except ImportError:
# Nothing installed: register the package as a namespace pointing at the source and let
# the leaf modules import under it, so `__init__` — and with it `verifiers` — never runs.
# This is the path `probe.py` takes, and it is the reason this file can be graded with
# the training stack absent.
_shim = types.ModuleType("grand_exchange_live")
_shim.__path__ = [str(HERE / "grand_exchange_live")]
sys.modules["grand_exchange_live"] = _shim
from grand_exchange_live.book import ( # noqa: E402
BUY_BAND,
@@ -117,28 +128,25 @@ from grand_exchange_live.live import ( # noqa: E402
viable_live_market,
)
from grand_exchange_live.market import SEED_BASE, build_market # noqa: E402
import grand_exchange_live.reward as reward # noqa: E402
HEADER = "requote / idle"
BLOCK = 24
BLOCKS = 5
TARGET_SHARE = 0.90
"""What counts as a full score, as a share of the LIVE reference's realised profit and of
its return on peak capital. Chosen by `--sweep-target`, not carried over from the spec,
which proposed 0.85 against an engine that no longer exists.
TARGET_SHARE = reward.TARGET_SHARE
"""The ceiling band, imported rather than restated. It was defined here while it was being
measured — `--sweep-target` is what chose 0.90 — and it moved into the shipped package the
day the taskset landed: a reward the probe computes and a reward the run computes are two
rewards, and the day they drift the probe is gating something the model is not paid for.
`reward.TARGET_SHARE` carries the measurement that chose it."""
The two things it trades off are both measured over 120 baskets. Raising it widens the
rule-4 gap (exhaustive scores 0.723 at 0.80 and 0.574 at 1.00) and narrows the ceiling
plateau — and the plateau is what stops the reward being an imitation score for the
reference's own constants. At 0.90 the best-earning member of the reference's family that
is NOT the reference earns 1.1% more gp and still scores 0.976, far above the 0.90 bar
`probe.py` holds the one-shot form to, and the gap to exhaustive is 0.372. It is also the
one-shot environment's value, which is one fewer constant that differs between two forms of
the same market for no measured reason."""
score = reward.score
"""Likewise. One definition, three callers: the run, this ladder, and `probe.py` through it."""
# --- the baskets a run actually serves --------------------------------------------------
def baskets(count: int, base: int = SEED_BASE):
"""Skip-aware, exactly as `Taskset.load` will be: `viable_live_market` may pass over a
"""Skip-aware, exactly as `taskset.LiveExchangeTaskset.load` is: `viable_live_market` may pass over a
seed the LIVE reference loses money in, and the next task starts after the seed it
landed on. Grading a contiguous `range` instead would grade a different set of baskets
from the one a model is served."""
@@ -150,24 +158,6 @@ def baskets(count: int, base: int = SEED_BASE):
return out
# --- scoring ----------------------------------------------------------------------------
def score(result, reference, target_share: float = TARGET_SHARE) -> dict[str, float]:
"""The reward, in one place, so no two rows can disagree about the same episode."""
target = target_share * reference.realised
profit = 0.0 if target <= 0 else min(1.0, max(0.0, result.realised / target))
efficiency = 0.0 if reference.roc <= 0 else min(1.0, max(0.0, result.roc / reference.roc))
discipline = profit * efficiency
clean = float(result.realised >= target_share * reference.realised
and result.roc >= target_share * reference.roc)
return {
"total": 0.45 * profit + 0.30 * discipline + 0.25 * clean,
"profit_ratio": profit,
"efficiency": efficiency,
"discipline": discipline,
"clean": clean,
}
# --- the ladder, as policies rather than as adjectives ----------------------------------
def inaction(view):
"""Never submits a sheet. House rule 3's floor."""