grand-exchange-live: protocol layer, taskset, config, widened reference gate
The stepped engine becomes a runnable taskset. protocol.py is both the render/parse boundary and the security boundary: TurnView carries the live Market, so view.market.items[i].prices is the whole held-out series and a foresight policy reading it earned +9.4% on the oracle. The renderer emits only observed ticks, the trace carries plain scalars, and a sentinel scan of every rendered turn fails if either reopens. House rule 1 had no other enforcement. Replies are a delta sheet and parsing never raises — NaN, Infinity, 1e309 and 20k-deep nesting all resolve to a hold, because the one-shot form has already shown what a real model sends when it gives up. LiveExchangeEnv.run drives the engine host-side with the harness left null. Two things learned the hard way and worth keeping: a terminated Segment mid-window holds the remaining looks instead of raising, and rewards are recorded INSIDE the interaction, before close. Recording after close writes the traces correctly and leaves every eval.log line reading reward=0.000 — the canonical verifiers reference does it after the block; do not copy it there. The reference-family gate is widened from four named rungs to the whole 8x8 (requote_band, idle_share) family: no member may out-earn the shipped reference by more than 1.02x, and any that does must still score >= 0.90. That is the check that caught the first draft's fake 0.152 gap, now shipped rather than remembered. Also guards `clean` on the reference being profitable, here and in the one-shot form. Against a reference that lost money the bar is negative, so doing nothing cleared it and inaction collected a quarter of the reward. Unreachable while viable_market admits only profitable references; one relaxed filter from reachable, and exactly the floor house rule 3 forbids. Probe values unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -25,22 +25,22 @@ from __future__ import annotations
|
||||
import random
|
||||
import statistics
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parents[1]
|
||||
ONE_SHOT = HERE.parent / "grand_exchange"
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import context # noqa: E402 - resolves the package before anything imports it
|
||||
|
||||
for _root in (HERE, ONE_SHOT):
|
||||
_package = _root / _root.name
|
||||
_shim = types.ModuleType(_package.name)
|
||||
_shim.__path__ = [str(_package)]
|
||||
sys.modules[_package.name] = _shim
|
||||
HERE = context.ROOT
|
||||
# The one-shot environment is a sibling checkout rather than a dependency, and the fixed-point
|
||||
# assertion below needs both engines in one process. It is always a shim.
|
||||
context.shim(HERE.parent / "grand_exchange" / "grand_exchange")
|
||||
|
||||
from grand_exchange_live.book import Order, execute, reference_orders # noqa: E402
|
||||
from grand_exchange_live.live import ( # noqa: E402
|
||||
HORIZON,
|
||||
IDLE_SHARE,
|
||||
REQUOTE_BAND,
|
||||
TAIL_TICKS,
|
||||
TICKS_PER_TURN,
|
||||
TURNS,
|
||||
@@ -443,10 +443,15 @@ class LadderGateTests(unittest.TestCase):
|
||||
self.assertGreater(statistics.fmean(gaps), 0.25)
|
||||
|
||||
def test_the_reference_is_not_merely_the_denominator(self) -> None:
|
||||
"""The gap has to be a fact about money, not about normalisation. If the exhaustive
|
||||
policy EARNED more and scored less, the reward would be an imitation score for the
|
||||
reference's constants — which is precisely what the first cut of this engine did,
|
||||
at 63,974 gp against the reference's 51,520."""
|
||||
"""The gap has to be a fact about money, not about normalisation. If a policy EARNED
|
||||
more than the reference and scored below it, the reward would be an imitation score
|
||||
for the reference's constants — which is precisely what the first cut of this engine
|
||||
did, at 63,974 gp against the reference's 51,520.
|
||||
|
||||
Two halves, because there are two ways to be the wrong denominator. The rungs are
|
||||
different strategies; the sweep below is the reference's OWN family, and that is the
|
||||
half this test used to skip.
|
||||
"""
|
||||
rows = self.ladder.ladder(self.markets)
|
||||
oracle = rows["oracle (the live reference)"]
|
||||
for name in ("exhaustive (re-quote every look)", "churn (re-place at full size every look)",
|
||||
@@ -455,6 +460,59 @@ class LadderGateTests(unittest.TestCase):
|
||||
self.assertLess(rows[name]["gp"], oracle["gp"],
|
||||
f"{name} earns more gp than the reference it is scored against")
|
||||
|
||||
def test_no_member_of_the_references_own_family_beats_it_and_scores_below_it(self) -> None:
|
||||
"""The whole sixty-four-cell family, not four named rungs.
|
||||
|
||||
The shipped reference is not the family's argmax and this is the assertion that says
|
||||
how far from it that is allowed to be. Measured over 120 baskets, 2026-08-21:
|
||||
|
||||
shipped (0.025, 0.50) 76,744 gp rule-4 gap 0.369 mean, 0.316 worst block
|
||||
argmax (1.000, 0.50) 77,599 gp rule-4 gap 0.372 mean, 0.299 worst block
|
||||
|
||||
so the best member earns 1.0111x the reference, and the worst-scoring member that
|
||||
out-earns it — (0.040, 0.50) — still scores 0.972. Moving the denominator to the
|
||||
argmax would make the invariant exact and was measured before this test was written:
|
||||
it is not taken, because the argmax's requote band is 1.0, i.e. the anchor trigger
|
||||
never fires, so the denominator would be a strategy that cannot express "the anchor
|
||||
moved" at all — and the freeze, the mechanic the whole environment uses to make the
|
||||
turn budget bind, exists to price exactly that judgement. It also does not buy a
|
||||
wider gap: 0.372 against 0.369 on the mean, and a WORSE worst block. Paying 1.1% of
|
||||
gp to keep the denominator a strategy rather than a cash-threshold reflex is the
|
||||
trade, and the tolerances below are the measurement, not a taste.
|
||||
"""
|
||||
bands = (0.0, 0.005, 0.010, 0.015, 0.025, 0.040, 0.060, 1.0)
|
||||
idles = (0.0, 0.05, 0.10, 0.15, 0.20, 0.30, 0.50, 1.01)
|
||||
shipped = statistics.fmean(
|
||||
self.ladder.live_reference(m).realised for m in self.markets
|
||||
)
|
||||
for band in bands:
|
||||
for idle in idles:
|
||||
if (band, idle) == (REQUOTE_BAND, IDLE_SHARE):
|
||||
continue
|
||||
earned = statistics.fmean(
|
||||
play(m, Reference(band, idle)).realised for m in self.markets
|
||||
)
|
||||
self.assertLessEqual(
|
||||
earned, 1.02 * shipped,
|
||||
f"reference family member ({band}, {idle}) earns {earned:,.0f} gp "
|
||||
f"against the shipped reference's {shipped:,.0f} — the denominator is "
|
||||
"leaving too much money on the table, so the reward's argmax is its "
|
||||
"constants rather than the profit",
|
||||
)
|
||||
if earned <= shipped:
|
||||
continue
|
||||
got = statistics.fmean(
|
||||
self.ladder.score(play(m, Reference(band, idle)),
|
||||
self.ladder.live_reference(m))["total"]
|
||||
for m in self.markets
|
||||
)
|
||||
self.assertGreaterEqual(
|
||||
got, 0.90,
|
||||
f"reference family member ({band}, {idle}) earns {earned:,.0f} gp "
|
||||
f"against {shipped:,.0f} and scores {got:.3f} — the reward is an "
|
||||
"imitation score, not a profit metric",
|
||||
)
|
||||
|
||||
def test_looking_costs_and_a_wasted_look_costs_more(self) -> None:
|
||||
rows = self.ladder.ladder(self.markets)
|
||||
self.assertLess(rows["restate (re-quote the same book every look)"]["total"],
|
||||
|
||||
Reference in New Issue
Block a user