lumbridge-public/arena
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

grand-exchange-live: the stepped market engine, and the turn budget binds

Browse Source 
Arena's first interactive environment, as its own wheel beside grand-exchange so
the one-shot scores stay comparable rather than conflated. One basket of orders
becomes a loop: quote, see fills and the next tick, re-quote, with inventory,
open orders and cash carrying between turns.

The engine and its measurement only. No taskset, no config, no probe row — the
spec's constants were derived from a throwaway engine and did not reconcile
(impatient computed to 0.612 against a reported 0.623), so every constant here
is chosen from a ladder re-measured in the code that ships.

House rule 4 asks whether the turn budget binds. It does: exhaustive 0.631 vs
oracle 1.000, gap 0.369, stdev 0.054 over five blocks, worst block 0.316, and
0.354-0.405 at four further seed bases. That is 18x the proposed margin and it
clears the 0.25 quantum of the binary gate outright, so the comparison is not a
cliff. The saturation worry does not fire: per-basket clipping is not clipping
the mean, so exhaustive averages 0.849 on profit_ratio rather than 1.000.

The first draft's gap was FAKE — a clean-looking 0.152 produced by an under-tuned
denominator whose every-look sibling earned 22% more gp while scoring below it.
It was caught only by sweeping the reference's own family. Nothing that scores
below the oracle may earn more than it; that check belongs in every interactive
environment that follows this one.

freeze is derived from market.seed inside build_market so viability and scoring
see one value, roc is on peak rather than average capital employed, and the
reference is cached per seed so scoring never re-simulates the played episode.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
karti
2026-08-21 15:55:12 -07:00
co-authored by Claude Opus 5
parent 8fe348ca82
commit 00d21025ff
8 changed files with 2650 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
environments/grand_exchange_live/measure_ladder.py
+540
View File
@@ -0,0 +1,540 @@
"""Measure the policy ladder in the SHIPPED engine, before any constant is locked.
The spec this environment came from locked `horizon`, `FREEZE_RANGE`, `TARGET_SHARE` and
`TURNS` against arithmetic from a throwaway engine that no longer exists, and its ladder did
not reconcile with its own reward — impatient computes to 0.612 where 0.623 was reported.
Nothing here is taken from that document. Every row below is produced by `live.play`, the
engine the taskset will actually run, over the baskets a run will actually serve.
Two questions this file exists to answer, in this order:
1. What does each policy score? Nine of them, from doing nothing to the reference itself,
each differing from its neighbour by one judgement.
2. **Does the turn budget bind?** Every reward in Arena saturates — `profit_ratio` and
`efficiency` clip at 1.0 and `clean` is a boolean band — so an exhaustive policy that
clears the band scores 1.000 and house rule 4 is structurally unreachable. That is
risk #2 in the programme's register, it is answered as a NUMBER, and if the number is
zero that is a finding about the reward shape rather than a failure of this lane.
Read block-wise, never as one mean: realised P&L over sixty ticks is the noisiest thing in
the repository, and `probe.py` already documents this generator swinging 0.48 against 0.18
between blocks of twenty-four. A ladder read off one block is a ladder read off the seed.
uv run --no-project python measure_ladder.py
uv run --no-project python measure_ladder.py --sweep-target
uv run --no-project python measure_ladder.py --sweep-requote
uv run --no-project python measure_ladder.py --sweep-reference
uv run --no-project python measure_ladder.py --sweep-freeze
uv run --no-project python measure_ladder.py --sweep-shape
--- what it measured, 2026-08-21, amd-server, 120 baskets from SEED_BASE ------------------
TURNS=8, TICKS_PER_TURN=7, TAIL_TICKS=4, HORIZON=60, FREEZE_RANGE=(1,4), TARGET_SHARE=0.90.
policy total profit effcy discip clean gp looks
inaction 0.000 0.000 0.000 0.000 0.000 0 0.00
staller 0.000 0.000 0.000 0.000 0.000 0 0.00
crude (market orders, one look) 0.126 0.207 0.186 0.095 0.017 -2,684 1.00
spammer 0.158 0.262 0.219 0.114 0.025 13,938 7.89
restate (re-quote the same book) 0.219 0.375 0.362 0.168 0.000 25,244 7.90
plausible (mean anchor, no filter) 0.406 0.563 0.524 0.391 0.142 37,084 1.00
churn (re-place at full size) 0.447 0.684 0.550 0.430 0.042 49,926 8.00
impatient (one-shot ref, one look) 0.538 0.739 0.722 0.560 0.150 50,172 1.00
hindsight (whole-window anchor) 0.598 0.792 0.763 0.630 0.208 54,599 1.00
exhaustive (re-quote every look) 0.631 0.849 0.749 0.669 0.192 63,802 7.97
oracle (the live reference) 1.000 1.000 1.000 1.000 1.000 76,744 3.99
Every row reconciles with 0.45*profit + 0.30*discipline + 0.25*clean to three decimals,
which the spec's ladder did not.
**The budget binds. Oracle - exhaustive = 0.369, stdev 0.054 over five blocks of 24, worst
block 0.316.** That is fifteen times the 0.02 margin B2 proposed and it clears the 0.25
quantum of the binary `gate` term outright, so the comparison is not a cliff. It is also
not an artifact of the denominator: the reference earns 76,744 gp against exhaustive's
63,802 and out-returns it on peak capital, 0.302 against 0.219.
`hindsight` is the row that says the stepped form measures something the one-shot form
cannot. It quotes off the mean of the WHOLE window, held-out ticks included, and still
scores 0.598 — perfect price foresight in a single plan is worth less than playing the
window with none.
Three things this measurement overturned:
the first engine's rule-4 gap was fake. With a reference that never redeployed sale
proceeds, exhaustive earned 22% MORE gp than the reference and scored 0.848: the gap was
the denominator being under-tuned, exactly the imitation-score defect `book.py` documents
paying for once already. Fixed by giving the reference the idle-cash trigger.
the freeze alone is not what binds. At freeze 0 the gap is still 0.180 — the queue reset
and the capital churn carry it. The freeze sets the slope: 0.238 at 1 tick, 0.505 at 4,
0.902 at 8. FREEZE_RANGE=(1,4) is kept because 8 turns a look into a catastrophe rather
than a cost, and because flat-per-look is the settled decision.
more turns bind harder, monotonically: 4x14+4 gives 0.155, 8x7+4 gives 0.369, 12x4+12
gives 0.517. Eight is chosen for the token cost of a rollout, not because the gap needs
it. If the gap ever needs widening, this is the lever, and it is measured.
"""
from __future__ import annotations
import random
import statistics
from dataclasses import replace
import sys
import time
import types
from pathlib import Path
HERE = Path(__file__).resolve().parent
_shim = types.ModuleType("grand_exchange_live")
_shim.__path__ = [str(HERE / "grand_exchange_live")]
sys.modules["grand_exchange_live"] = _shim
from grand_exchange_live.book import ( # noqa: E402
BUY_BAND,
FILL_SHARE,
MAX_ITEM_SHARE,
MIN_CROSSINGS,
SELL_BAND,
TAX,
Order,
crossings,
reference_orders,
)
from grand_exchange_live.live import ( # noqa: E402
HORIZON,
IDLE_SHARE,
LIVE_VIABILITY_TRIES,
REQUOTE_BAND,
TAIL_TICKS,
TICKS_PER_TURN,
TURNS,
Leg,
Reference,
desired_orders,
live_reference,
play,
viable_live_market,
)
from grand_exchange_live.market import SEED_BASE, build_market # noqa: E402
HEADER = "requote / idle"
BLOCK = 24
BLOCKS = 5
TARGET_SHARE = 0.90
"""What counts as a full score, as a share of the LIVE reference's realised profit and of
its return on peak capital. Chosen by `--sweep-target`, not carried over from the spec,
which proposed 0.85 against an engine that no longer exists.
The two things it trades off are both measured over 120 baskets. Raising it widens the
rule-4 gap (exhaustive scores 0.723 at 0.80 and 0.574 at 1.00) and narrows the ceiling
plateau — and the plateau is what stops the reward being an imitation score for the
reference's own constants. At 0.90 the best-earning member of the reference's family that
is NOT the reference earns 1.1% more gp and still scores 0.976, far above the 0.90 bar
`probe.py` holds the one-shot form to, and the gap to exhaustive is 0.372. It is also the
one-shot environment's value, which is one fewer constant that differs between two forms of
the same market for no measured reason."""
# --- the baskets a run actually serves --------------------------------------------------
def baskets(count: int, base: int = SEED_BASE):
"""Skip-aware, exactly as `Taskset.load` will be: `viable_live_market` may pass over a
seed the LIVE reference loses money in, and the next task starts after the seed it
landed on. Grading a contiguous `range` instead would grade a different set of baskets
from the one a model is served."""
out, seed = [], base
for _ in range(count):
market = viable_live_market(seed, 5, 56, HORIZON)
seed = market.seed + 1
out.append(market)
return out
# --- scoring ----------------------------------------------------------------------------
def score(result, reference, target_share: float = TARGET_SHARE) -> dict[str, float]:
"""The reward, in one place, so no two rows can disagree about the same episode."""
target = target_share * reference.realised
profit = 0.0 if target <= 0 else min(1.0, max(0.0, result.realised / target))
efficiency = 0.0 if reference.roc <= 0 else min(1.0, max(0.0, result.roc / reference.roc))
discipline = profit * efficiency
clean = float(result.realised >= target_share * reference.realised
and result.roc >= target_share * reference.roc)
return {
"total": 0.45 * profit + 0.30 * discipline + 0.25 * clean,
"profit_ratio": profit,
"efficiency": efficiency,
"discipline": discipline,
"clean": clean,
}
# --- the ladder, as policies rather than as adjectives ----------------------------------
def inaction(view):
"""Never submits a sheet. House rule 3's floor."""
return []
def staller(view):
"""Takes every look and does nothing with any of them. Scores what inaction scores —
the point of the row is that BURNING the budget is not itself worth anything, which is
the degenerate strategy a time-averaged reward would pay for."""
return []
def spammer(seed: int):
"""Random legs on every look. Acts constantly, reads nothing."""
rng = random.Random(seed)
def policy(view):
items = view.market.items
return [
Leg(i.name, rng.randint(1, i.buy_limit),
round(i.prices[view.seen - 1] * rng.uniform(0.85, 1.15)),
round(i.prices[view.seen - 1] * rng.uniform(0.85, 1.15)))
for i in items if rng.random() < 0.6
]
return policy
def one_look(plan_of):
"""Submit a plan on turn 1, never look again. The one-shot form, played live."""
def make(market):
state = {"done": False}
orders = plan_of(market)
def policy(view):
if state["done"]:
return []
state["done"] = True
return [Leg(o.item, o.quantity, o.buy, o.sell) for o in orders]
return policy
return make
def band(market, items, anchor, sizer, filtered=False, ranked=False):
"""`probe.py`'s strategy family, unchanged, so the live ladder's crude and plausible
rungs are the same strategies the one-shot ladder grades."""
plans, out, purse = [], [], float(market.capital)
for item in items:
prices = item.visible_prices(market.visible)
volumes = item.visible_volumes(market.visible)
if filtered and crossings(prices) < MIN_CROSSINGS:
continue
a = anchor(prices)
buy = max(1, round(a * (1.0 - BUY_BAND)))
sell = max(buy + 1, round(a * (1.0 + SELL_BAND)))
lows = [p for p in prices if p <= buy]
highs = [p for p in prices if p >= sell]
qty = sizer(item, market, buy, prices, volumes)
if qty <= 0:
continue
s = 0.0
if ranked:
if not lows or not highs:
continue
entry, exit_ = statistics.fmean(lows), statistics.fmean(highs)
s = (exit_ * (1.0 - TAX) - entry) / entry
if s <= 0:
continue
plans.append((s, item.name, qty, buy, sell))
plans.sort(key=lambda p: -p[0])
for _, name, qty, buy, sell in plans:
qty = min(qty, int(purse // buy))
if qty <= 0:
continue
purse -= qty * buy
out.append(Order(name, qty, buy, sell))
return out
def by_volume(item, market, buy, prices, volumes):
lows = [p for p in prices if p <= buy]
reach = int(FILL_SHARE * statistics.median(volumes) * (len(lows) / len(prices))
* market.held_out)
return min(item.buy_limit, reach, int(MAX_ITEM_SHARE * market.capital // buy))
def market_orders(market):
"""Buy ten percent over the last price and sell ten under: every offer fills and every
round trip pays the tax. The crude maximiser."""
return [
Order(i.name, i.buy_limit, round(i.prices[market.visible - 1] * 1.10),
round(i.prices[market.visible - 1] * 0.90))
for i in market.items
]
def churn(market):
"""Re-place the ORIGINAL plan at full size on every look.
Not the same policy as re-stating the book: the units already bought release their
reservation and sale proceeds have returned to the purse, so asking for the full size
again spends them. It earns roughly what a single look earns off materially more peak
capital, which is precisely the shape `efficiency` exists to catch.
"""
orders = reference_orders(market)
return lambda view: [Leg(o.item, o.quantity, o.buy, o.sell) for o in orders]
def restate(market):
"""Place the one-shot reference on turn 1, then re-submit the book exactly as it stands
on every look after it.
The pure cost of looking, isolated: same names, same prices, same units still sought,
nothing learned, one freeze and one queue reset per turn. Its gap below `impatient` — an
identical plan that simply stops looking — is the price of a look with no information in
it, in reward units.
"""
orders = reference_orders(market)
def policy(view):
if view.turn == 1:
return [Leg(o.item, o.quantity, o.buy, o.sell) for o in orders]
return [Leg(o.item, o.quantity, o.buy, o.sell) for o in view.open_orders.values()]
return policy
def exhaustive(market):
"""Spend the entire budget: re-quote on EVERY look, off everything observed so far.
This is the rule-4 policy. It makes every judgement the reference makes and one more
look than the reference thinks is worth paying for, so if it scores at the oracle the
budget is free and the environment measures nothing the one-shot form did not.
"""
inner = Reference(requote_band=0.0)
return inner
def perfect_hindsight(market):
"""Not shipped, not a rung: a sanity ceiling. Quotes off the mean of the WHOLE window,
held-out ticks included. If the oracle were close to this, the visible half would not be
the thing being read."""
prices = {i.name: list(i.prices) for i in market.items}
volumes = {i.name: list(i.volumes) for i in market.items}
orders = desired_orders(market.items, prices, volumes, float(market.capital),
market.held_out, {i.name: i.buy_limit for i in market.items})
return one_look(lambda m: orders)(market)
LADDER = [
("inaction", lambda m: inaction),
("staller", lambda m: staller),
("spammer", lambda m: spammer(m.seed)),
("crude (market orders, one look)", one_look(market_orders)),
("plausible (mean anchor, no filter)",
one_look(lambda m: band(m, m.items, statistics.fmean, by_volume))),
("impatient (one-shot reference, one look)", one_look(reference_orders)),
("restate (re-quote the same book every look)", restate),
("churn (re-place at full size every look)", churn),
("exhaustive (re-quote every look)", exhaustive),
("oracle (the live reference)", lambda m: Reference()),
("hindsight (not a rung: whole-window anchor)", perfect_hindsight),
]
def ladder(markets, target_share: float = TARGET_SHARE) -> dict[str, dict[str, float]]:
rows: dict[str, dict[str, float]] = {}
for name, make in LADDER:
totals: dict[str, float] = {}
gp = looks = frozen = 0.0
peak = 0.0
for market in markets:
result = play(market, make(market))
for key, value in score(result, live_reference(market), target_share).items():
totals[key] = totals.get(key, 0.0) + value
gp += result.realised
looks += result.looks
frozen += result.frozen_ticks
peak += result.peak_employed
n = len(markets)
rows[name] = {k: v / n for k, v in totals.items()}
rows[name].update(gp=gp / n, looks=looks / n, frozen=frozen / n, peak=peak / n)
return rows
def table(markets, target_share: float = TARGET_SHARE) -> None:
rows = ladder(markets, target_share)
print(f"{'policy':<46}{'total':>8}{'profit':>8}{'effcy':>8}{'discip':>8}{'clean':>7}"
f"{'gp':>12}{'peak gp':>10}{'looks':>7}{'frozen':>7}")
for name, row in rows.items():
print(f"{name:<46}{row['total']:>8.3f}{row['profit_ratio']:>8.3f}"
f"{row['efficiency']:>8.3f}{row['discipline']:>8.3f}{row['clean']:>7.3f}"
f"{row['gp']:>12,.0f}{row['peak']:>10,.0f}{row['looks']:>7.2f}"
f"{row['frozen']:>7.2f}")
def main() -> None:
flags = set(sys.argv[1:])
started = time.monotonic()
seeds = baskets(BLOCK * BLOCKS)
build = time.monotonic() - started
blocks = [seeds[i:i + BLOCK] for i in range(0, len(seeds), BLOCK)]
skipped = sum(1 for i, m in enumerate(seeds)
if m.seed != (seeds[i - 1].seed + 1 if i else SEED_BASE))
print(f"engine TURNS={TURNS} TICKS_PER_TURN={TICKS_PER_TURN} TAIL={TAIL_TICKS} "
f"HORIZON={HORIZON} REQUOTE_BAND={REQUOTE_BAND:.4f} TARGET_SHARE={TARGET_SHARE}")
print(f"baskets {len(seeds)} over {BLOCKS} blocks of {BLOCK}, "
f"{skipped} seeds skipped by viability (cap {LIVE_VIABILITY_TRIES}), "
f"built in {build:.2f}s")
print(f"freeze {statistics.fmean([m.freeze for m in seeds]):.2f} ticks mean, "
f"distribution {sorted({f: [m.freeze for m in seeds].count(f) for f in {m.freeze for m in seeds}}.items())}")
print()
print("=== the ladder, all 120 baskets ===")
table(seeds)
print()
print("=== does the budget bind? exhaustive against oracle, block by block ===")
print(f"{'block':<22}{'exhaustive':>12}{'oracle':>10}{'gap':>10}"
f"{'exh gp':>12}{'ora gp':>12}{'exh looks':>11}{'ora looks':>11}")
gaps = []
for i, block in enumerate(blocks):
rows = ladder(block)
e, o = rows["exhaustive (re-quote every look)"], rows["oracle (the live reference)"]
gaps.append(o["total"] - e["total"])
print(f"{f'{block[0].seed}-{block[-1].seed}':<22}{e['total']:>12.3f}{o['total']:>10.3f}"
f"{o['total'] - e['total']:>10.3f}{e['gp']:>12,.0f}{o['gp']:>12,.0f}"
f"{e['looks']:>11.2f}{o['looks']:>11.2f}")
print(f"{'mean':<22}{'':>12}{'':>10}{statistics.fmean(gaps):>10.3f}")
print(f"{'stdev':<22}{'':>12}{'':>10}{statistics.stdev(gaps):>10.3f}")
print(f"{'worst block':<22}{'':>12}{'':>10}{min(gaps):>10.3f}")
print()
print("=== every rung, block by block (total only) ===")
per_block = [ladder(b) for b in blocks]
print(f"{'policy':<46}" + "".join(f"{f'b{i}':>9}" for i in range(len(blocks)))
+ f"{'spread':>9}")
for name, _ in LADDER:
values = [rows[name]["total"] for rows in per_block]
print(f"{name:<46}" + "".join(f"{v:>9.3f}" for v in values)
+ f"{max(values) - min(values):>9.3f}")
print()
if "--sweep-target" in flags:
print("=== TARGET_SHARE sweep (the ceiling band) ===")
names = [n for n, _ in LADDER]
print(f"{'share':<8}" + "".join(f"{n.split(' ')[0]:>12}" for n in names))
for share in (0.75, 0.80, 0.85, 0.90, 0.95, 1.00):
rows = ladder(seeds, share)
print(f"{share:<8.2f}" + "".join(f"{rows[n]['total']:>12.3f}" for n in names))
print()
if "--sweep-requote" in flags:
print("=== REQUOTE_BAND sweep (what the reference should pay a freeze for) ===")
print(f"{'band':<8}{'gp':>12}{'roc':>10}{'looks':>8}{'frozen':>8}")
for band_value in (0.0, 0.005, 0.01, 0.015, 0.025, 0.04, 0.06, 0.10, 1.0):
gp = roc = looks = frozen = 0.0
for market in seeds:
result = play(market, Reference(requote_band=band_value))
gp += result.realised
roc += result.roc
looks += result.looks
frozen += result.frozen_ticks
n = len(seeds)
print(f"{band_value:<8.3f}{gp / n:>12,.0f}{roc / n:>10.4f}"
f"{looks / n:>8.2f}{frozen / n:>8.2f}")
print()
if "--sweep-reference" in flags:
# The denominator has to be the profit-maximising point of its OWN family, or some
# other member earns more gp and scores less, and the reward's argmax is a pair of
# constants rather than a strategy. This is the same sweep `book.py` documents for
# the one-shot bands, over the two triggers that make the live reference act.
print("=== reference family sweep: gp per basket ===")
idles = (0.0, 0.05, 0.10, 0.15, 0.20, 0.30, 0.50, 1.01)
print(f"{HEADER:<16}" + "".join(f"{i:>10.2f}" for i in idles))
best = (0.0, None)
for band_value in (0.0, 0.005, 0.010, 0.015, 0.025, 0.040, 0.060, 1.0):
row = []
for idle in idles:
gp = sum(play(m, Reference(band_value, idle)).realised for m in seeds)
row.append(gp / len(seeds))
if row[-1] > best[0]:
best = (row[-1], (band_value, idle))
print(f"{band_value:<16.3f}" + "".join(f"{v:>10,.0f}" for v in row))
print(f"argmax {best[1]} at {best[0]:,.0f} gp per basket; shipped "
f"({REQUOTE_BAND}, {IDLE_SHARE}) is "
f"{sum(play(m, Reference()).realised for m in seeds) / len(seeds):,.0f}")
print()
print("=== the same family, in return on peak capital ===")
print(f"{HEADER:<16}" + "".join(f"{i:>10.2f}" for i in idles))
for band_value in (0.0, 0.005, 0.010, 0.015, 0.025, 0.040, 0.060, 1.0):
row = []
for idle in idles:
roc = sum(play(m, Reference(band_value, idle)).roc for m in seeds)
row.append(roc / len(seeds))
print(f"{band_value:<16.3f}" + "".join(f"{v:>10.4f}" for v in row))
print()
print("=== and what each family member SCORES against the shipped reference ===")
print(f"{HEADER:<16}" + "".join(f"{i:>10.2f}" for i in idles))
for band_value in (0.0, 0.005, 0.010, 0.015, 0.025, 0.040, 0.060, 1.0):
row = []
for idle in idles:
total = sum(score(play(m, Reference(band_value, idle)), live_reference(m))["total"]
for m in seeds)
row.append(total / len(seeds))
print(f"{band_value:<16.3f}" + "".join(f"{v:>10.3f}" for v in row))
print()
if "--sweep-freeze" in flags:
# If no freeze makes looking expensive enough for the optimum to be interior, the
# freeze is decoration and rule 4 is being carried by the denominator alone.
print("=== freeze sweep: what a look has to cost before it is worth declining ===")
print(f"{'freeze':<10}{'exhaust gp':>12}{'oracle gp':>12}{'exh score':>11}"
f"{'gap':>8}{'ora looks':>11}{'ora frozen':>12}")
for forced in (0, 1, 2, 3, 4, 6, 8, 12):
shaped = [replace(m, freeze=forced) for m in seeds]
e = g = 0.0
egp = ogp = looks = frozen = 0.0
for m in shaped:
ref = play(m, Reference())
exh = play(m, Reference(requote_band=0.0, idle_share=0.0))
if ref.realised <= 0:
continue
e += score(exh, ref)["total"]
g += 1.0 - score(exh, ref)["total"]
egp += exh.realised
ogp += ref.realised
looks += ref.looks
frozen += ref.frozen_ticks
n = len(shaped)
print(f"{forced:<10}{egp / n:>12,.0f}{ogp / n:>12,.0f}{e / n:>11.3f}"
f"{g / n:>8.3f}{looks / n:>11.2f}{frozen / n:>12.2f}")
print()
if "--sweep-shape" in flags:
print("=== window shape sweep: does a different budget bind harder? ===")
print(f"{'turns x step + tail':<22}{'horizon':>9}{'exhaust':>9}{'oracle':>9}"
f"{'gap':>9}{'ora looks':>11}{'held out':>10}")
for turns, step, tail in ((4, 14, 4), (6, 9, 6), (8, 7, 4), (8, 5, 20),
(10, 5, 10), (12, 4, 12)):
horizon = turns * step + tail
shaped = [build_market(m.seed, 5, 56, horizon) for m in seeds]
e = o = 0.0
looks = 0.0
for market in shaped:
ref = play(market, Reference(), turns=turns, step=step, tail=tail)
exh = play(market, Reference(requote_band=0.0), turns=turns, step=step,
tail=tail)
if ref.realised <= 0:
continue
e += score(exh, ref)["total"]
o += score(ref, ref)["total"]
looks += ref.looks
n = len(shaped)
print(f"{f'{turns} x {step} + {tail}':<22}{horizon:>9}{e / n:>9.3f}{o / n:>9.3f}"
f"{(o - e) / n:>9.3f}{looks / n:>11.2f}{step + tail:>10}")
print()
print(f"total wall clock {time.monotonic() - started:.1f}s")
if __name__ == "__main__":
main()