karti-ai
eb88138d15
Five parallel lanes plus an integration pass. The header, gallery, router and sitemap are all generated from the demo registry, so adding src/demos/<slug>/ puts a demo everywhere with zero edits to shared files — which is the whole reason demo nine cannot break demo one. check-demos enforces the twelve contract rules: 142 checks over one live demo. Two worth naming. The shell may not mention a specific slug, because an 'if (slug === wordle)' in src/components/demo/ is a contract bug wearing a patch. And a spec-status demo must ship a real specification — task, actions, grader, counterweight, eval command — since a coming-soon card reads worse than an honest empty gallery. Bundle budget holds: entry 108.79 kB gzipped against a 160 kB ceiling, the demo chunk 21.15 kB against 90 kB. recharts is 108 kB gzipped and lives behind a lazy import so it never touches the entry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mt6sHQHEnEYrJZvoMCJSB
54 lines
1.9 KiB
Python
54 lines
1.9 KiB
Python
"""The reward this demo quotes, as a runnable placeholder.
|
|
|
|
A real demo points `reward.source.path` at its environment package — e.g.
|
|
`envs/__package__/__package__/reward.py` — and quotes the function the grader
|
|
actually runs. This file exists so a freshly scaffolded demo has a receipt that
|
|
RESOLVES on day one: an empty receipt panel reads to a visitor as the code not
|
|
existing, which is the exact impression this site is built to avoid.
|
|
|
|
Move the region markers into the environment and repoint `source.path` as soon
|
|
as the environment lands. `scripts/check-receipts.mjs` will tell you the moment
|
|
the two disagree.
|
|
"""
|
|
|
|
from dataclasses import dataclass
|
|
|
|
|
|
@dataclass
|
|
class Episode:
|
|
"""One rollout, as the environment records it."""
|
|
|
|
escalated: frozenset[str]
|
|
needed: frozenset[str]
|
|
malformed_actions: int
|
|
|
|
|
|
# region: pig-demo/score
|
|
def score(episode: Episode) -> dict[str, float]:
|
|
"""Three terms, weighted 0.60 / 0.25 / 0.15 in the demo's RewardSpec.
|
|
|
|
`caught` is the objective. `restraint` is the counterweight: it is what
|
|
stops the objective being maximised the crude way, by escalating the whole
|
|
queue. `well_formed` is a gate — every competent policy scores 1.0 on it,
|
|
so it is declared a gate rather than dressed up as a second counterweight.
|
|
"""
|
|
needed = episode.needed
|
|
escalated = episode.escalated
|
|
|
|
caught = len(escalated & needed) / len(needed) if needed else 1.0
|
|
|
|
noise = escalated - needed
|
|
quiet = len(escalated) - len(noise)
|
|
restraint = 1.0 - (len(noise) / len(escalated)) if escalated else 1.0
|
|
|
|
well_formed = 0.0 if episode.malformed_actions else 1.0
|
|
|
|
return {
|
|
"caught": caught,
|
|
"restraint": restraint,
|
|
"well_formed": well_formed,
|
|
# Unweighted diagnostic. Rendered, never summed into the reward.
|
|
"escalations_that_landed": float(quiet),
|
|
}
|
|
# endregion: pig-demo/score
|