Files
arena/tests/test_probe.py
T
kartiandClaude Opus 5 16f4ee21ae
arena-environments / validate (3.11) (push) Failing after 5s
arena-environments / validate (3.12) (push) Failing after 5s
Extend the merged CI and probe test to all seven environments
The redaction-pressure hardening landed a test asserting probe.py gates every
environment and a CI step building every wheel, and both carried a hardcoded
list of four. Merging three more environments in left the assertion counting
four "ok" lines against seven, and left the new wheels unbuilt by CI.

Both lists now name all seven. The count assertion is kept rather than relaxed:
a new environment missing from the list fails the build instead of quietly
narrowing what the test covers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 03:58:25 -07:00

47 lines
1.4 KiB
Python

from __future__ import annotations
import subprocess
import sys
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
# Every environment probe.py gates. A new environment that is not added here still has to
# print "ok", because the count assertion below counts them — so this list going stale fails
# the build rather than quietly narrowing what the test covers.
ENVIRONMENTS = (
"redaction-pressure",
"canary-trap",
"fault-localisation",
"schema-migration",
"bot-detection",
"grand-exchange",
"drop-table-inference",
)
class RootProbeIntegrationTests(unittest.TestCase):
def test_public_root_probe_gates_every_environment(self) -> None:
completed = subprocess.run(
[sys.executable, "probe.py"],
cwd=ROOT,
text=True,
capture_output=True,
timeout=30,
check=False,
)
self.assertEqual(
completed.returncode,
0,
msg=f"root probe failed:\nstdout:\n{completed.stdout}\nstderr:\n{completed.stderr}",
)
for environment in ENVIRONMENTS:
with self.subTest(environment=environment):
self.assertIn(environment, completed.stdout)
self.assertEqual(completed.stdout.count(" ok"), len(ENVIRONMENTS))
if __name__ == "__main__":
unittest.main()