Integrate full-speed event policy with isolated judging versions

roadscore/EVENT_INTEGRATION.md

@@ -0,0 +1,84 @@
+# Local event integration candidate
+
+This candidate combines the compact UI, judging preparation/export fixes, and
+fail-closed judging preflight. No hardware validation follows from local tests.
+
+## Hardware handoff and first run
+
+Only the explicitly authorized hardware owner may execute this sequence.
+
+1. Inventory the current device revision, working changes, real offroad state,
+   owned processes, GPU/link health, mute safeguards, caches and original attempt
+   ledgers. Preserve the device's existing theme changes. Do not stop another
+   owner's processes. Reuse verified device caches before transferring assets.
+2. Stage only the reviewed source diff from `90bd992ced` to this candidate using
+   a local patch/direct transfer. Check applicability first; no remote Git push,
+   no replacement of generated files, routes, private ledgers or frozen policies.
+3. Verify actual firmware power state, not just the environment. Historical caps
+   may persist: an unset `AM_POWER_LIMIT` requests driver default but does not
+   prove the physical limit reverted. Establish the supported stock/default
+   setting and read it back before calling a run full speed. No speculative
+   resets or wattage sweeps. Preserve any reproduced link failure before using
+   an explicitly versioned practical mitigation.
+4. Prepare Prism on the device with `AM_POWER_LIMIT` removed from the environment
+   (the ACE worker no longer supplies an implicit 30 W cap). Use the established
+   regression route for the first full-speed native replay, muted, with the
+   integrated overlay. Keep UI capture/audit and the unchanged causal guard.
+5. Validate visible READY/GENERATING/DEGRADED, buffer and gesture behavior. Then
+   perform Bluetooth listening only with the user attending and explicitly
+   verified sink/volume. Do not globally remove mute safeguards.
+
+Native preparation command, only after the checks above:
+
+```sh
+env -u AM_POWER_LIMIT /usr/local/venv/bin/python /data/roadscore/prototype/worker_service.py start --composer ace --profile prism
+```
+
+Native regression invocation uses the already verified private route variable:
+
+```sh
+env -u AM_POWER_LIMIT /data/openpilot/onroad --routeid "$REGRESSION_ROUTE" --roadscore --composer ace --profile prism --muted
+```
+
+For every significant run, save requested and verified actual power mode/cap,
+preparation timing, warm generation RTF, decoder time, GPU/link samples,
+underflows, accepted-music holds and minimum playback buffer. Retain native
+summary, trace, audio callback log, UI audit and preparation/generation metadata.
+Missing telemetry is unknown, never a passing measurement. Investigate callback
+misses if reproduced and materially blocking; the old A underflow did not show a
+GPU fault or depleted buffer.
+
+## Versioned judging continuation
+
+Do not invoke the legacy batch scheduler: it uses the historical unversioned
+ledger directory. Invoke the guarded runner one ready label at a time under the
+single hardware owner's supervision.
+
+The new handoff schema requires an explicit `configuration.policy_version`,
+`hardware_mode: full-speed`, `power_limit_watts: null`, generation authorization,
+and per-label readiness with no blockers. Capped mitigation requires a distinct
+policy version and explicit numeric limit. The runner strips inherited power
+caps for full-speed mode and refuses legacy manifests for new attempts.
+
+After reconciling device cache contents, create a new private runtime handoff
+from the prepared export. Retain mapping/seeds and the explicit full-route and
+457-second range decisions. Do not relabel, tune per route, or substitute a
+partial log silently. Freeze the reviewed common runtime, quality/gesture and
+profile file hashes in `configuration.freeze.private.json` beside that handoff;
+it must be nonempty and every hash must verify before any attempt starts.
+
+New ledgers and preparations live under
+`results/community_judging/<policy_version>/`. Historical A's technically invalid
+result and B's `paused_by_user` preparation remain unchanged. B may begin its
+first replay in the new common version; A's technically invalid repeat must
+retain a provenance reference to the original. All comparable official results
+must use this same new policy; never mix the old 45 W run into a full-speed cohort.
+
+```sh
+/usr/local/venv/bin/python /data/roadscore/tools/judging_run.py "$PRIVATE_MANIFEST" --label "$LABEL"
+```
+
+A finished process is not necessarily a technical pass: require the native audit
+before listing a valid demo. Preserve invalid audio as diagnostic evidence,
+separate from the valid listening shortlist. Human ranking remains the user's.
+Live input/coexistence changes are separate and must not be staged prematurely.

roadscore/experiments/ace_chestnut_20260916/ace_worker.py

@@ -1,8 +1,7 @@
 """Opt-in, offroad resident ACE composer. No route reads or physical audio output."""
 import os
 os.environ.setdefault('TC_OPT','2')
-# Event default: unrestricted VAE execution lost GPU access; 30 W passed fixed repeats.
-os.environ.setdefault('AM_POWER_LIMIT','30')
+# Full-speed is the event default. A cap must be an explicit failure mitigation.
 import time,json,fcntl,signal,traceback,sys,resource
 BOOT=time.monotonic()
 from pathlib import Path

roadscore/prototype/worker_service.py

@@ -34,7 +34,7 @@ def offroad():
 def serve():
  STATE.parent.mkdir(exist_ok=True);lock=(STATE.parent/'worker_service.lock').open('w');fcntl.flock(lock,fcntl.LOCK_EX|fcntl.LOCK_NB)
  if not offroad():raise SystemExit('Resident preparation requires a verified offroad bench')
- record={'pid':os.getpid(),'start_ticks':stamp(os.getpid()),'started_wall':time.time(),'gpu_power_limit_watts':os.environ.get('AM_POWER_LIMIT','30' if choice()=='ace' else None),'ownership':'explicit resident service','phase':'Preparing','composer':choice(),'profile':selected() if choice()=='ace' else None};write_state(record);child=None;stop=False
+ record={'pid':os.getpid(),'start_ticks':stamp(os.getpid()),'started_wall':time.time(),'gpu_power_limit_watts':os.environ.get('AM_POWER_LIMIT'),'ownership':'explicit resident service','phase':'Preparing','composer':choice(),'profile':selected() if choice()=='ace' else None};write_state(record);child=None;stop=False
  def interrupt(*_):
   nonlocal stop;stop=True
  signal.signal(signal.SIGTERM,interrupt);signal.signal(signal.SIGINT,interrupt)

roadscore/tools/judging_run.py

@@ -3,6 +3,8 @@ import argparse
 import hashlib
 import json
 import os
+import math
+import re
 from pathlib import Path
 import subprocess
 import time
@@ -19,9 +21,9 @@ def save(path, value):
 
 
 
-def validate_frozen_configuration(root):
+def validate_frozen_configuration(root, frozen=None):
     """Refuse official work unless a nonempty local freeze verifies completely."""
-    frozen = root / 'generated/judging_configuration.private.json'
+    frozen = frozen or root / 'generated/judging_configuration.private.json'
     try:
         entries = json.loads(frozen.read_text())
     except (OSError, ValueError) as error:
@@ -53,6 +55,27 @@ def validate_handoff_readiness(manifest, row):
             raise SystemExit('Judging handoff preparation is incomplete or blocked')
 
 
+def attempt_configuration(manifest, manifest_path, root):
+    """Keep future policy evidence separate from historical attempts and freezes."""
+    config = manifest['configuration']
+    if manifest.get('schema') != 'roadscore-judging-handoff-v1':
+        raise SystemExit('Export a versioned judging handoff; historical attempts are preserved')
+    version = config.get('policy_version', '')
+    if not isinstance(version, str) or not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9_-]{0,79}', version):
+        raise SystemExit('A safe, explicit judging policy_version is required')
+    mode, cap = config.get('hardware_mode'), config.get('power_limit_watts')
+    env = os.environ.copy()
+    env.pop('AM_POWER_LIMIT', None)
+    if mode == 'full-speed' and cap is None:
+        pass
+    elif mode == 'capped' and isinstance(cap, (int, float)) and not isinstance(cap, bool) and math.isfinite(cap) and cap > 0:
+        env['AM_POWER_LIMIT'] = str(cap)
+    else:
+        raise SystemExit('Declare full-speed with no cap, or an explicit capped policy')
+    return (root / 'results/community_judging' / version,
+            manifest_path.with_name('configuration.freeze.private.json'), env)
+
+
 def main():
     parser = argparse.ArgumentParser()
     parser.add_argument('manifest', type=Path)
@@ -67,9 +90,9 @@ def main():
     if row.get('range_resolution') == 'pending metadata':
         raise SystemExit('Resolve submitted range before launching this entry')
     validate_handoff_readiness(manifest, row)
-    validate_frozen_configuration(ROOT)
-    out = ROOT / 'results/community_judging'
-    out.mkdir(exist_ok=True, mode=0o700)
+    out, frozen, env = attempt_configuration(manifest, args.manifest, ROOT)
+    validate_frozen_configuration(ROOT, frozen)
+    out.mkdir(parents=True, exist_ok=True, mode=0o700)
     ledger = out / f'official_{args.label}.json'
     if ledger.exists():
         record = json.loads(ledger.read_text())
@@ -81,8 +104,7 @@ def main():
         with ledger.open('x') as stream:
             json.dump(record, stream, indent=2)
         os.chmod(ledger, 0o600)
-    env = os.environ.copy()
-    env.update(ROADSCORE_GENERATION_SEED=str(row['seed']), AM_POWER_LIMIT='45')
+    env['ROADSCORE_GENERATION_SEED'] = str(row['seed'])
     service = ROOT / 'prototype/worker_service.py'
     try:
         if not args.resume_preparation:

roadscore/tools/test_judging_run.py

@@ -13,20 +13,21 @@ class OfficialAttemptPreflightTests(unittest.TestCase):
         self.temporary = tempfile.TemporaryDirectory()
         self.addCleanup(self.temporary.cleanup)
         self.root = Path(self.temporary.name)
-        self.freeze = self.root / 'generated/judging_configuration.private.json'
-        self.freeze.parent.mkdir()
+        self.freeze = self.root / 'configuration.freeze.private.json'
         self.policy = self.root / 'policy.json'
         self.policy.write_text('{"shared_policy": 1}')
         self.manifest = self.root / 'manifest.json'
-        self.manifest.write_text(json.dumps({'configuration': {}, 'submissions': [
-            {'label': 'A', 'seed': 123, 'route': 'synthetic/route'}]}))
+        self.manifest.write_text(json.dumps({'schema': 'roadscore-judging-handoff-v1', 'generation_authorized': True,
+            'configuration': {'policy_version': 'event-full-speed-v1', 'hardware_mode': 'full-speed', 'power_limit_watts': None},
+            'submissions': [{'label': 'A', 'seed': 123, 'route': 'synthetic/route',
+                             'preparation_ready': True, 'preparation_blockers': []}]}))
 
     def write_freeze(self):
         self.freeze.write_text(json.dumps({
             'policy.json': hashlib.sha256(self.policy.read_bytes()).hexdigest()}))
 
     def assert_preflight_rejected_without_side_effects(self):
-        ledger = self.root / 'results/community_judging/official_A.json'
+        ledger = self.root / 'results/community_judging/event-full-speed-v1/official_A.json'
         previous = ledger.read_bytes() if ledger.exists() else None
         with patch.object(judging_run, 'ROOT', self.root), patch.object(judging_run.subprocess, 'run') as run:
             with patch('sys.argv', ['judging_run', str(self.manifest), '--label', 'A']):
@@ -46,7 +47,7 @@ class OfficialAttemptPreflightTests(unittest.TestCase):
 
     def test_matching_freeze_passes_but_changed_or_missing_policy_blocks(self):
         self.write_freeze()
-        judging_run.validate_frozen_configuration(self.root)
+        judging_run.validate_frozen_configuration(self.root, self.freeze)
         self.policy.write_text('changed')
         self.assert_preflight_rejected_without_side_effects()
         self.policy.unlink()
@@ -59,7 +60,7 @@ class OfficialAttemptPreflightTests(unittest.TestCase):
                 self.assert_preflight_rejected_without_side_effects()
 
     def test_failed_preflight_preserves_existing_official_attempt(self):
-        ledger = self.root / 'results/community_judging/official_A.json'
+        ledger = self.root / 'results/community_judging/event-full-speed-v1/official_A.json'
         ledger.parent.mkdir(parents=True)
         ledger.write_text('{"phase":"finished","attempt":1,"seed":123}')
         self.assert_preflight_rejected_without_side_effects()
@@ -84,6 +85,27 @@ class OfficialAttemptPreflightTests(unittest.TestCase):
         row.update(preparation_ready=True, preparation_blockers=[])
         judging_run.validate_handoff_readiness(manifest, row)
 
+    def test_full_speed_clears_inherited_cap_and_uses_isolated_attempt_paths(self):
+        manifest = json.loads(self.manifest.read_text())
+        with patch.dict('os.environ', {'AM_POWER_LIMIT': '45'}):
+            out, frozen, env = judging_run.attempt_configuration(manifest, self.manifest, self.root)
+        self.assertNotIn('AM_POWER_LIMIT', env)
+        self.assertEqual(out, self.root / 'results/community_judging/event-full-speed-v1')
+        self.assertEqual(frozen, self.freeze)
+        manifest['configuration'].update(hardware_mode='capped', power_limit_watts=45)
+        self.assertEqual(judging_run.attempt_configuration(manifest, self.manifest, self.root)[2]['AM_POWER_LIMIT'], '45')
+
+    def test_ambiguous_power_or_policy_cannot_start_attempt(self):
+        manifest = json.loads(self.manifest.read_text())
+        for update in ({'hardware_mode': 'full-speed', 'power_limit_watts': 45},
+                       {'hardware_mode': 'capped', 'power_limit_watts': None},
+                       {'policy_version': '../old'}):
+            with self.subTest(update=update):
+                changed = json.loads(json.dumps(manifest))
+                changed['configuration'].update(update)
+                with self.assertRaises(SystemExit):
+                    judging_run.attempt_configuration(changed, self.manifest, self.root)
+
 
 if __name__ == '__main__':
     unittest.main()