Files
StarPilot/roadscore/judging

Community judging preparation

Agent 3 scope: Mac-only route preparation. This package has no SSH, device, replay, composer, worker or audio-output control. Final RoadScore generation is deliberately absent. Do not start it as part of preparation.

Current handoff

The repository guidance, STATUS.md, README.md and EVENT.md were reviewed. No applicable AGENTS.md was found in the root or RoadScore tree. The only repository AGENTS.md, under tinygrad_repo/, was also read: it specifies tinygrad test/typecheck/lint commands and prohibits commit amendments. No tinygrad changes were made. EVENT.md records the native prerequisite pass, but subjective listening and community judging remain pending. The user explicitly authorized community preparation while reserving hardware for another agent.

The user subsequently identified ROADSCORE_COMMA_HACK_7_CONTEXT.md in the integration checkout. It was read in full; its private appendix supplies the community entries. The eleven context candidates plus the explicitly reported duplicate reconstruct twelve submission instances, normalized to eleven distinct candidates, preserving the duplicate's provenance and the integration owner's existing labels and deterministic seeds. All live data, mappings, metadata and decisions are in ignored local results/. Final generation is not authorized and has not been run.

Connect semantics were checked against upstream commit 5c7f276: URL suffixes are seconds, not segment indices. Its current route reducer uses the full route when only a start suffix is supplied. The user explicitly chose that full-route behavior for the ambiguous single-timestamp submission. The explicit two-timestamp submission retains its exact end-exclusive interval. Private handoff files record the concrete ranges without exposing them in tracked documentation.

Input and normalization

Store inputs in an ignored roadscore/results/community_INPUT/submissions.private.json. Use a JSON array of route strings or objects with route, optional submitter, note, and selection fields. For example (synthetic identifiers only):

[
  {"route":"0000000000000000/2026-01-01--00-00-00", "segments":[2,3]},
  {"route":"0000000000000001|2026-01-01--00-00-00", "start_s":60, "end_s":120}
]

Supported inputs: DONGLE/ROUTE, DONGLE|ROUTE, HTTPS comma/Konik Connect route links. In bare native identifiers, --N or /N selects one segment and /START:STOP an end-exclusive segment slice. In Connect URLs, /START/END denotes seconds, and a lone positive /START remains pending intent confirmation because the current web page displays the full route. Explicit segments are sorted and deduplicated. No selection means the full route. Time bounds are route-relative seconds, with exclusive end. URL queries/fragments, open native slices, dash ranges, conflicting selections and mixed time/segment selections block preparation rather than guessing. Convert those using the submitter's intent and retain the original text/decision in note. Never infer a time range from a URL without checking its semantics.

Exact normalized route+selection duplicates collapse, retaining submission references. Different selections of the same route remain distinct and require an explicit --acknowledge-same-route-selections decision; the tool does not silently merge them. Eleven unique selections are required for A–K. A failed attempt writes only its normalization report; correct the source and use a fresh output directory.

python3 roadscore/judging/prepare.py prepare \
  --submissions roadscore/results/community_INPUT/submissions.private.json \
  --output roadscore/results/community_BATCH

The private manifest preserves source checksum, exact selection and provenance. Use --existing-manifest PATH to preserve a previously issued mapping and seeds. Otherwise a random permutation is drawn once with SystemRandom and frozen on disk. Seeds always derive from SHA256 of roadscore-judging-v1: plus canonical route, first four bytes big endian; labels and submission order do not affect seeds. The tool refuses an existing output directory. The blind batch ID hashes the private mapping; no identity or submitter is included in the blind folder. Commit only source; all batch outputs belong under ignored results/. Do not publish recordings or mappings.

Cache and characterize on the Mac

Local cache sources are read-only. Copies are written under the chosen batch only; no existing score archives, driver camera, weights or generation assets are copied.

python3 roadscore/judging/prepare.py cache \
  --manifest roadscore/results/community_BATCH/manifest.private.json \
  --source /Users/dominickthompson/Desktop/RoadScore/routes

Missing rlog/front camera and segment holes remain explicit failures. qlog/qcamera are retained as diagnostic alternatives but do not satisfy the full-data gate. Each copied file has byte count and SHA-256; corrupt cached copies are repaired from the source. Local presence cannot prove remote EOF, so extent stays unverified.

For uncached routes, run the following using an existing Mac Python environment with openpilot imports, native API authentication and requests available:

python3 roadscore/judging/prepare.py metadata \
  --manifest roadscore/results/community_BATCH/manifest.private.json
python3 roadscore/judging/prepare.py fetch --logs-only \
  --manifest roadscore/results/community_BATCH/manifest.private.json
python3 roadscore/judging/analyze.py \
  --manifest roadscore/results/community_BATCH/manifest.private.json

Fetch uses the repository's authenticated route API host fallback and HTTPS assets. It downloads selected full logs and front camera, reuses verified files, and saves no tokens or signed URLs. Authentication, access and download failures are reported by exception type without leaking signed URL text. A fetched listing proves only the available remote extent, not that the owner uploaded every recorded segment. The default free-space reserve is 8 GiB, checked before and during each download. Use --logs-only for characterization first, --preview-video for smaller qcamera previews, or omit both for full front video when space allows. Missing rlogs fall back to qlogs for partial characterization; they remain failures of the full-log gate. Metadata/analysis do not declare missing video ready. No hardware access or score generation follows fetch.

Time-to-segment conversion is nominal 60 seconds. Remote fetch also obtains segment zero as a log-only clock reference for time selections. Analysis uses original log timestamps and an end-exclusive cut; the reference segment never contributes to a later excerpt's statistics. Local-only caches lacking segment zero remain blocked. Camera decode and timestamp alignment still require later replay validation.

Analysis checks original log hashes, inventories valid message types/rates/gaps, records timestamp reversals, speed/steering distributions and navigation/blinker sample coverage. The private reports remain outside the blind folder. These are technical proxies, not route quality rankings. They must never be imported into the causal music runtime or used to select a favorable seed. Missing data is not a pass.

Blind judging and later handoff

Open blind/index.html locally and load its adjacent batch.json. The page uses no network requests and supports A–K navigation, explicit local media attachment, 1–5 / not-observable ratings, notes, browser-local save and JSON export tied to the batch ID and anonymous judging session. Playback never autostarts. Browser storage may be unavailable on file URLs; export remains available. Reopened recordings must be reattached and acknowledged as reviewed. Blank scores are not zeros. Only give judges the blind folder and deliberately anonymized approved recordings; never serve the parent batch directory. Road footage itself can reveal recognizable locations.

Before the hardware owner later runs final generation, review each private selection, cache completeness/extent, message coverage and clock ambiguity. Use the same Prism configuration, quality/gesture policy and deterministic route-derived seed policy consistently across candidates. One official run per candidate is allowed unless technically invalid; automatic normal quality rerolls remain allowed. Preserve failed runs; do not choose a winning seed from offline route characterization. Use the existing event replay audit for timing/causality evidence and collect human ratings separately. Keep identity reveal and any winner decision deferred until judging ends.

Validation

python3 -m unittest discover -s roadscore/judging -p 'test_*.py'

Tests cover normalization/deduplication, explicit range semantics, frozen blind mapping, privacy boundary, missing/corrupt cache data, and technical analysis validity/time cuts. No real routes, network, device access or music generation are needed for tests.

Nine synthetic preparation/cache/analysis tests pass, including resumable authenticated fetch mocks and Connect/native range distinctions. Python compilation and JavaScript syntax checks pass. Interactive browser verification was not completed: the browser security policy rejected the local file URL. No workaround or audio playback was used.

Offline integration export

export_handoff.py adapts the preserved preparation manifest to the event runner's submissions, start_seconds, and duration_seconds schema. It preserves the existing labels, seeds and configuration, and rejects unresolved ranges. Always choose a fresh ignored output directory; it never overwrites integration manifests or official-attempt ledgers.

python3 roadscore/judging/export_handoff.py \
  --manifest roadscore/results/community_BATCH/manifest.private.json \
  --output roadscore/results/community_EXPORT

The default verifies cached hashes and writes a private transfer plan, without copying assets. --materialize copies those verified assets to a new portable routes/ native layout, retaining an 8 GiB disk reserve (--reserve-gib). It does not access the device or download files. Copy failures preserve originals and remove partial copies. Local prepare.py cache copies enforce the same reserve. Fetch retries preserve unprocessed inventory even if a download fails midway.

A native cache_manifest.json is emitted only for complete selected full logs, route-zero clock reference, full front camera, verified extent and completed analysis. Incomplete materialized caches carry .acquiring and cannot pass the normal complete-cache check. qlogs/previews remain diagnostics. Excerpt caches cover the selected interval and reference only; their manifest does not claim whole-route completeness.

The export always sets generation_authorized: false. Its schema is roadscore-judging-handoff-v1; the runner must reject it unless generation is explicitly authorized, preparation_ready is true, and preparation_blockers is empty. Frozen configuration, hardware handoff, physical/cache transfer and existing official-attempt reconciliation still belong to the integration owner. Neither exporting nor cache materialization grants generation permission. Never replace an existing attempt merely to adopt this schema.