Community judging preparation
Agent 3 scope: Mac-only route preparation. This package has no SSH, device, replay, composer, worker or audio-output control. Final RoadScore generation is deliberately absent. Do not start it as part of preparation.
Current handoff
The repository guidance, STATUS.md, README.md and EVENT.md were reviewed. No applicable AGENTS.md was found in the root or RoadScore tree. The only repository AGENTS.md, under tinygrad_repo/, was also read: it specifies tinygrad test/typecheck/lint commands and prohibits commit amendments. No tinygrad changes were made. EVENT.md records the native prerequisite pass, but subjective listening and community judging remain pending. The user explicitly authorized community preparation while reserving hardware for another agent.
The user subsequently identified ROADSCORE_COMMA_HACK_7_CONTEXT.md in the integration
checkout. It was read in full; its private appendix supplies the community entries.
The eleven context candidates plus the explicitly reported duplicate reconstruct
twelve submission instances, normalized to eleven distinct candidates, preserving the duplicate's
provenance and the integration owner's existing labels and deterministic seeds.
All live data, mappings, metadata and decisions are in ignored local results/.
Final generation is not authorized and has not been run.
Connect semantics were checked against upstream commit
5c7f276:
URL suffixes are seconds, not segment indices. Its current
route reducer
uses the full route when only a start suffix is supplied. The user explicitly chose
that full-route behavior for the ambiguous single-timestamp submission. The explicit
two-timestamp submission retains its exact end-exclusive interval. Private handoff
files record the concrete ranges without exposing them in tracked documentation.
Input and normalization
Store inputs in an ignored roadscore/results/community_INPUT/submissions.private.json.
Use a JSON array of route strings or objects with route, optional submitter, note,
and selection fields. For example (synthetic identifiers only):
[
{"route":"0000000000000000/2026-01-01--00-00-00", "segments":[2,3]},
{"route":"0000000000000001|2026-01-01--00-00-00", "start_s":60, "end_s":120}
]
Supported inputs: DONGLE/ROUTE, DONGLE|ROUTE, HTTPS comma/Konik Connect route links. In bare native identifiers,
--N or /N selects one segment and /START:STOP an end-exclusive segment slice.
In Connect URLs, /START/END denotes seconds, and a lone positive /START
remains pending intent confirmation because the current web page displays the full route.
Explicit segments are sorted and deduplicated. No selection means the full route.
Time bounds are route-relative seconds, with exclusive end. URL queries/fragments,
open native slices, dash ranges, conflicting selections and mixed time/segment selections
block preparation rather than guessing. Convert those using the submitter's intent
and retain the original text/decision in note. Never infer a time range from a URL
without checking its semantics.
Exact normalized route+selection duplicates collapse, retaining submission references.
Different selections of the same route remain distinct and require an explicit
--acknowledge-same-route-selections decision; the tool does not silently merge them.
Eleven unique selections are required for A–K. A failed attempt writes only its
normalization report; correct the source and use a fresh output directory.
python3 roadscore/judging/prepare.py prepare \
--submissions roadscore/results/community_INPUT/submissions.private.json \
--output roadscore/results/community_BATCH
The private manifest preserves source checksum, exact selection and provenance.
Use --existing-manifest PATH to preserve a previously issued mapping and seeds.
Otherwise a random permutation is drawn once with SystemRandom and frozen on disk.
Seeds always derive from SHA256 of roadscore-judging-v1: plus canonical route,
first four bytes big endian; labels and submission order do not affect seeds. The tool
refuses an existing output directory. The blind batch ID hashes the private mapping;
no identity or submitter is included in the blind folder. Commit only source; all
batch outputs belong under ignored results/. Do not publish recordings or mappings.
Cache and characterize on the Mac
Local cache sources are read-only. Copies are written under the chosen batch only; no existing score archives, driver camera, weights or generation assets are copied.
python3 roadscore/judging/prepare.py cache \
--manifest roadscore/results/community_BATCH/manifest.private.json \
--source /Users/dominickthompson/Desktop/RoadScore/routes
Missing rlog/front camera and segment holes remain explicit failures. qlog/qcamera are retained as diagnostic alternatives but do not satisfy the full-data gate. Each copied file has byte count and SHA-256; corrupt cached copies are repaired from the source. Local presence cannot prove remote EOF, so extent stays unverified.
For uncached routes, run the following using an existing Mac Python environment with openpilot imports, native API authentication and requests available:
python3 roadscore/judging/prepare.py metadata \
--manifest roadscore/results/community_BATCH/manifest.private.json
python3 roadscore/judging/prepare.py fetch --logs-only \
--manifest roadscore/results/community_BATCH/manifest.private.json
python3 roadscore/judging/analyze.py \
--manifest roadscore/results/community_BATCH/manifest.private.json
Fetch uses the repository's authenticated route API host fallback and HTTPS assets.
It downloads selected full logs and front camera, reuses verified files,
and saves no tokens or signed URLs. Authentication, access and download failures are
reported by exception type without leaking signed URL text. A fetched listing proves
only the available remote extent, not that the owner uploaded every recorded segment.
The default free-space reserve is 8 GiB, checked before and during each download.
Use --logs-only for characterization first, --preview-video for smaller qcamera
previews, or omit both for full front video when space allows. Missing rlogs fall back
to qlogs for partial characterization; they remain failures of the full-log gate. Metadata/analysis do not declare missing video ready. No hardware access or
score generation follows fetch.
Time-to-segment conversion is nominal 60 seconds. Remote fetch also obtains segment zero as a log-only clock reference for time selections. Analysis uses original log timestamps and an end-exclusive cut; the reference segment never contributes to a later excerpt's statistics. Local-only caches lacking segment zero remain blocked. Camera decode and timestamp alignment still require later replay validation.
Analysis checks original log hashes, inventories valid message types/rates/gaps, records timestamp reversals, speed/steering distributions and navigation/blinker sample coverage. The private reports remain outside the blind folder. These are technical proxies, not route quality rankings. They must never be imported into the causal music runtime or used to select a favorable seed. Missing data is not a pass.
Blind judging and later handoff
Open blind/index.html locally and load its adjacent batch.json. The page uses no
network requests and supports A–K navigation, explicit local media attachment,
1–5 / not-observable ratings, notes, browser-local save and JSON export tied to the
batch ID and anonymous judging session. Playback never autostarts. Browser storage
may be unavailable on file URLs; export remains available. Reopened recordings must
be reattached and acknowledged as reviewed. Blank scores are not zeros. Only give
judges the blind folder and deliberately anonymized approved recordings; never serve
the parent batch directory. Road footage itself can reveal recognizable locations.
Before the hardware owner later runs final generation, review each private selection, cache completeness/extent, message coverage and clock ambiguity. Use the same Prism configuration, quality/gesture policy and deterministic route-derived seed policy consistently across candidates. One official run per candidate is allowed unless technically invalid; automatic normal quality rerolls remain allowed. Preserve failed runs; do not choose a winning seed from offline route characterization. Use the existing event replay audit for timing/causality evidence and collect human ratings separately. Keep identity reveal and any winner decision deferred until judging ends.
Validation
python3 -m unittest discover -s roadscore/judging -p 'test_*.py'
Tests cover normalization/deduplication, explicit range semantics, frozen blind mapping, privacy boundary, missing/corrupt cache data, and technical analysis validity/time cuts. No real routes, network, device access or music generation are needed for tests.
Nine synthetic preparation/cache/analysis tests pass, including resumable authenticated fetch mocks and Connect/native range distinctions. Python compilation and JavaScript syntax checks pass. Interactive browser verification was not completed: the browser security policy rejected the local file URL. No workaround or audio playback was used.
Offline integration export
export_handoff.py adapts the preserved preparation manifest to the event runner's
submissions, start_seconds, and duration_seconds schema. It preserves the
existing labels, seeds and configuration, and rejects unresolved ranges. Always
choose a fresh ignored output directory; it never overwrites integration manifests
or official-attempt ledgers.
python3 roadscore/judging/export_handoff.py \
--manifest roadscore/results/community_BATCH/manifest.private.json \
--output roadscore/results/community_EXPORT
The default verifies cached hashes and writes a private transfer plan, without
copying assets. --materialize copies those verified assets to a new portable
routes/ native layout, retaining an 8 GiB disk reserve (--reserve-gib). It does
not access the device or download files. Copy failures preserve originals and
remove partial copies. Local prepare.py cache copies enforce the same reserve.
Fetch retries preserve unprocessed inventory even if a download fails midway.
A native cache_manifest.json is emitted only for complete selected full logs,
route-zero clock reference, full front camera, verified extent and completed
analysis. Incomplete materialized caches carry .acquiring and cannot pass the
normal complete-cache check. qlogs/previews remain diagnostics. Excerpt caches
cover the selected interval and reference only; their manifest does not claim
whole-route completeness.
The export always sets generation_authorized: false. Its schema is
roadscore-judging-handoff-v1; the runner must reject it unless generation is
explicitly authorized, preparation_ready is true, and preparation_blockers is
empty. Frozen configuration, hardware handoff, physical/cache transfer and existing
official-attempt reconciliation still belong to the integration owner. Neither
exporting nor cache materialization grants generation permission. Never replace
an existing attempt merely to adopt this schema.