Polish RoadScore overlay states and document the demo

roadscore/DEMO.md

@@ -0,0 +1,43 @@
+# RoadScore demo
+
+RoadScore makes an adaptive soundtrack from road context. The native demo shows a recorded drive in the normal comma UI while ACE-Step generates new musical passages on Chestnut. Prism is the prepared musical profile. RoadScore supplies the native port, musical continuation, buffering, quality checks, immediate gestures and synchronization; ACE-Step is a pretrained model, not a model trained by this project.
+
+The conductor receives only road messages already delivered by replay. It requests future passages and uses immediate, source-derived gestures for short road events. Generation takes time: the buffer lets the current score continue while the next passage is being made. A curve gesture does not imply a complete new neural composition in that instant.
+
+## Reading the overlay
+
+The compact lower-left panel leaves the existing camera, lane and path display in place. Its rows show RoadScore's activity, profile, section, compute backend and buffered music.
+
+| Display | Meaning |
+| --- | --- |
+| Prism / Aurora | Reported prepared musical profile. Prism is the primary candidate. |
+| INTENT: VERSE > PRECHORUS | Current section conditioning and the next scheduled section, when provided by the runtime. These are musical intentions, not verified structural labels. |
+| ACE · CHESTNUT | Reported composer and compute backend. This is not an independent GPU health probe. |
+| PREPARING | The status does not yet report playback readiness. Cold preparation is separate from warm generation time. |
+| READY | The runtime reports ready and no degraded condition is present. This does not certify listening quality or live-driving readiness. |
+| GENERATING | A fresh passage is in flight. The elapsed time is job time, not an estimated countdown. Buffered music may continue playing. |
+| DEGRADED | The runtime reports a failure, quality issue or accepted-music hold. This label takes priority even if a generation job is still in flight. |
+| Holding accepted music | Previously accepted music is being reused. It is not fresh generation. |
+| BUFFER 64s | Reported queued playback time. It can include accepted-music holds; it is not a fresh-music counter. `--` means unavailable. |
+| STORED SCORE · NO COMPUTE | Archived score playback. It must not be presented as fresh Chestnut generation. |
+
+Native text uses ASCII labels and measured truncation to avoid unsupported glyphs or text spilling outside the panel. The middle dot in the backend label is drawn geometrically. Long section labels end with `...`; full source values remain in the existing status/audit JSON.
+
+## Presenter explanation
+
+“Prism is the musical identity. The recorded road messages guide the score as they arrive. ACE generates future passages locally on Chestnut, while the buffer keeps playback moving. Short gestures can react sooner than a new generated passage. READY means the runtime is ready; GENERATING means a new passage is being made. If generation cannot keep up or a quality check fails, DEGRADED makes that visible, and the system can reuse accepted music.”
+
+Use [EVENT.md](EVENT.md) for measured event claims. The recorded 45 W native pass was muted and does not establish physical audio, Bluetooth, live driving or modeld coexistence. Historical performance reports describe a different bench. Operational commands belong in [README.md](README.md#run-on-the-event-comma) and are for the agent/operator who owns the hardware.
+
+## Offline UI review
+
+From the repository root, with a Python environment containing Pillow:
+
+```sh
+python3 -m unittest discover -s roadscore/tests -v
+python3 roadscore/tools/preview_overlay.py --output roadscore/results/ui-polish/overlay-states.png
+```
+
+The preview renders six synthetic states with the repository's Inter bitmap font atlas: preparation, ready, generating, accepted-music hold, failed composer and stored score. It uses CPU image drawing and does not open a window, contact a device, start a worker, run replay or open an audio stream. It is a layout review, not a native-rendering or hardware-validation claim. The output stays in ignored `results/`.
+
+The active native overlay is `prototype/overlay.py`; the pure presentation helper is `prototype/overlay_view.py`. `prototype/index.html` and `prototype/native_display.py` are older excerpt-specific bench interfaces, not the normal onroad demo. The historical style selector is not an ACE profile selector. Keep those older entrypoints distinct when presenting the current demo.

roadscore/README.md

@@ -10,11 +10,23 @@ recorded cereal → semantic road state → section requests → ACE on Chestnut
 
 Only already-delivered messages enter musical decisions. The ordinary replay engine owns route resolution, camera video, path/lane display, and playback timing. Existing generated music may be held when fresh material is unavailable; holds are recorded and are not counted as new generation.
 
+## Demo and source guide
+
+- [Demo explanation and overlay legend](DEMO.md): what the audience sees and what each state means.
+- [Event evidence](EVENT.md): measurements and preserved failures on the event hardware; read the latest checkpoint.
+- [Status history](STATUS.md): event summary followed by explicitly historical pre-event results.
+- `prototype/overlay.py` and `prototype/overlay_view.py`: native overlay integration and read-only presentation.
+- `prototype/`: score runtime, composition, gestures, transport and archival code.
+- `tools/`: development and event utilities; [offline UI preview](DEMO.md#offline-ui-review) needs no device.
+- `tests/`: isolated UI tests. Existing runtime tests remain beside their modules and in `experiments/`.
+- `experiments/`, `chestnut_music/`, `chestnut_stable_audio/`: research and historical backend work.
+- `routes/`, `results/`, model assets: private local data, excluded from Git.
+
 ## Event status
 
 Source lives in this repository on the `RoadScore` branch and on the event comma at `/data/openpilot/roadscore`. `/data/roadscore` is a compatibility symlink. See [EVENT.md](EVENT.md) for current measurements and preserved failures.
 
-The official Chestnut example passed. Unrestricted decoder runs lost GPU access; a 30 W cap passed 40 repeated decodes and full Prism preparation. At that cap, warm continuation takes approximately 44s per 28s of new music (RTF 1.57). A 45 W comparison is underway. **The event native-replay acceptance gate is not yet complete.** Live/modeld coexistence and physical speaker/Bluetooth validation remain pending.
+The latest recorded checkpoint in [EVENT.md](EVENT.md#first-clean-event-native-replay--45-w) reports a complete 254.1-second muted native Prism replay at an explicitly selected 45 W: six accepted fresh jobs, no holds, no underflows and no worker failure. This is one instrumented pass, not a production-reliability claim. Live/modeld coexistence and physical speaker/Bluetooth validation remain pending. Earlier 30 W results and failed runs are preserved in that report.
 
 ## Run on the event comma
 

roadscore/prototype/overlay.py

@@ -1,12 +1,13 @@
 """Optional external overlay on the existing normal UI; no public source modifications."""
 import json,time,os
+from overlay_view import draw_panel
 from pathlib import Path
 
 def install():
  if os.environ.get('ROADSCORE_OVERLAY')!='1':return
  import pyray as rl
  from openpilot.system.ui.lib.application import gui_app,FontWeight
- original=gui_app.render;last=0.;state={};icons={};frames=0;captured=False;capture_ready_since=None
+ original=gui_app.render;last=0.;state={};frames=0;captured=False;capture_ready_since=None
  path=Path(os.environ['ROADSCORE_STATUS_FILE'])
  def draw():
   nonlocal last,state,frames,captured,capture_ready_since
@@ -15,36 +16,13 @@ def install():
    try:state=json.loads(path.read_text())
    except (OSError,ValueError):pass
    last=now
-  readiness=state.get('readiness','Preparing');ready=readiness=='READY'
-  extra=bool(state.get('gesture_active') or state.get('gesture_queued') or state.get('turn_signal_music'));height=126 if extra else 104
-  x=12;y=max(12,gui_app.height-height-12);width=min(320,gui_app.width*.55)
-  rl.draw_rectangle_rounded(rl.Rectangle(x,y,width,height),.12,8,rl.Color(15,22,31,220))
-  key='crossed' if state.get('compute')=='none' else ('green' if ready else 'loading')
-  if key not in icons:
-   try:icons[key]=gui_app.texture('icons_mici/egpu_'+key+'.png',24,18)
-   except Exception:icons[key]=None
-  if icons[key]:rl.draw_texture(icons[key],x+10,y+10,rl.WHITE)
-  font=gui_app.font(FontWeight.NORMAL)
-  lines=[f"RoadScore / {state.get('style','Preparing')}",state.get('section',readiness)]
-  if state.get('next_section'):lines[1]+=' > '+state['next_section']
-  lead=state.get('lead');detail=f'Curve {lead:.1f}s / ' if isinstance(lead,(int,float)) and lead>0 else ''
-  detail+=f"{readiness} / Buffer {(state.get('buffered') or 0):.0f}s"
-  elapsed=state.get('generation_elapsed_seconds')
-  backend='ACE / CHESTNUT' if state.get('composer')=='ace' else str(state.get('composer','LOCAL')).upper()
-  activity='GENERATING' if state.get('job_inflight') else readiness
-  if state.get('job_inflight') and isinstance(elapsed,(int,float)):activity+=f' {elapsed:.1f}s'
-  lines.append(f'{backend} / {activity}')
-  lines.append(detail)
-  if extra:
-   active=state.get('gesture_active') or [];queued=state.get('gesture_queued') or []
-   gesture='Signal percussion' if state.get('turn_signal_music') else (active[0] if active else ('Queued '+queued[0]['kind'] if queued else ''))
-   lines.append(str(gesture).replace('_',' '))
-  for i,line in enumerate(lines):rl.draw_text_ex(font,str(line),rl.Vector2(x+(42 if i==0 else 10),y+10+i*22),13,0,rl.WHITE)
+  view=draw_panel(rl,gui_app.font(FontWeight.NORMAL),state,gui_app.width,gui_app.height)
+  ready=view['ready']
   frames+=1
   if frames==1:path.with_name('ui_capture_origin.json').write_text(json.dumps({'wall':time.monotonic(),'frame':0}))
   if os.environ.get('ROADSCORE_CAPTURE_TIMING')=='1':
    with path.with_name('ui_frames.jsonl').open('a') as audit:audit.write(json.dumps({'frame':frames-1,'wall':time.monotonic(),'state':state})+'\n')
-  if frames%60==1:path.with_name('overlay_status.json').write_text(json.dumps({'frames':frames,'native_gpu_icon':bool(icons[key]),'state':state}))
+  if frames%60==1:path.with_name('overlay_status.json').write_text(json.dumps({'frames':frames,'native_gpu_icon':False,'presentation':view,'state':state}))
   from openpilot.selfdrive.ui.ui_state import ui_state
   if ready and ui_state.started and capture_ready_since is None:capture_ready_since=now
   if capture_ready_since is not None and now-capture_ready_since>=5 and ui_state.started and not captured and os.environ.get("ROADSCORE_OVERLAY_CAPTURE"):

roadscore/prototype/overlay_view.py

@@ -0,0 +1,105 @@
+"""Read-only presentation of RoadScore status; no runtime or device dependencies."""
+import math
+import unicodedata
+
+
+def display_text(value):
+  """The native bitmap font reliably supports ASCII; draw separators as shapes."""
+  text = str(value or '').replace('→', ' > ').replace('·', ' / ').replace('—', '-').replace('–', '-')
+  return ' '.join(unicodedata.normalize('NFKD', text).encode('ascii', 'ignore').decode().split())
+
+
+def seconds(value):
+  if isinstance(value, (int, float)) and not isinstance(value, bool) and math.isfinite(value):
+    return max(0., float(value))
+  return None
+
+
+def overlay_view(state):
+  readiness = str(state.get('readiness') or 'PREPARING').upper()
+  degraded = readiness == 'DEGRADED' or any(state.get(k) for k in ('worker_failed', 'holding_accepted_music', 'quality_failures'))
+  activity = 'DEGRADED' if degraded else ('GENERATING' if state.get('job_inflight') else readiness)
+  if activity not in ('READY', 'GENERATING', 'DEGRADED'):
+    activity = 'PREPARING'
+  stored = state.get('compute') == 'none'
+  composer = display_text(state.get('composer')).upper()
+  backend = ('STORED SCORE', 'NO COMPUTE') if stored else ((composer, 'CHESTNUT') if composer in ('ACE', 'SA3') else ('LOCAL SCORE', ''))
+  profile = state.get('style') or str(state.get('profile') or 'Preparing').title()
+  section = display_text(state.get('section') or 'Waiting for score').upper()
+  section = section.removesuffix(' / CONTINUOUS')
+  if state.get('next_section'):
+    section += ' > ' + display_text(state['next_section']).upper()
+  if state.get('form_labels_are_intent'):
+    section = 'INTENT: ' + section
+  elapsed = seconds(state.get('generation_elapsed_seconds'))
+  note = ''
+  if state.get('holding_accepted_music'):
+    note = 'Holding accepted music'
+  elif state.get('worker_failed'):
+    note = 'Composer unavailable'
+  elif degraded:
+    note = 'Quality check / reserve in use'
+  elif state.get('job_inflight'):
+    note = 'New passage' + (f' / {elapsed:.1f}s elapsed' if elapsed is not None else '')
+  event = ''
+  if state.get('turn_signal_music'):
+    event = 'Signal percussion'
+  elif state.get('gesture_active'):
+    event = display_text(state['gesture_active'][0]).replace('_', ' ')
+  elif state.get('gesture_queued'):
+    event = 'Queued ' + display_text(state['gesture_queued'][0].get('kind')).replace('_', ' ')
+  elif (lead := seconds(state.get('lead'))) is not None and lead > 0:
+    event = f'Curve in {lead:.1f}s'
+  if event:
+    note = f'{note} / {event}' if note else event
+  return dict(profile=display_text(profile), section=section, backend=backend, activity=activity,
+              buffered=seconds(state.get('buffered')), note=note,
+              ready=readiness == 'READY' and not degraded, stored=stored)
+
+
+def fit_text(text, max_width, measure):
+  """Fit with the actual font metrics, using a supported ellipsis."""
+  if measure(text) <= max_width + .01:
+    return text
+  while text and measure(text + '...') > max_width + .01:
+    text = text[:-1]
+  return text.rstrip() + '...' if text else ''
+
+
+def draw_panel(rl, font, state, screen_width, screen_height):
+  view = overlay_view(state)
+  width = min(340, screen_width - 24)
+  height = 146 if view['note'] else 126
+  x, y = 12, max(12, screen_height - height - 12)
+  color = {'READY': (123, 229, 193), 'GENERATING': (131, 192, 255),
+           'DEGRADED': (255, 197, 112), 'PREPARING': (180, 190, 204)}[view['activity']]
+  accent = rl.Color(*color, 255)
+  muted = rl.Color(180, 190, 204, 255)
+  rl.draw_rectangle_rounded(rl.Rectangle(x, y, width, height), .12, 8, rl.Color(15, 22, 31, 235))
+
+  def text(label, left, top, size=13, tint=rl.WHITE, available=None):
+    available = width - left - 12 if available is None else available
+    label = fit_text(label, available, lambda s: rl.measure_text_ex(font, s, size, 0).x)
+    rl.draw_text_ex(font, label, rl.Vector2(x + left, y + top), size, 0, tint)
+
+  badge_width = rl.measure_text_ex(font, view['activity'], 11, 0).x + 16
+  rl.draw_rectangle_rounded(rl.Rectangle(x + width - badge_width - 10, y + 10, badge_width, 22), .4, 8, rl.Color(*color, 28))
+  text(view['activity'], width - badge_width - 2, 15, 11, accent)
+  text('RoadScore', 12, 12, 17, available=width - badge_width - 34)
+  text(view['profile'], 12, 39, 17)
+  text(view['section'], 12, 62, 12, muted)
+  left, right = view['backend']
+  text(left, 12, 83, 11, muted)
+  if right:
+    offset = 12 + rl.measure_text_ex(font, left, 11, 0).x
+    # A geometric middle dot avoids missing-glyph boxes in the native font.
+    rl.draw_circle(int(x + offset + 7), int(y + 89), 1.5, muted)
+    text(right, offset + 15, 83, 11, muted)
+  buffer = view['buffered']
+  text('BUFFER', 12, 105, 11, muted)
+  label = '--' if buffer is None else f'{buffer:.0f}s'
+  label_width = rl.measure_text_ex(font, label, 13, 0).x
+  text(label, width - label_width - 12, 103, 13, accent)
+  if view['note']:
+    text(view['note'], 12, 126, 11, accent)
+  return view

roadscore/tests/test_overlay_view.py

@@ -0,0 +1,51 @@
+"""Presentation contracts only; these tests never import the runtime or device tools."""
+import sys
+import unittest
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'prototype'))
+from overlay_view import display_text, fit_text, overlay_view
+
+
+class OverlayTests(unittest.TestCase):
+  def test_degradation_takes_priority_over_generation(self):
+    for reason in ('worker_failed', 'holding_accepted_music', 'quality_failures'):
+      with self.subTest(reason=reason):
+        view = overlay_view(dict(readiness='READY', job_inflight=True, **{reason: True}))
+        self.assertEqual(view['activity'], 'DEGRADED')
+        self.assertFalse(view['ready'])
+    self.assertEqual(overlay_view(dict(readiness='DEGRADED', job_inflight=True))['activity'], 'DEGRADED')
+
+  def test_active_generation_preserves_playback_readiness(self):
+    view = overlay_view(dict(readiness='READY', job_inflight=True, generation_elapsed_seconds=12.3))
+    self.assertEqual(view['activity'], 'GENERATING')
+    self.assertTrue(view['ready'])
+    self.assertIn('12.3s', view['note'])
+
+  def test_missing_buffer_is_not_zero(self):
+    for value in (None, float('nan'), float('inf'), '12', True):
+      self.assertIsNone(overlay_view({'buffered': value})['buffered'])
+    self.assertEqual(overlay_view({'buffered': -1})['buffered'], 0)
+    self.assertEqual(overlay_view({})['activity'], 'PREPARING')
+
+  def test_stored_score_never_claims_chestnut_compute(self):
+    view = overlay_view(dict(composer='ace', compute='none', readiness='READY'))
+    self.assertEqual(view['backend'], ('STORED SCORE', 'NO COMPUTE'))
+
+  def test_profile_and_section_intent(self):
+    view = overlay_view(dict(profile='prism', composer='ace', section='VERSE / CONTINUOUS',
+                             next_section='PRECHORUS', form_labels_are_intent=True))
+    self.assertEqual(view['profile'], 'Prism')
+    self.assertEqual(view['section'], 'INTENT: VERSE > PRECHORUS')
+    self.assertEqual(view['backend'], ('ACE', 'CHESTNUT'))
+
+  def test_native_labels_and_truncation_use_supported_characters(self):
+    self.assertEqual(display_text('Prism · VERSE → CHORUS — café'), 'Prism / VERSE > CHORUS - cafe')
+    result = fit_text('A long section label', 10, len)
+    self.assertLessEqual(len(result), 10)
+    self.assertTrue(result.endswith('...'))
+    self.assertEqual(fit_text('long', 2, len), '')
+
+
+if __name__ == '__main__':
+  unittest.main()

roadscore/tools/preview_overlay.py

@@ -0,0 +1,79 @@
+"""Render synthetic overlay states with the repo's font atlas, entirely on CPU.
+
+Requires Pillow. No device, network, audio, replay, worker, or window is opened.
+"""
+import argparse
+import re
+import sys
+from pathlib import Path
+from types import SimpleNamespace
+
+from PIL import Image, ImageDraw
+
+ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(ROOT / 'roadscore/prototype'))
+from overlay_view import draw_panel
+
+
+class Canvas:
+  WHITE = (255, 255, 255, 255)
+  Color = staticmethod(lambda *rgba: rgba)
+  Rectangle = staticmethod(lambda x, y, w, h: (x, y, x + w, y + h))
+  Vector2 = staticmethod(lambda x, y: SimpleNamespace(x=x, y=y))
+
+  def __init__(self, image, font_path):
+    self.image = image
+    self.atlas = Image.open(font_path.with_suffix('.png')).convert('RGBA')
+    lines = font_path.read_text().splitlines()
+    self.base = abs(int(re.search(r'size=(-?\d+)', lines[0])[1]))
+    self.glyphs = {}
+    for line in lines:
+      if line.startswith('char '):
+        glyph = {key: int(value) for key, value in re.findall(r'(\w+)=(-?\d+)', line)}
+        self.glyphs[glyph['id']] = glyph
+
+  def measure_text_ex(self, font, text, size, spacing):
+    return SimpleNamespace(x=sum(self.glyphs[ord(c)]['xadvance'] for c in text) * size / self.base)
+
+  def draw_text_ex(self, font, text, pos, size, spacing, color):
+    scale = size / self.base
+    for char in text:
+      g = self.glyphs[ord(char)]
+      if g['width'] and g['height']:
+        mask = self.atlas.crop((g['x'], g['y'], g['x'] + g['width'], g['y'] + g['height'])).getchannel('A')
+        mask = mask.resize((max(1, round(g['width'] * scale)), max(1, round(g['height'] * scale))))
+        ink = Image.new('RGBA', mask.size, color)
+        self.image.paste(ink, (round(pos.x + g['xoffset'] * scale), round(pos.y + g['yoffset'] * scale)), mask)
+      pos.x += g['xadvance'] * scale
+
+  def draw_rectangle_rounded(self, rect, roundness, segments, color):
+    layer = Image.new('RGBA', self.image.size)
+    ImageDraw.Draw(layer).rounded_rectangle(rect, radius=6, fill=color)
+    self.image.alpha_composite(layer)
+
+  def draw_circle(self, x, y, radius, color):
+    ImageDraw.Draw(self.image).ellipse((x-radius, y-radius, x+radius, y+radius), fill=color)
+
+
+def main():
+  parser = argparse.ArgumentParser(description=__doc__)
+  parser.add_argument('--output', type=Path, required=True)
+  args = parser.parse_args()
+  base = dict(composer='ace', style='Prism', readiness='READY', section='VERSE / CONTINUOUS',
+              next_section='PRECHORUS', form_labels_are_intent=True, buffered=64)
+  cases = [{}, base, dict(base, job_inflight=True, generation_elapsed_seconds=23.4, turn_signal_music=True),
+           dict(base, readiness='DEGRADED', holding_accepted_music=True, job_inflight=True, buffered=18),
+           dict(base, worker_failed=True, buffered=0),
+           dict(readiness='READY', style='Stored score', section='ARCHIVED SCORE', compute='none', buffered=92)]
+  board = Image.new('RGBA', (728, 564), '#080e16')
+  for index, state in enumerate(cases):
+    panel = Image.new('RGBA', (364, 188), '#080e16')
+    draw_panel(Canvas(panel, ROOT / 'selfdrive/assets/fonts/Inter-Medium.fnt'), None, state, 364, 176)
+    board.paste(panel, ((index % 2) * 364, (index // 2) * 188))
+  args.output.parent.mkdir(parents=True, exist_ok=True)
+  board.convert('RGB').save(args.output)
+  print(args.output)
+
+
+if __name__ == '__main__':
+  main()