Merge branch 'master' into paramwatcher

I created a service handler in pyzmq and memory tested and it was in the MegaBytes, comparable to direct Params access, fudge that.

selfdrive/ui/sunnypilot/ui_state.py

@@ -23,6 +23,7 @@ class UIStateSP:
     self.sunnylink_state = SunnylinkState()
 
     self.custom_interactive_timeout: int = self.params.get("InteractivityTimeout", return_default=True)
+    self.global_brightness_override: int = self.params.get("Brightness", return_default=True)
 
   def update(self) -> None:
     if self.sunnylink_enabled:
@@ -77,6 +78,7 @@ class UIStateSP:
     self.chevron_metrics = self.params.get("ChevronInfo")
     self.active_bundle = self.params.get("ModelManager_ActiveBundle")
     self.custom_interactive_timeout = self.params.get("InteractivityTimeout", return_default=True)
+    self.global_brightness_override = self.params.get("Brightness", return_default=True)
 
 
 class DeviceSP:

selfdrive/ui/ui_state.py

@@ -258,9 +258,18 @@ class Device(DeviceSP):
       else:
         clipped_brightness = ((clipped_brightness + 16.0) / 116.0) ** 3.0
 
-      clipped_brightness = float(np.interp(clipped_brightness, [0, 1], [30, 100]))
+      if gui_app.sunnypilot_ui():
+        if ui_state.global_brightness_override <= 0:
+          min_global_brightness = 1 if ui_state.global_brightness_override < 0 else 30
+          clipped_brightness = float(np.interp(clipped_brightness, [0, 1], [min_global_brightness, 100]))
+      else:
+        clipped_brightness = float(np.interp(clipped_brightness, [0, 1], [30, 100]))
 
     brightness = round(self._brightness_filter.update(clipped_brightness))
+
+    if gui_app.sunnypilot_ui() and ui_state.global_brightness_override > 0:
+      brightness = ui_state.global_brightness_override
+
     if not self._awake:
       brightness = 0
 

sunnypilot/common/README.md

@@ -0,0 +1,129 @@
+<center>
+
+# Comparative Analysis of Parameter Getter Methods: Params vs. ParamWatcher
+
+James (sunnypilot Developer) <br>
+December 13, 2025
+
+</center>
+
+
+## <br><br> Abstract
+
+This research report examines the inefficiencies in standard parameter access methods within sunnypilot and proposes an optimized alternative, ParamWatcher. The standard `Params::get()` method incurs significant CPU and memory overhead due to repeated file I/O operations (sunnypilot, 2025). ParamWatcher utilizes OS-level file system events (inotify on Linux, FSEvents on macOS) to maintain an in-memory cache, reducing I/O to near zero (Linux man-pages, 2025-c; Apple Inc., n.d.-a). Empirical benchmarks with ~10 million parameter accesses demonstrate a 14.5x CPU speedup and flat memory usage (514.7 KB vs. 497.7 KB for base Params, with only 17 KB overhead). The implementation employs a process-local singleton pattern for efficiency in multi-process architectures (Gamma et al., 1994). Results indicate ParamWatcher eliminates UI stutters and GC pauses, enhancing system responsiveness without compromising data freshness.
+
+**Keywords:** parameter access, file I/O optimization, event-driven caching, autonomous driving systems, performance benchmarking
+
+## Introduction
+
+In sunnypilot, efficient parameter management is important for real-time system access. The standard `Params::get()` method, implemented in C++ and wrapped in Python, performs full file I/O cycles for each access, leading to high CPU overhead and memory churn (sunnypilot, 2025). This is particularly problematic in UI loops where parameters are queried frequently (e.g., 50 toggles at 20 FPS equates to ~1,000 reads/second).
+
+This inefficiency stems from architectural mismatches: C++ streams are designed for throughput, not latency (cppreference.com, n.d.-a). Each call triggers kernel mode switches, heap allocations, and garbage collection in Python, causing UI stutters (Linux man-pages, 2025-a; Linux man-pages, 2025-b).
+
+### Inefficiencies in Standard Parameter Access
+The standard `Params::get()` method executes a full file I/O lifecycle—opening, allocating, reading, and closing—for every function call. This results in significant CPU overhead and memory churn due to the frequency of these operations in the user interface loop.
+
+#### System Overhead Analysis
+- **System Call Overhead**: Every read operation requires context switches into kernel mode. The `Params::get` function calls `util::read_file` (sunnypilot, 2025), which subsequently invokes `std::ifstream` (sunnypilot, 2025).
+- **Impact**: Frequent context switching degrades performance (Linux man-pages, 2025-a; Linux man-pages, 2025-b).
+- **C++ Stream Overhead**: The use of `std::ifstream` introduces additional overhead for maintaining stream state and buffering compared to raw file descriptors (cppreference.com, n.d.-a; Codezup, 2025).
+- **Memory Churn**: The instantiation of `std::string result(size, '\0');` forces heap allocation and deallocation during every call (sunnypilot, 2025). This stresses the memory allocator and can lead to fragmentation (cppreference.com, n.d.-b).
+
+This report introduces ParamWatcher, an event-driven caching solution using OS file system events. It shifts from polling to notifications, caching converted values in static RAM. I propose that ParamWatcher achieves minimum 10x+ CPU gains with bounded non increasing memory, improving sunnypilot's performance both on latency and responsiveness.
+
+## Method
+
+### Materials
+- **System:** sunnypilot, running on macOS, Ubuntu/Linux, comma 3x, and comma four.
+- **Parameters:** 231 defined keys in `param_keys.h`.
+- **Tools:** Python 3, tracemalloc for memory profiling, time.perf_counter for CPU timing, ctypes for OS integration (Python Software Foundation, 2025).
+
+### Implementation Details
+ParamWatcher provides cross-platform file system monitoring using ctypes for direct OS integration (Python Software Foundation, 2025).
+
+#### Linux Implementation
+On Linux, ParamWatcher uses the inotify subsystem for efficient file change detection (Linux man-pages, 2025-c). It loads `libc.so.6` to access system calls, initializes an inotify instance, and watches the parameters directory for events like `IN_MODIFY` and `IN_CLOSE_WRITE` (Linux Kernel Organization, 2005). Events are polled with `select.epoll()` and parsed using `struct.unpack_from()` to avoid ctypes overhead. Filenames are extracted and passed to cache invalidation, ensuring real-time updates without polling (Codezup, 2025).
+
+- **Library Loading**: `libc = ctypes.CDLL('libc.so.6')` loads the standard C library to access system calls.
+- **Initialization**: `inotify_init()` is called to create a new inotify instance, returning a file descriptor.
+- **Watch Setup**: `inotify_add_watch(fd, path, mask)` registers the parameters directory. The mask includes `IN_MODIFY | IN_CREATE | IN_DELETE | IN_MOVED_TO | IN_CLOSE_WRITE` (Linux Kernel Organization, 2005) to capture all relevant file changes.
+- **Event Loop**:
+  - **Polling**: `select.epoll()` is used to efficiently wait for activity on the file descriptor without busy-waiting.
+  - **Reading**: When events occur, `os.read(fd, 2048)` retrieves the raw binary event data.
+  - **Parsing**: The code uses Python's `struct` module (`struct.unpack_from("iIII", ...)`) to parse the C-style `inotify_event` structures directly from the buffer, avoiding the overhead of defining `ctypes` structures.
+  - **Handling**: Extracted filenames are passed to `_trigger_callbacks`, which invalidates the specific cache entry (`self._cache.pop(path, None)`), forcing a fresh read on the next access.
+
+#### macOS Implementation
+On macOS, ParamWatcher leverages FSEvents from CoreServices for directory monitoring (Apple Inc., n.d.-a). It defines a C-compatible callback using `CFUNCTYPE`, creates an `FSEventStream` with `kFSEventStreamCreateFlagFileEvents`, and schedules it on the run loop (Apple Inc., n.d.-b). Events are filtered for modifications, creations, and renames (Apple Inc., n.d.-c), triggering cache invalidation for affected parameters.
+
+- **Framework Loading**: `ctypes.cdll.LoadLibrary` loads `CoreServices` and `CoreFoundation`.
+- **Callback Definition**: `CFUNCTYPE` is used to define a C-compatible callback function. This function is invoked by the OS whenever a change occurs in the watched directory.
+- **Stream Creation**: `FSEventStreamCreate` creates a stream for the target directory. The `kFSEventStreamCreateFlagFileEvents` flag is used to request file-level granularity where available.
+- **Event Filtering**: The callback filters events using flags such as `kFSEventStreamEventFlagItemCreated` and `kFSEventStreamEventFlagItemModified` to ensure only relevant file changes trigger updates (Apple Inc., n.d.-c).
+- **Scheduling**: `FSEventStreamScheduleWithRunLoop` attaches the stream to the current thread's run loop (Apple Inc., n.d.-b).
+- **Execution**: `CFRunLoopRun()` starts the event loop. This passes control to the OS, which wakes the thread only when necessary.
+- **Handling**: Inside the callback, the code iterates through the changed paths provided by the OS. It extracts the filename and calls `_trigger_callbacks` to invalidate the cache for that specific parameter.
+
+### Procedure
+Benchmarks simulated heavy load:
+- **Memory Test:** ~10 million gets (43,290 loops over 231 keys), measured with tracemalloc.
+- **CPU Test:** Same load, timed with perf_counter.
+- Comparisons: Base Params vs. ParamWatcher.
+
+## Results
+
+### Memory Usage
+Using tracemalloc for peak memory measurement during ~10 million parameter accesses (43,290 loops over 231 keys), base Params peaked at 497.7 KB, while ParamWatcher peaked at 514.7 KB (17 KB overhead). ParamWatcher's memory remained flat post-initialization, preventing churn.
+
+| Condition | Memory (KB) | Overhead |
+|-----------|-------------|----------|
+| Base Params | 497.7 | - |
+| ParamWatcher | 514.7 | 17 KB |
+### CPU Performance
+ParamWatcher was 14.5x faster: 4.52s vs. 65.43s to complete ~10 million param gets.
+
+| Condition | Time (s) | Speedup |
+|-----------|----------|---------|
+| Base Params | 65.43 | 1x |
+| ParamWatcher | 4.52 | 14.5x |
+
+### Scalability
+No degradation at scale; cache invalidation maintained freshness.
+
+## Discussion
+
+ParamWatcher successfully optimizes parameter access, delivering substantial CPU gains with minimal memory overhead. The event-driven approach eliminates I/O bottlenecks, reducing GC pressure and UI stutters (cppreference.com, n.d.-b). The 17 KB memory overhead is negligible compared to the megabytes of churn from base Params, ensuring bounded usage in multi-process environments via the singleton pattern (Gamma et al., 1994).
+
+Results demonstrate scalability without degradation, with cache invalidation maintaining data freshness. This optimization enhances system responsiveness.
+Limitations include potential event latency in high-load scenarios (<10 ms, imperceptible for UI) and increased complexity from background threads.
+Trade-offs: Static RAM (~17 KB) vs. dynamic churn; benefits outweigh costs for param-heavy workloads.
+
+## <br>References
+
+Apple Inc. (n.d.-a). *File System Events*. Retrieved from https://developer.apple.com/documentation/coreservices/file_system_events
+
+Apple Inc. (n.d.-b). *CFRunLoop*. Retrieved from https://developer.apple.com/documentation/corefoundation/cfrunloop
+
+Apple Inc. (n.d.-c). *FSEventStreamEventFlags*. Retrieved from https://developer.apple.com/documentation/coreservices/1455361-fseventstreameventflags
+
+Codezup. (2025). *Efficient File I/O in C++*. Retrieved from https://codezup.com/efficient-file-io-cpp-best-practices/
+
+cppreference.com. (n.d.-a). *std::basic_ifstream*. Retrieved from https://en.cppreference.com/w/cpp/io/basic_ifstream
+
+cppreference.com. (n.d.-b). *std::basic_string*. Retrieved from https://en.cppreference.com/w/cpp/string/basic_string/basic_string
+
+Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1994). *Design Patterns: Elements of Reusable Object-Oriented Software*. Addison-Wesley.
+
+Linux Kernel Organization. (2005). *include/uapi/linux/inotify.h*. Retrieved from https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/inotify.h
+
+Linux man-pages. (2025-a). *open(2)*. Retrieved from https://man7.org/linux/man-pages/man2/open.2.html
+
+Linux man-pages. (2025-b). *read(2)*. Retrieved from https://man7.org/linux/man-pages/man2/read.2.html
+
+Linux man-pages. (2025-c). *inotify(7)*. Retrieved from https://man7.org/linux/man-pages/man7/inotify.7.html
+
+Python Software Foundation. (2025). *ctypes — A foreign function library for Python*. Retrieved from https://docs.python.org/3/library/ctypes.html
+
+sunnypilot. (2025). *common/params.cc* [Source code]. GitHub. https://github.com/sunnypilot/sunnypilot/blob/master/common/params.cc#L180C1-L206C2
+
+sunnypilot. (2025). *common/util.cc* [Source code]. GitHub. https://github.com/sunnypilot/sunnypilot/blob/master/common/util.cc#L79C1-L117C2

sunnypilot/common/param_watcher.py

@@ -0,0 +1,193 @@
+"""
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+This file is part of sunnypilot and is licensed under the MIT License.
+See the LICENSE.md file in the root directory for more details.
+"""
+import os
+import platform
+import struct
+import select
+import threading
+import time
+import ctypes
+import ctypes.util
+import traceback
+from ctypes import c_void_p, c_size_t, POINTER, c_uint32, c_uint64
+
+from openpilot.common.params import Params
+from openpilot.common.swaglog import cloudlog
+from openpilot.system.hardware.hw import Paths
+
+IN_MODIFY = 0x00000002
+IN_CREATE = 0x00000100
+IN_DELETE = 0x00000200
+IN_MOVED_TO = 0x00000080
+IN_CLOSE_WRITE = 0x00000008
+
+
+class ParamWatcher(Params):
+  _instance = None
+
+  def __new__(cls):
+    if cls._instance is None:
+      cls._instance = super().__new__(cls)
+      cls._instance._initialized = False
+    return cls._instance
+
+  def __init__(self):
+    if self._initialized:
+      return
+    super().__init__()
+    cloudlog.warning("ParamWatcher initialized")
+    self._cache = {}
+    self._last_trigger = {}
+    self._version = {}
+    self._lock = threading.Lock()
+    self._callbacks = []
+    self.last_accessed_param = None
+    self._initialized = True
+    self.start()
+
+  def start(self):
+    if getattr(self, '_thread', None) and self._thread.is_alive():
+      return
+    self._thread = threading.Thread(target=self._run_watcher, daemon=True)
+    self._thread.start()
+
+  def is_watching(self):
+    return getattr(self, '_thread', None) and self._thread.is_alive()
+
+  def add_watcher(self, callback):
+    if callback not in self._callbacks:
+      self._callbacks.append(callback)
+
+  def _trigger_callbacks(self, path):
+    with self._lock:
+      if (now := time.monotonic()) - self._last_trigger.get(path, 0) < 0.1:
+        return
+      self._last_trigger[path] = now
+      self._version[path] = self._version.get(path, 0) + 1
+      self._cache.pop(path, None)
+
+    for callback in self._callbacks:
+      try:
+        callback(path)
+      except Exception:
+        cloudlog.exception("Param watcher callback failed")
+
+  def _get_cached(self, key, getter, sig):
+    k = str(key)
+    with self._lock:
+      bucket = self._cache.get(k)
+      if bucket and sig in bucket:
+        if bucket[sig][0] == self._version.get(k, 0):
+          return bucket[sig][1]
+
+    start_ver = self._version.get(k, 0)
+    val = getter()
+    with self._lock:
+      if self._version.get(k, 0) != start_ver:
+        val = getter()
+      self._cache.setdefault(k, {})[sig] = (self._version.get(k, 0), val)
+    return val
+
+  def get(self, key, block=False, return_default=False):
+    self.last_accessed_param = key
+    if block:
+      return super().get(key, block, return_default)
+    fetcher = super().get
+    return self._get_cached(key, lambda: fetcher(key, block, return_default), (block, return_default))
+
+  def get_bool(self, key, block=False):
+    self.last_accessed_param = key
+    if block:
+      return super().get_bool(key, block)
+    fetcher = super().get_bool
+    return self._get_cached(key, lambda: fetcher(key, block), ("bool", block))
+
+  def _run_watcher(self):
+    system = platform.system()
+    while True:
+      try:
+        if system == "Linux":
+          self._run_linux()
+        elif system == "Darwin":
+          self._run_darwin()
+      except Exception:
+        cloudlog.exception("Param watcher crashed")
+        time.sleep(2)
+
+  def _run_linux(self):
+    path = Paths.params_root()
+    libc = ctypes.CDLL('libc.so.6')
+    fd = libc.inotify_init()
+    libc.inotify_add_watch(fd, path.encode(), IN_MODIFY | IN_CREATE | IN_DELETE | IN_MOVED_TO | IN_CLOSE_WRITE)
+
+    try:
+      poll = select.epoll()
+      poll.register(fd, select.EPOLLIN)
+      while True:
+        for fileno, _ in poll.poll():
+          if fileno == fd:
+            buffer = os.read(fd, 2048)
+            i = 0
+            while i + 16 <= len(buffer):
+              _, mask, _, name_len = struct.unpack_from("iIII", buffer, i)
+              if mask & (IN_MODIFY | IN_CREATE | IN_DELETE | IN_MOVED_TO | IN_CLOSE_WRITE):
+                name = buffer[i+16:i+16+name_len].rstrip(b"\0").decode()
+                if not name.startswith("."):
+                  self._trigger_callbacks(name)
+              i += 16 + name_len
+    finally:
+      if 'poll' in locals():
+        poll.unregister(fd)
+        poll.close()
+      os.close(fd)
+
+  def _run_darwin(self):
+    CS = ctypes.cdll.LoadLibrary(ctypes.util.find_library("CoreServices"))
+    CF = ctypes.cdll.LoadLibrary(ctypes.util.find_library("CoreFoundation"))
+
+    kCFAllocatorDefault = c_void_p(0)
+    kCFStringEncodingUTF8 = 0x08000100
+    kFSEventStreamCreateFlagFileEvents = 0x00000010
+    kFSEventStreamEventFlagItemCreated = 0x00000100
+    kFSEventStreamEventFlagItemRemoved = 0x00000200
+    kFSEventStreamEventFlagItemRenamed = 0x00000800
+    kFSEventStreamEventFlagItemModified = 0x00001000
+
+    CF.CFStringCreateWithCString.restype = c_void_p
+    CF.CFStringCreateWithCString.argtypes = [c_void_p, ctypes.c_char_p, c_uint32]
+    CF.CFArrayCreate.restype = c_void_p
+    CF.CFArrayCreate.argtypes = [c_void_p, POINTER(c_void_p), c_size_t, c_void_p]
+    CS.FSEventStreamCreate.restype = c_void_p
+    CS.FSEventStreamCreate.argtypes = [c_void_p, c_void_p, c_void_p, c_void_p, c_uint64, ctypes.c_double, c_uint32]
+    CS.FSEventStreamScheduleWithRunLoop.argtypes = [c_void_p, c_void_p, c_void_p]
+    CS.FSEventStreamStart.argtypes = [c_void_p]
+    CF.CFRunLoopGetCurrent.restype = c_void_p
+
+    def _cb(stream, ctx, num, paths, flags, ids):
+      try:
+        paths_arr = ctypes.cast(paths, POINTER(c_void_p))
+        flags_arr = ctypes.cast(flags, POINTER(c_uint32))
+        for i in range(num):
+          path = ctypes.cast(paths_arr[i], ctypes.c_char_p).value
+          if path and (flags_arr[i] & (kFSEventStreamEventFlagItemCreated | kFSEventStreamEventFlagItemRemoved |
+                                       kFSEventStreamEventFlagItemRenamed | kFSEventStreamEventFlagItemModified)):
+            self._trigger_callbacks(os.path.basename(path.decode('utf-8').rstrip('/')))
+      except Exception:
+        traceback.print_exc()
+
+    self._darwin_cb = ctypes.CFUNCTYPE(None, c_void_p, c_void_p, c_size_t, c_void_p, POINTER(c_uint32), POINTER(c_uint64))(_cb)
+
+    path_str = Paths.params_root().encode('utf-8')
+    cf_path = CF.CFStringCreateWithCString(kCFAllocatorDefault, path_str, kCFStringEncodingUTF8)
+    cf_paths = CF.CFArrayCreate(kCFAllocatorDefault, (c_void_p * 1)(cf_path), 1, None)
+    stream = CS.FSEventStreamCreate(kCFAllocatorDefault, self._darwin_cb, None, cf_paths, -1, 0.05, kFSEventStreamCreateFlagFileEvents)
+
+    run_loop = CF.CFRunLoopGetCurrent()
+    kDefaultMode = CF.CFStringCreateWithCString(kCFAllocatorDefault, b"kCFRunLoopDefaultMode", kCFStringEncodingUTF8)
+    CS.FSEventStreamScheduleWithRunLoop(stream, run_loop, kDefaultMode)
+    CS.FSEventStreamStart(stream)
+    CF.CFRunLoopRun()

sunnypilot/common/params.py

@@ -0,0 +1,4 @@
+from openpilot.common.params_pyx import ParamKeyFlag, ParamKeyType, UnknownKeyName
+from openpilot.sunnypilot.common.param_watcher import ParamWatcher as Params
+
+__all__ = ["Params", "ParamKeyFlag", "ParamKeyType", "UnknownKeyName"]

sunnypilot/common/tests/test_param_watcher.py

@@ -0,0 +1,94 @@
+import time
+import pytest
+import threading
+import tracemalloc
+
+from openpilot.common.params import Params
+from openpilot.common.params_pyx import UnknownKeyName
+
+from openpilot.sunnypilot.common.param_watcher import ParamWatcher
+
+
+class TestParamWatcher:
+  BYTES_KEYS = ["LocationFilterInitialState", "UpdaterCurrentReleaseNotes", "UpdaterNewReleaseNotes"]
+  BOOL_KEYS = [
+    "IsMetric", "AdbEnabled", "AlwaysOnDM", "ExperimentalMode",
+    "ExperimentalModeConfirmed", "DisengageOnAccelerator",
+    "OpenpilotEnabledToggle", "RecordAudio", "RecordFront"
+  ]
+  _key_counter = 0
+
+  @pytest.fixture(autouse=True)
+  def setup_method(self):
+    self.params = Params()
+    self.key_index = TestParamWatcher._key_counter
+    TestParamWatcher._key_counter += 1
+    self.bytes_key = self.BYTES_KEYS[self.key_index % len(self.BYTES_KEYS)]
+    self.bool_key = self.BOOL_KEYS[self.key_index % len(self.BOOL_KEYS)]
+
+  @pytest.fixture
+  def param_watcher(self):
+    ParamWatcher._instance = None
+    param_watch = ParamWatcher()
+    param_watch.start()
+    assert param_watch.is_watching(), "ParamWatcher thread died"
+    return param_watch
+
+  def teardown_method(self):
+    for key in (self.bytes_key, self.bool_key):
+      try:
+        self.params.remove(key)
+      except UnknownKeyName:
+        pass
+
+  def test_watcher_detects_change(self, param_watcher):
+    val = b"123"
+    self.params.put(self.bytes_key, val)
+    assert param_watcher.get(self.bytes_key) == val
+
+  def test_watcher_get_bool(self, param_watcher):
+    self.params.put_bool(self.bool_key, True)
+    assert param_watcher.get_bool(self.bool_key) is True  # First read should populate internal cache
+
+  def test_performance_comparison(self, param_watcher):
+    plain_params = self.params
+
+    for key in self.BYTES_KEYS:
+      plain_params.put(key, b"x" * 10000)
+      param_watcher.get(key)
+    for key in self.BOOL_KEYS:
+      plain_params.put_bool(key, True)
+      param_watcher.get_bool(key)
+
+    def bench(get_bytes, get_bool):
+      tracemalloc.start()
+      start_time = time.process_time()
+      for _ in range(1000):
+        for key in self.BYTES_KEYS:
+          get_bytes(key)
+        for key in self.BOOL_KEYS:
+          get_bool(key)
+      duration = time.process_time() - start_time
+      _, memory = tracemalloc.get_traced_memory()
+      tracemalloc.stop()
+      return duration, memory
+
+    plain_cpu, plain_memory = bench(plain_params.get, plain_params.get_bool)
+    watcher_cpu, watcher_memory = bench(param_watcher.get, param_watcher.get_bool)
+
+    # ParamWatcher *should* be significantly faster and use less memory than Params()
+    assert watcher_cpu < plain_cpu * 0.6, f"PW CPU ({watcher_cpu:.4f}s) should be < 60% of Param call ({plain_cpu:.4f}s)"
+    assert watcher_memory < plain_memory * 0.5, f"PW Memory ({watcher_memory}B) should be < 50% of Param call ({plain_memory}B)"
+
+  def test_cache_invalidation_simulation(self, param_watcher):
+    self.params.put(self.bytes_key, b"old")
+    assert param_watcher.get(self.bytes_key) == b"old"
+    time.sleep(0.2)
+
+    event = threading.Event()
+    param_watcher.add_watcher(lambda key: event.set())
+    param_watcher._trigger_callbacks(self.bytes_key)
+    assert event.wait(timeout=2), "Callback not triggered"
+
+    self.params.put(self.bytes_key, b"new")
+    assert param_watcher.get(self.bytes_key) == b"new"

sunnypilot/sunnylink/params_metadata.json

@@ -5,17 +5,15 @@
   },
   "AdbEnabled": {
     "title": "Enable ADB",
-    "description": "Allow ADB connections to the device.",
-    "long_description": "ADB (Android Debug Bridge) allows connecting to your device over USB or over the network. See https://docs.comma.ai/how-to/connect-to-comma for more info."
+    "description": ""
   },
   "AlphaLongitudinalEnabled": {
-    "title": "sunnypilot Longitudinal Control (Alpha)",
-    "description": "Enable sunnypilot longitudinal control alpha for this car (disables AEB).",
-    "long_description": "<b>WARNING: sunnypilot longitudinal control is in alpha for this car and will disable Automatic Emergency Braking (AEB).</b><br><br>On this car, sunnypilot defaults to the car's built-in ACC instead of sunnypilot's longitudinal control. Enable this to switch to sunnypilot longitudinal control. Enabling Experimental mode is recommended when enabling sunnypilot longitudinal control alpha. Changing this setting will restart sunnypilot if the car is powered on."
+    "title": "Alpha Longitudinal",
+    "description": ""
   },
   "AlwaysOnDM": {
-    "title": "Always-On Driver Monitoring",
-    "description": "Enable driver monitoring even when sunnypilot is not engaged."
+    "title": "Always-on Driver Monitor",
+    "description": ""
   },
   "ApiCache_Device": {
     "title": "Api Cache  Device",
@@ -84,11 +82,11 @@
     ]
   },
   "BackupManager_CreateBackup": {
-    "title": "Backup Settings",
+    "title": "Create Backup",
     "description": ""
   },
   "BackupManager_RestoreVersion": {
-    "title": "Restore Settings",
+    "title": "Restore Version",
     "description": ""
   },
   "BlindSpot": {
@@ -128,13 +126,11 @@
     "description": "Virtually shift camera's perspective to move model's center to Left(+ values) or Right (- values)",
     "min": -0.35,
     "max": 0.35,
-    "step": 0.01,
-    "unit": "meters"
+    "step": 0.01
   },
   "CarBatteryCapacity": {
     "title": "Car Battery Capacity",
-    "description": "Battery Size",
-    "unit": "kWh"
+    "description": ""
   },
   "CarList": {
     "title": "Supported Car List",
@@ -223,17 +219,16 @@
     "description": ""
   },
   "DeviceBootMode": {
-    "title": "Wake Up Behavior",
-    "description": "Choose device behavior after boot/sleep.",
-    "long_description": "Controls state of the device after boot/sleep.\n\nDefault: Device will boot/wake-up normally & will be ready to engage.\nOffroad: Device will be in Always Offroad mode after boot/wake-up.",
+    "title": "Device Boot Mode",
+    "description": "",
     "options": [
       {
         "value": 0,
-        "label": "Default"
+        "label": "Standard"
       },
       {
         "value": 1,
-        "label": "Offroad"
+        "label": "Always Offroad"
       }
     ]
   },
@@ -250,8 +245,8 @@
     "description": ""
   },
   "DisengageOnAccelerator": {
-    "title": "Disengage on Accelerator Pedal",
-    "description": "When enabled, pressing the accelerator pedal will disengage sunnypilot."
+    "title": "Disengage On Accelerator",
+    "description": ""
   },
   "DoReboot": {
     "title": "Reboot",
@@ -266,7 +261,7 @@
     "description": ""
   },
   "DongleId": {
-    "title": "Dongle ID",
+    "title": "Device ID",
     "description": ""
   },
   "DriverTooDistracted": {
@@ -286,9 +281,8 @@
     "description": ""
   },
   "EnableSunnylinkUploader": {
-    "title": "Enable sunnylink uploader (infrastructure test)",
-    "description": "Upload driving data to sunnypilot servers (tier-gated).",
-    "long_description": "Enable sunnylink uploader to allow sunnypilot to upload your driving data to sunnypilot servers. (Only for highest tiers, and does NOT bring ANY benefit to you yet. We are just testing data volume.)"
+    "title": "Enable sunnylink Uploader",
+    "description": ""
   },
   "EnforceTorqueControl": {
     "title": "Enforce Torque Control",
@@ -296,8 +290,7 @@
   },
   "ExperimentalMode": {
     "title": "Experimental Mode",
-    "description": "Enable alpha experimental features and new driving visualization.",
-    "long_description": "sunnypilot defaults to driving in chill mode. Experimental mode enables alpha-level features that aren't ready for chill mode. Experimental features are listed below:<br><h4>End-to-End Longitudinal Control</h4><br>Let the driving model control the gas and brakes. sunnypilot will drive as it thinks a human would, including stopping for red lights and stop signs. Since the driving model decides the speed to drive, the set speed will only act as an upper bound. This is an alpha quality feature; mistakes should be expected.<br><h4>New Driving Visualization</h4><br>The driving visualization will transition to the road-facing wide-angle camera at low speeds to better show some turns. The Experimental mode logo will also be shown in the top right corner."
+    "description": ""
   },
   "ExperimentalModeConfirmed": {
     "title": "Experimental Mode Confirmed",
@@ -360,7 +353,7 @@
     "description": ""
   },
   "HardwareSerial": {
-    "title": "Serial",
+    "title": "Serial Number",
     "description": ""
   },
   "HasAcceptedTerms": {
@@ -403,8 +396,7 @@
   },
   "InteractivityTimeout": {
     "title": "Interactivity Timeout",
-    "description": "",
-    "unit": "seconds"
+    "description": ""
   },
   "IsDevelopmentBranch": {
     "title": "Is Development Branch",
@@ -419,13 +411,12 @@
     "description": ""
   },
   "IsLdwEnabled": {
-    "title": "Enable Lane Departure Warnings",
-    "description": "Alert when drifting over lane lines without signaling.",
-    "long_description": "Receive alerts to steer back into the lane when your vehicle drifts over a detected lane line without a turn signal activated while driving over 31 mph (50 km/h)."
+    "title": "Lane Departure Warnings",
+    "description": ""
   },
   "IsMetric": {
-    "title": "Use Metric System",
-    "description": "Display speed in km/h instead of mph."
+    "title": "Use Metric Units",
+    "description": ""
   },
   "IsOffroad": {
     "title": "Is Offroad",
@@ -461,30 +452,27 @@
   },
   "LagdToggle": {
     "title": "Live Learning Steer Delay",
-    "description": "Let the car learn and adapt its steering response time.",
-    "long_description": "Enable this for the car to learn and adapt its steering response time. Disable to use a fixed steering response time. Keeping this on provides the stock openpilot experience."
+    "description": "Allow device to learn and adapt car's steering response time"
   },
   "LagdToggleDelay": {
-    "title": "Adjust Software Delay",
-    "description": "Adjust the software delay when Live Learning Steer Delay is toggled off. The default software delay value is 0.2",
+    "title": "Manual Software Delay",
+    "description": "Software delay to use when Live Learning Steer Delay is toggled off",
     "min": 0.05,
     "max": 0.5,
-    "step": 0.01,
-    "unit": "seconds"
+    "step": 0.01
   },
   "LagdValueCache": {
     "title": "LaGD Value Cache",
     "description": ""
   },
   "LaneTurnDesire": {
-    "title": "Use Lane Turn Desires",
-    "description": "Plan a turn at low speeds when signaling.",
-    "long_description": "If you're driving at 20 mph (32 km/h) or below and have your blinker on, the car will plan a turn in that direction at the nearest drivable path. This prevents situations (like at red lights) where the car might plan the wrong turn direction."
+    "title": "Lane Turn Desire",
+    "description": "Force model to plan an intent to turn based on blinker"
   },
   "LaneTurnValue": {
-    "title": "Adjust Lane Turn Speed",
-    "description": "Set the maximum speed for lane turn desires. Default is 19 mph.",
-    "min": 5,
+    "title": "Lane Turn Speed",
+    "description": "Maximum speed for lane turn desire",
+    "min": 0,
     "max": 20,
     "step": 1
   },
@@ -578,8 +566,7 @@
   },
   "LongitudinalPersonality": {
     "title": "Driving Personality",
-    "description": "Choose relaxed, standard, or aggressive following behavior.",
-    "long_description": "Standard is recommended. In aggressive mode, sunnypilot will follow lead cars closer and be more aggressive with the gas and brake. In relaxed mode sunnypilot will stay further away from lead cars. On supported cars, you can cycle through these personalities with your steering wheel distance button.",
+    "description": "",
     "options": [
       {
         "value": 0,
@@ -643,68 +630,18 @@
   },
   "MaxTimeOffroad": {
     "title": "Max Time Offroad",
-    "description": "Device will automatically shutdown after set time once the engine is turned off.\n(30h is the default)",
-    "options": [
-      {
-        "value": 0,
-        "label": "Always On"
-      },
-      {
-        "value": 5,
-        "label": "5m"
-      },
-      {
-        "value": 10,
-        "label": "10m"
-      },
-      {
-        "value": 15,
-        "label": "15m"
-      },
-      {
-        "value": 30,
-        "label": "30m"
-      },
-      {
-        "value": 60,
-        "label": "1h"
-      },
-      {
-        "value": 120,
-        "label": "2h"
-      },
-      {
-        "value": 180,
-        "label": "3h"
-      },
-      {
-        "value": 300,
-        "label": "5h"
-      },
-      {
-        "value": 600,
-        "label": "10h"
-      },
-      {
-        "value": 1440,
-        "label": "24h"
-      },
-      {
-        "value": 1800,
-        "label": "30h (Default)"
-      }
-    ]
+    "description": ""
   },
   "ModelManager_ActiveBundle": {
-    "title": "Current Model",
+    "title": "Model Manager Active Bundle",
     "description": ""
   },
   "ModelManager_ClearCache": {
-    "title": "Clear Model Cache",
+    "title": "Model Manager Clear Cache",
     "description": ""
   },
   "ModelManager_DownloadIndex": {
-    "title": "Cancel Download",
+    "title": "Model Manager Download Index",
     "description": ""
   },
   "ModelManager_Favs": {
@@ -712,7 +649,7 @@
     "description": ""
   },
   "ModelManager_LastSyncTime": {
-    "title": "Refresh Model List",
+    "title": "Model Manager Last Sync Time",
     "description": ""
   },
   "ModelManager_ModelsCache": {
@@ -770,7 +707,7 @@
     "description": ""
   },
   "OffroadMode": {
-    "title": "Always Offroad",
+    "title": "Force Offroad Mode",
     "description": ""
   },
   "Offroad_CarUnrecognized": {
@@ -847,56 +784,9 @@
   "OnroadScreenOffTimer": {
     "title": "Onroad Brightness Delay",
     "description": "",
-    "options": [
-      {
-        "value": 15,
-        "label": "15s"
-      },
-      {
-        "value": 30,
-        "label": "30s"
-      },
-      {
-        "value": 60,
-        "label": "1m"
-      },
-      {
-        "value": 120,
-        "label": "2m"
-      },
-      {
-        "value": 180,
-        "label": "3m"
-      },
-      {
-        "value": 240,
-        "label": "4m"
-      },
-      {
-        "value": 300,
-        "label": "5m"
-      },
-      {
-        "value": 360,
-        "label": "6m"
-      },
-      {
-        "value": 420,
-        "label": "7m"
-      },
-      {
-        "value": 480,
-        "label": "8m"
-      },
-      {
-        "value": 540,
-        "label": "9m"
-      },
-      {
-        "value": 600,
-        "label": "10m"
-      }
-    ]
+    "min": 0,
+    "max": 60,
+    "step": 1
   },
   "OnroadUploads": {
     "title": "Onroad Uploads",
@@ -904,8 +794,7 @@
   },
   "OpenpilotEnabledToggle": {
     "title": "Enable sunnypilot",
-    "description": "Enable sunnypilot adaptive cruise and lane keep.",
-    "long_description": "Use the sunnypilot system for adaptive cruise control and lane keep driver assistance. Your attention is required at all times to use this feature."
+    "description": ""
   },
   "OsmDbUpdatesCheck": {
     "title": "OSM DB Updates Check",
@@ -957,8 +846,7 @@
   },
   "PlanplusControl": {
     "title": "Plan Plus Controls",
-    "description": "Adjust planplus model recentering strength.",
-    "long_description": "Adjust planplus model recentering strength. The higher this number the more aggressively the model will recover to lanecenter, too high and it will ping-pong.",
+    "description": "Adjust planplus model recentering strength. The higher this number the more aggressively the model will recover to lanecenter, too high and it will ping-pong",
     "min": 0.0,
     "max": 2.0,
     "step": 0.1
@@ -980,16 +868,16 @@
     "description": ""
   },
   "RecordAudio": {
-    "title": "Record and Upload Microphone Audio",
-    "description": "Record and store microphone audio while driving. The audio will be included in the dashcam video in comma connect."
+    "title": "Record & Upload Mic Audio",
+    "description": ""
   },
   "RecordAudioFeedback": {
     "title": "Record Audio Feedback",
     "description": ""
   },
   "RecordFront": {
-    "title": "Record and Upload Driver Camera",
-    "description": "Upload data from the driver facing camera and help improve the driver monitoring algorithm."
+    "title": "Record & Upload Driver Camera",
+    "description": ""
   },
   "RecordFrontLock": {
     "title": "Record Front Lock",
@@ -1117,13 +1005,12 @@
     "description": ""
   },
   "SubaruStopAndGo": {
-    "title": "Stop and Go (Beta)",
-    "description": "Experimental feature to enable auto-resume during stop-and-go for certain supported Subaru platforms."
+    "title": "Subaru Stop and Go",
+    "description": ""
   },
   "SubaruStopAndGoManualParkingBrake": {
-    "title": "Stop and Go for Manual Parking Brake (Beta)",
-    "description": "Enable stop-and-go for Subaru Global models with manual handbrake.",
-    "long_description": "Experimental feature to enable stop and go for Subaru Global models with manual handbrake. Models with electric parking brake should keep this disabled. Thanks to martinl for this implementation!"
+    "title": "Subaru Stop and Go Manual Parking Brake",
+    "description": ""
   },
   "SunnylinkCache_Roles": {
     "title": "sunnylink Cache  Roles",
@@ -1134,12 +1021,12 @@
     "description": ""
   },
   "SunnylinkDongleId": {
-    "title": "Dongle ID",
+    "title": "sunnylink Dongle ID",
     "description": ""
   },
   "SunnylinkEnabled": {
-    "title": "Enable sunnylink",
-    "description": "This is the master switch, it will allow you to cutoff any sunnylink requests should you want to do that."
+    "title": "sunnylink Enabled",
+    "description": ""
   },
   "SunnylinkTempFault": {
     "title": "sunnylink Temp Fault",
@@ -1173,12 +1060,11 @@
     "description": "",
     "min": 0.1,
     "max": 5.0,
-    "step": 0.1,
-    "unit": "m/s²"
+    "step": 0.1
   },
   "ToyotaEnforceStockLongitudinal": {
-    "title": "Enforce Factory Longitudinal Control",
-    "description": "sunnypilot will not take over control of gas and brakes. Factory Toyota longitudinal control will be used."
+    "title": "Toyota: Enforce Factory Longitudinal Control",
+    "description": "When enabled, sunnypilot will not take over control of gas and brakes. Factory Toyota longitudinal control will be used."
   },
   "TrainingVersion": {
     "title": "Training Version",

system/hardware/hw.py

@@ -95,3 +95,10 @@ class Paths:
       return str(Path(Paths.comma_home()) / "media" / "0" / "osm")
     else:
       return "/data/media/0/osm"
+
+  @staticmethod
+  def params_root() -> str:
+    if PC:
+      return str(Path(Paths.comma_home()) / "params" / "d")
+    else:
+      return "/data/params/d"