Process replay is a regression test designed to identify any changes in the output of a process. This test replays available segments through individual processes and compares the output to a known good replay.

All 16 currently selected recording inputs remain required in a full run. Volvo C1/V40 has no available recording. This exact platform is reported as an unrecorded gap in coverage.json and the GitHub job summary; it is not a passing replay. The interfaces job requires its existing host tests, and both native safety jobs require its existing C1 safety tests. New unrecorded brands, changed active Volvo platform scope, missing required test suites, and a newly available Volvo recording require coverage review and fail input accounting. Source/native tests and recorded replay outcomes remain separate evidence; neither this inventory nor the missing-input classification qualifies the fleet.

If the test fails, make sure that you didn't unintentionally change anything. If there are intentional changes, the reference logs will be updated.

Use test_processes.py to run the test locally. Log files are cached by default. Use DISABLE_FILEREADER_CACHE='1' test_processes.py to disable caching.

Currently the following processes are tested:

controlsd
radard
plannerd
calibrationd
dmonitoringd
locationd
paramsd
ubloxd
torqued

Usage

Usage: test_processes.py [-h] [--whitelist-procs PROCS] [--whitelist-cars CARS] [--blacklist-procs PROCS]
                         [--blacklist-cars CARS] [--ignore-fields FIELDS] [--ignore-msgs MSGS] [--update-refs]
Regression test to identify changes in a process's output
optional arguments:
  -h, --help            show this help message and exit
  --whitelist-procs PROCS               Whitelist given processes from the test (e.g. controlsd)
  --whitelist-cars WHITELIST_CARS       Whitelist given cars from the test (e.g. HONDA)
  --blacklist-procs BLACKLIST_PROCS     Blacklist given processes from the test (e.g. controlsd)
  --blacklist-cars BLACKLIST_CARS       Blacklist given cars from the test (e.g. HONDA)
  --ignore-fields IGNORE_FIELDS         Extra fields or msgs to ignore (e.g. driverMonitoringState.events)
  --ignore-msgs IGNORE_MSGS             Msgs to ignore (e.g. onroadEvents)
  --update-refs                         Updates reference logs using current commit

Forks

openpilot forks can use this test with their own reference logs, by default test_proccesses.py saves logs locally.

To generate new logs:

./test_processes.py

Then, check in the new logs through the external test-data catalog. Make sure to also update the ref_commit file to the current commit.

API

Process replay test suite exposes programmatic APIs for simultaneously running processes or groups of processes on provided logs.

def replay_process_with_name(name: Union[str, Iterable[str]], lr: LogIterable, *args, **kwargs) -> List[capnp._DynamicStructReader]:

def replay_process(
  cfg: Union[ProcessConfig, Iterable[ProcessConfig]], lr: LogIterable, frs: Optional[Dict[str, Any]] = None,
  fingerprint: Optional[str] = None, return_all_logs: bool = False, custom_params: Optional[Dict[str, Any]] = None, disable_progress: bool = False
) -> List[capnp._DynamicStructReader]:

Example usage:

from openpilot.selfdrive.test.process_replay import replay_process_with_name
from openpilot.tools.lib.logreader import LogReader

lr = LogReader(...)

# provide a name of the process to replay
output_logs = replay_process_with_name('locationd', lr)

# or list of names
output_logs = replay_process_with_name(['ubloxd', 'locationd'], lr)

Supported processes:

controlsd
radard
plannerd
calibrationd
dmonitoringd
locationd
paramsd
ubloxd
torqued
modeld
dmonitoringmodeld

Certain processes require saved initial state in Params, such as CalibrationParams or LiveParametersV2. Managed schema caches in custom_params must contain verified envelopes created by openpilot.starpilot.schema_cache.put_cache from current-schema messages whose origin is known. Synthetic current-schema fixtures can use this path.

Historical logs require source schema provenance and an explicit schema-specific conversion before their values can seed these caches. Reading old bytes through the current schema does not establish compatibility. get_custom_params_from_lr now refuses this operation, and automatic firmware CarParamsCache seeding also refuses unless a verified envelope was supplied. Configuration is checked before Params writes. An explicit current vehicle fingerprint can avoid firmware-cache seeding, but does not qualify the remaining logged messages.

The checked-in replay recording catalog supplies a separate, reviewed firmware cache adapter. It pins each input and its independently published card reference, decodes the reference with its producing source schemas, checks firmware/VIN identity against the recording, and converts the reference CarParams into a verified current envelope. The current vehicle factory still recomputes control and safety settings. This adapter does not alter expected outputs or authorize implicit cache conversion for other historical recordings. Input, reference or schema changes require a new review; do not regenerate expected outputs to hide differences.

Default ref_commit and unlisted expected logs use the immutable official artifact revision. Fourteen exact segment/process pairs use reviewed StarPilot references in refs/starpilot, preserving intentional controller and capability changes. Their manifest pins inputs, comparison configurations, producing source snapshots and both official and reviewed output hashes. Every original comparison remains active; an arbitrary local output cannot override a default expectation. Deliberate alternate ref_commit and --update-refs retain their local reference behavior.

The EV6 and Bolt EUV control references include the default Turn Assist behavior; explicit TurnAssist=False independently restores every prior compared field. The legacy Corolla recording contains no native axis or safety acknowledgement, so its control reference requires both axes disabled and zero torque and acceleration even when stock enable is true. Actual Controls and native Toyota safety regressions verify fresh grants and missing acknowledgement denial. The four reference updates record the prior output hash, exact difference fields and counts, regression source hashes, and committed producing sources in the manifest. These records document the reviewed changes; they do not exclude any field from replay comparison.

Use --output DIRECTORY to place coverage, new logs and both diff reports outside the source tree. The predeploy runner supplies its results directory automatically.

Replaying processes that use VisionIPC (e.g. modeld, dmonitoringmodeld) require additional frs dictionary with camera states as keys and FrameReader objects as values.

from openpilot.tools.lib.framereader import FrameReader

frs = {
  'narrowRoadCameraState': FrameReader(...),
  'wideRoadCameraState': FrameReader(...),
  'cabinCameraState': FrameReader(...),
}

output_logs = replay_process_with_name(['modeld', 'dmonitoringmodeld'], lr, frs=frs)

To capture stdout/stderr of the replayed process, captured_output_store can be provided.

output_store = dict()
# pass dictionary by reference, it will be filled with standard outputs - even if process replay fails
output_logs = replay_process_with_name(['radard', 'plannerd'], lr, captured_output_store=output_store)

# entries with captured output in format { 'out': '...', 'err': '...' } will be added to provided dictionary for each replayed process
print(output_store['radard']['out']) # radard stdout
print(output_store['radard']['err']) # radard stderr