Files
StarPilot/tools/lib
Shane Smiskol 618a25a612 LogReader: simplify sourcing logic and improve AUTO performance (#35753)
* sourcing supports any file type!

* stuff

* no camera for now

* i guess these are fine without

* rename

* get rid of these too!

* fix

* fix

* this is better

* start to clean up!

* better

* holy

holy

* clean up sources

* more robust

* working but needs some clean up

* clean up

* remove some trash

* nl

* auto_source can only return if it finds acceptable logs from sources

* double negative is confusing

* default

* list isn't hashable

* fix typing

* clean up

* speed up -- test_models got zst before bz2 in openpilotci, so do that (some segments have both bz2 and zst!)

* don't be a hero

* same behavior for now
2025-07-18 05:15:55 -07:00
..
2025-06-19 09:49:51 -07:00
2020-01-17 10:07:22 -08:00
2020-01-17 10:07:22 -08:00
2024-01-17 14:21:47 -08:00
2024-02-23 11:50:33 -08:00
2025-07-17 18:22:36 -07:00
2025-07-17 16:50:52 -07:00

LogReader

Route is a class for conveniently accessing all the logs from your routes. The LogReader class reads the non-video logs, i.e. rlog.bz2 and qlog.bz2. There's also a matching FrameReader class for reading the videos.

from openpilot.tools.lib.route import Route
from openpilot.tools.lib.logreader import LogReader

r = Route("a2a0ccea32023010|2023-07-27--13-01-19")

# get a list of paths for the route's rlog files
print(r.log_paths())

# and road camera (fcamera.hevc) files
print(r.camera_paths())

# setup a LogReader to read the route's first rlog
lr = LogReader(r.log_paths()[0])

# print out all the messages in the log
import codecs
codecs.register_error("strict", codecs.backslashreplace_errors)
for msg in lr:
  print(msg)

# setup a LogReader for the route's second qlog
lr = LogReader(r.log_paths()[1])

# print all the steering angles values from the log
for msg in lr:
  if msg.which() == "carState":
    print(msg.carState.steeringAngleDeg)

Segment Ranges

We also support a new format called a "segment range":

344c5c15b34f2d8a   /   2024-01-03--09-37-12   /     2:6    /       q
[   dongle id     ] [       timestamp        ] [ selector ]  [ query type]

you can specify which segments from a route to load

lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4")   # 4th segment
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4:6") # 4th and 5th segment
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/-1")  # last segment
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/:5")  # first 5 segments
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/1:")  # all except first segment

and can select which type of logs to grab

lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4/q") # get qlogs
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4/r") # get rlogs (default)