mawei/StarPilot
1
0
Fork 0
mirror of https://github.com/firestar5683/StarPilot.git synced 2026-10-27 08:33:45 +08:00
Code Issues Packages Projects Releases Wiki Activity

Jetlink d24328d

Browse Source 
Vendor only the comma payload jetlink commits to (jetlink/, LICENSE,
pyproject.toml, README.md) with an include list, so the exclude list goes
and upstream's README stays as published; StarPilot's notes live in
JETLINK.md. The root helper now ships inside the package.

The adapter moves to API 3 in the next commit; until then Jetlink stays
off.

d24328d stands in for the jetlink release tag commit: re-run
tools/vendor/sync.py jetlink_repo <tag commit> --apply before merging and
update the revision in CREDITS.md and JETLINK.md with it.
This commit is contained in:
Zeph Leggett
2026-10-16 03:29:23 -05:00
parent 31840b07d7
commit de40ad2ff1
37 changed files with 3747 additions and 1297 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CREDITS.md
+1 -1
View File
@@ -2,7 +2,7 @@
## Jetlink external inference
The comma-side Jetlink client and USB transport in `jetlink_repo` are by Zeph Leggett and the [Jetlink contributors](https://github.com/zoompilot/jetlink). They are vendored from revision `c17cd5c52be0ee83b9a6265b7da69d24ad6c5d02` under the MIT license, retained in [jetlink_repo/LICENSE](jetlink_repo/LICENSE). StarPilot owns the adapter to its model runner, settings and process lifecycle. Companion applications and inference servers remain distributed by Jetlink.
The comma-side Jetlink client and USB transport in `jetlink_repo` are by Zeph Leggett and the [Jetlink contributors](https://github.com/zoompilot/jetlink). They are vendored from revision `d24328d849b0e5154b55f149dc18c5dfe8bd79cc` under the MIT license, retained in [jetlink_repo/LICENSE](jetlink_repo/LICENSE). StarPilot owns the adapter to its model runner, settings and process lifecycle. Companion applications and inference servers remain distributed by Jetlink.
## MRR35 Hyundai angle and radar
jetlink_repo/README.md
+74 -15
View File
@@ -1,20 +1,79 @@
# Jetlink runtime for StarPilot
<p align="center">
<img src="macos/Resources/Assets.xcassets/AppIcon.appiconset/icon_128x128@2x.png" width="128" height="128" alt="Jetlink icon">
</p>
This subtree supplies the Jetlink remote-model client used by **StarPilot**.
For installing, connecting, selecting, and running it with StarPilot, follow
[the StarPilot JetLink guide](../openpilot/starpilot/models/JETLINK.md).
<h1 align="center">Jetlink</h1>
The comma runs StarPilot; the connected host runs the upstream Jetlink application.
Host apps, installers, development references, and original history are maintained
in [ZoomPilot / Jetlink](https://github.com/zoompilot/jetlink/tree/c17cd5c).
This runtime subtree does not contain the upstream host applications or docs;
their source attribution does not require installing ZoomPilot on the comma.
<p align="center">
Run openpilot's large driving models on a computer connected to your comma.
</p>
Jetlink is experimental. Use separate power and USB 3, set up parked, and verify
StarPilot's active model status. See the local guide for recovery and limitations.
**Jetlink is experimental.** It requires a comma running an openpilot fork
with Jetlink support: zoompilot's
[`main` release](https://github.com/zoompilot/zoompilot/tree/main), or StarPilot.
If the link drops or lags while engaged, the comma says **TAKE CONTROL** and
stays engaged on the small model. Be ready to take over.
## Attribution
## Quick start
Jetlink was developed by Zeph Leggett / ZoomPilot. Copyright (c) 2026 Zeph Leggett.
The original [MIT license](LICENSE) is preserved without modification.
[Upstream README and credits](https://github.com/zoompilot/jetlink/blob/c17cd5c/README.md).
You need a **comma 3X or comma 4**, a **USB 3 data cable**, and **separate power
for the comma and computer**. Set up offroad with both devices online.
Choose your computer. Each guide covers installation and connecting the comma.
| Computer | Setup guide |
| --- | --- |
| Jetson Orin Nano Super | [Set up a Jetson](docs/jetson.md) |
| Apple silicon Mac | [Set up a Mac](docs/macos-app.md) |
| Linux PC with an NVIDIA GPU | [Set up a Linux PC](docs/linux-pc.md) |
| iPhone or iPad with USB-C | [Install with TestFlight](docs/iphone-app.md) |
| Android phone with USB 3 | [Try the experimental Android app](docs/android-app.md) |
## Demos
### Jetson
<a href="docs/images/install-demo.mp4"><img src="docs/images/install-demo.webp" width="100%" alt="Jetlink installer on a Jetson"></a>
<details>
<summary>Watch the web page</summary>
<a href="docs/images/webui-demo.mp4"><img src="docs/images/webui-demo.webp" width="100%" alt="The Jetson's web page on a phone and a computer, with simulated drive data: sign in, frame timing against the 50 ms budget, a model downloaded and prepared (sped up), the power setting applied, and keep awake"></a>
</details>
### Mac
<a href="docs/images/mac-demo.mp4"><img src="docs/images/mac-demo.webp" width="100%" alt="Jetlink for Mac: Use Model downloads and prepares a model, the comma connects over USB, and Status shows each frame against the 50 ms budget"></a>
### iPhone and iPad (experimental)
Simulator demo.
<a href="docs/images/iphone-demo.mp4"><img src="docs/images/iphone-demo.webp" width="100%" alt="Jetlink for iPhone, recorded in the iOS Simulator: Get downloads and prepares a model, the comma connects over USB, and Status shows each frame against the 50 ms budget, with timings modeled on an iPhone 17 Pro measurement"></a>
<a href="https://testflight.apple.com/join/DAsYk5sP"><img src="docs/images/testflight-badge.svg" alt="Available on TestFlight" height="40"></a>
### Android (experimental)
Emulator demo.
<a href="docs/images/android-demo.mp4"><img src="docs/images/android-demo.webp" width="100%" alt="Jetlink for Android, recorded in the Android emulator: Get downloads a model, the comma connects over USB, and Status shows each frame against the 50 ms budget. The download is sped up, the emulator's CPU preparation cut, and the model's time modeled on an estimate for a Snapdragon 8 Gen 3"></a>
<a id="comma-setup-all-platforms"></a>
<a id="jetson-or-linux-pc"></a>
<a id="linux-pc"></a>
<a id="what-to-expect-when-driving"></a>
<a id="if-something-is-wrong"></a>
<a id="more"></a>
## Help
[Setup guides](docs/README.md#set-up) ·
[Daily use](docs/using-jetlink.md) ·
[Troubleshooting](docs/troubleshooting.md) ·
[Operating limits](docs/status.md)
## License
[MIT](LICENSE).
jetlink_repo/jetlink/__init__.py
+1 -1
View File
@@ -10,4 +10,4 @@ jetlink: run openpilot's large driving models on an attached Jetson.
# so does the release check (macos/scripts/check-version.sh). The comma runs
# jetlink from a checkout, not an install, so this is also all it has. The Swift
# reads it from Pinned.swift: run JetlinkKit/Scripts/make_pins.py after a bump.
__version__ = '0.8.3'
__version__ = '0.8.7'
jetlink_repo/jetlink/client.py
+61 -23
View File
@@ -102,12 +102,16 @@ class JetlinkClient:
# the newest model output read off the link, whichever call read it: what
# a frame whose own reply is late publishes again (JetlinkModelState.end)
self.last_output: np.ndarray | None = None
# frames sent and not yet answered, oldest first. More than one while
# frames go out without being waited for (drain), or after a reply a
# frame gave up on (infer_end with hold)
# frames sent and not yet answered, oldest first. More than one after a
# reply a frame gave up on (infer_end with hold)
self._in_flight: deque[Sent] = deque()
# seqs sent with no reply wanted (leave): an ERROR to one is not this link's
self._no_reply_wanted: set[int] = set()
# frames the server never answered this session, for the log: a lost
# datagram is one, and to the caller a held frame like any late reply
self.frames_lost = 0
# take a server's offer of frame datagrams (_take_datagrams); False keeps frames on the stream, for A/B runs
self.allow_datagrams = True
# -- construction ---------------------------------------------------------
@@ -128,8 +132,8 @@ class JetlinkClient:
"""This end is the USB gadget, over a gadget another process owns.
The owner holds ep0 and the UDC bind for as long as the link is enabled,
so a drive starting or ending is no longer an unplug the host has to
recover from. See FfsTransport.borrowed.
so a drive starting or ending is not an unplug the host has to recover
from. See FfsTransport.borrowed.
"""
from jetlink.transport.ffs import FfsTransport
return cls(FfsTransport.borrowed(mount, udc, bounce=bounce, owner_gadget=owner_gadget), **kw)
@@ -210,8 +214,8 @@ class JetlinkClient:
self._dispatch(msg)
continue
if msg.msg_type == P.Msg.INFER_RESP and any(f.seq == msg.seq for f in self._in_flight):
# the answer to a frame nobody waited for, or one a frame gave up
# waiting for: read like any other, then on to what is wanted
# the answer to a frame that gave up waiting for it (a held frame):
# read like any other, then on to what is wanted
self._take_reply(msg)
continue
# A reply a caller gave up on. Drop it, or every frame reads one behind.
@@ -238,8 +242,20 @@ class JetlinkClient:
# its last output is nothing the new model may publish
self._in_flight.clear()
self.last_output = None
self.frames_lost = 0
self.t.send_json(P.Msg.HELLO_REQ, seq, {'client': client})
return json.loads(bytes(self._expect(P.Msg.HELLO_RESP, seq, timeout).payload))
resp = json.loads(bytes(self._expect(P.Msg.HELLO_RESP, seq, timeout).payload))
self._take_datagrams(resp)
return resp
def _take_datagrams(self, hello: dict) -> None:
"""Frames as datagrams when the server offers them (only a phone's server,
over the cable). A session that offers none puts frames back on the link."""
port, token = hello.get('frame_port'), hello.get('frame_token')
if port and token is not None and self.allow_datagrams and self.t.use_datagrams(int(port), int(token)):
log.info("frames go to the server as datagrams (port %d)", int(port))
else:
self.t.stop_datagrams()
def state(self, timeout: float = 2.0) -> dict:
seq = self._next_seq()
@@ -379,36 +395,51 @@ class JetlinkClient:
def infer_begin(self, warped: np.ndarray, packed: np.ndarray, frame_id: int = 0,
reset: bool = False, want_state: bool = False, deadline: float | None = None,
silence: float | None = None) -> int:
skip_if_busy: bool = False) -> int | None:
"""Send a frame and return immediately with its sequence number.
Split from infer_end so the caller can work while the Jetson is busy;
openpilot publishes chestnutState in that window. A frame nobody calls
infer_end for is read by the next drain or infer_end. A host that has
answered none of them for `silence` (the client's deadline by default)
is gone, not slow, and the link is done here, before another frame goes
out to it. `deadline` bounds this frame's send.
openpilot publishes chestnutState in that window. A frame whose reply
infer_end stopped waiting for is read by the next drain or infer_end. A
host that has answered none of them for the client's deadline is gone,
not slow, and the link is done here, before another frame goes out to
it. `deadline` bounds this frame's send.
`skip_if_busy` says the caller can do without this frame's reply: it
holds the last output, as JetlinkModelState.end does. A frame the link
cannot take without waiting for the host to drain earlier ones is then
not sent, and None comes back: the link is as good as it was
(Transport.try_send). Over a phone's cable it goes as datagrams instead,
where a lost one is a frame the server never answers, and the next reply
takes it out of flight (_take_reply).
"""
if self.spec is None:
raise LinkError("ensure_engine() first")
if self.dead:
raise LinkError("link previously failed")
silence = self.deadline if silence is None else silence
if self.waiting_for() > silence:
if self.waiting_for() > self.deadline:
self.dead = True
raise LinkError(f"no answer to {len(self._in_flight)} frames in {silence:.1f}s; link abandoned")
raise LinkError(f"no answer to {len(self._in_flight)} frames in {self.deadline:.1f}s; link abandoned")
warped = _as_bytes(warped, self.spec.warped_nbytes, 'warped')
packed = _as_bytes(packed, self.spec.packed_nbytes, 'packed')
seq = self._next_seq()
flags = ((P.Flag.RESET_QUEUES if reset else 0) | (P.Flag.WANT_STATE if want_state else 0)
| (P.Flag.WANT_HIDDEN if self.want_hidden else 0))
parts = (P.pack_infer_req(frame_id, flags), warped, packed)
sent = Sent(seq, frame_id, flags, time.monotonic())
timeout = self.deadline if deadline is None else deadline
try:
self._in_flight.append(Sent(seq, frame_id, flags, time.monotonic()))
self.t.send(P.Msg.INFER_REQ, seq, (P.pack_infer_req(frame_id, flags), warped, packed),
timeout=self.deadline if deadline is None else deadline)
if skip_if_busy and self.t.datagrams:
self.t.send_datagrams(P.Msg.INFER_REQ, seq, parts)
elif skip_if_busy:
if not self.t.try_send(P.Msg.INFER_REQ, seq, parts, timeout=timeout):
return None # the seq is skipped; replies are matched by seq, not counted
else:
self.t.send(P.Msg.INFER_REQ, seq, parts, timeout=timeout)
except LinkError:
self.dead = True
raise
self._in_flight.append(sent)
return seq
def infer_end(self, seq: int, deadline: float | None = None, hold: float | None = None) -> np.ndarray | None:
@@ -476,12 +507,15 @@ class JetlinkClient:
def _take_reply(self, msg: Message) -> np.ndarray:
"""One INFER_RESP off the stream: checked against the frame it answers,
its timings and telemetry kept, the frame and any older one taken out of
flight. Raises, with the link dead, on anything but a good answer."""
flight. Raises NotFinite on Status.NOT_FINITE, the link kept, and with
the link dead on anything else but a good answer."""
while self._in_flight and self._in_flight[0].seq != msg.seq:
# an older frame the server never answered: it answers in order, and a
# frame that fell out of its session (a hello between) is not coming
# an older frame the server never answered: it answers in order, so a
# frame that fell out of its session (a hello between), or whose
# datagrams did not all arrive, is not coming
skipped = self._in_flight.popleft()
log.warning('frame %d (seq %d) was never answered', skipped.frame_id, skipped.seq)
self.frames_lost += 1
if not self._in_flight:
self.dead = True
raise LinkError(f'inference response seq {msg.seq} answers no frame sent')
@@ -491,6 +525,10 @@ class JetlinkClient:
raise LinkError('inference response is missing its header')
fid, status, gpu_us, queue_us, total_us = P.unpack_infer_resp(msg.payload)
self.last_timings = (gpu_us, queue_us, total_us)
if status == P.Status.NOT_FINITE and fid == frame_id:
# the server's session goes on, keeping nothing of the frame
# (Session.infer), so the link stays and last_output is the one before
raise P.NotFinite(f"inference output is not finite: NOT_FINITE (frame {fid})")
if status != P.Status.OK:
self.dead = True
raise LinkError(f"inference failed: {_name(P.Status, status)} (frame {fid})")
jetlink_repo/jetlink/comma/gadget.py
+54 -39
View File
@@ -12,8 +12,8 @@ Holding ep0 needs sysfs and a unix socket; `openpilot.common.swaglog` costs
28 MB because it drags numpy, capnp and zmq in to publish a log line, and
`openpilot.common.params` imports swaglog, so a module that touches either
prices the owner out of being minimal. Measured on the comma: python plus this
plus the FunctionFS transport is 10.4 MB against 47.5 MB for the daemon that
imported the world.
plus the FunctionFS transport is 10.4 MB against 47.5 MB with openpilot's
modules imported.
Nothing in jetlink.comma may import openpilot or knows a param: the settings
come from jetlink.openpilot.settings, over the directory and the keys the
@@ -146,7 +146,7 @@ UDC_PATH = Path(UDC_SYSFS)
# the network function jetlink-root.sh gadget --ios adds after ffs.jetlink. Its
# netdev is not usb0, which the modem holds, but whatever the kernel names it
NET_FUNCTION = 'ncm.usb0'
# written by scripts/comma/jetlink-root.sh gadget, which the owner runs when the
# written by jetlink/comma/jetlink-root.sh gadget, which the owner runs when the
# link is on and there is no gadget: "ok", or "error: <reason>"
GADGET_STATUS = Path("/dev/shm/jetlink-gadget")
# the owner's own: "error: the lender could not listen: <reason>" while nothing
@@ -158,15 +158,17 @@ CC_ORIENTATION = Path('/sys/class/power_supply/usb/typec_cc_orientation')
# sleep. Presence comes from this, not the UDC; a marker whose writer is dead is
# a leftover from a kill
DORMANT = Path("/dev/shm/jetlink-dormant")
# hardwared's request to power the Jetson off; see backend.shutdown
# hardwared's request to power the Jetson off; see request_shutdown
SHUTDOWN_REQUEST = Path("/dev/shm/jetlink-shutdown")
# the owner's status record: everything the readers used to take from the
# files above one by one, rewritten whole every step, which makes it the
# owner's heartbeat too. A clean stop removes it
# the owner's status record: the owner's state in one file, rewritten whole
# every step, which makes it the owner's heartbeat too. A clean stop removes it
STATUS = Path("/dev/shm/jetlink/status.json")
# how old that record may be before a reader takes its owner for gone: six of
# its 0.5 s steps. A wait inside a step renews it (Owner.beat)
HEARTBEAT_TIMEOUT = 3.0
# how the owner's crash-loop backoff starts its record's error, which
# status.Status.reason_code reads it by
BACKING_OFF = "the service keeps stopping"
# the owner's start times, for its crash-loop backoff (see the owner)
STARTS = Path("/dev/shm/jetlink/starts.json")
# held with flock by the owner for its whole life, so there is only ever one
@@ -174,11 +176,15 @@ OWNER_LOCK = Path("/dev/shm/jetlink/owner.lock")
# what the server's last hello said (lending.SERVER_FIELDS), kept apart from
# the status record so a clean stop leaves it for the next owner
SERVER = Path("/dev/shm/jetlink/server.json")
# how long a host that stopped reading configured still counts as there. The
# owner holds the gadget for as long as the link is enabled, so presence no
# longer blinks at every handover; what is left to bridge is a USB3 link
# recovery passing through "addressed", and a bounce made on purpose when a
# host will not enumerate (wait_for_host)
# the joining model's runtime record, rewritten by a thread in modeld once a
# second and on every handover: whether the big model drives, for a reader
# with no modelV2 (status.runtime). modeld's close() leaves it stopping
MODELD = Path("/dev/shm/jetlink/modeld.json")
# how often modeld rewrites that record without a handover
RECORD_PERIOD = 1.0
# how long a host that stopped reading configured still counts as there: long
# enough to bridge a USB3 link recovery passing through "addressed", and a
# bounce made on purpose when a host will not enumerate (wait_for_host)
PRESENCE_HOLD = 5.0
@@ -199,22 +205,31 @@ def write_record(path: Path, record) -> None:
def clear_leftovers(directory: Path) -> None:
"""The temporaries a writer killed between its write and its rename left
in `directory`. For the owner's start, when nothing else writes there."""
in `directory`, for the owner's start. Only a dead writer's (write_record
names them by pid): modeld writes its runtime record there too, and a
live one's is mid-write."""
try:
for leftover in directory.glob('.*.json.*'):
leftover.unlink(missing_ok=True)
pid = leftover.name.rsplit('.', 1)[-1]
if not (pid.isdigit() and pid_alive(int(pid))):
leftover.unlink(missing_ok=True)
except OSError:
pass
def read_record(path: Path) -> dict | None:
"""A JSON record write_record wrote, or None: missing, or not a record."""
try:
record = json.loads(read(path))
except ValueError:
return None
return record if isinstance(record, dict) else None
def owner_status() -> dict | None:
"""The owner's status record, or None without one: no owner has run since
boot, the last one stopped cleanly, or it is one too old to write it."""
try:
record = json.loads(read(STATUS))
except ValueError:
return None
return record if isinstance(record, dict) else None
return read_record(STATUS)
def owner_alive(record: dict) -> bool:
@@ -316,8 +331,8 @@ def wait_for_host(timeout: float, bounce=None, should_stop=None, report=None, mo
"""Wait for the Jetson to enumerate us, bouncing a bus that stalled.
The gadget stays bound throughout. An unbind is an unplug as the far end sees
it, and while one boots it takes ~50 s a cycle: doing that on a timer landed
an unplug on a box that was seconds from enumerating.
it, and while one boots it takes ~50 s a cycle, so an unbind on a timer can
land on a box seconds from enumerating.
The one case that needs an edge is a host that took the bind as a wake, reset
the bus and stopped. The UDC then sits in default or addressed with the CC
@@ -420,9 +435,15 @@ def dormant() -> bool:
pid = int(read(DORMANT))
except ValueError:
return False
return pid_alive(pid)
def pid_alive(pid) -> bool:
if not isinstance(pid, int) or isinstance(pid, bool) or pid <= 0:
return False # 0 and below name process groups, never one process
try:
os.kill(pid, 0)
except ProcessLookupError:
except (ProcessLookupError, OverflowError, ValueError):
return False
except PermissionError:
pass # alive, just not ours to signal
@@ -430,7 +451,15 @@ def dormant() -> bool:
def request_shutdown(reason: str) -> bool:
return _write(SHUTDOWN_REQUEST, json.dumps({'reason': reason}), "write the shutdown request")
"""Hand the owner a power-off request, whole or not at all: a partial one (a
full /dev/shm) is one the owner cannot read, so it never takes it. False,
and logged, when it cannot be written."""
try:
write_record(SHUTDOWN_REQUEST, {'reason': reason})
return True
except OSError:
log.exception("jetlink: could not write the shutdown request")
return False
def pending_shutdown() -> str | None:
@@ -441,22 +470,8 @@ def pending_shutdown() -> str | None:
"""
if not SHUTDOWN_REQUEST.exists():
return None
try:
return str(json.loads(read(SHUTDOWN_REQUEST)).get('reason', ''))
except ValueError:
return None
def await_shutdown(timeout: float, poll: float = 0.25) -> bool:
"""Wait for the owner's run to take a shutdown request. False if nobody did
within `timeout`, and then the request is withdrawn."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if not SHUTDOWN_REQUEST.exists():
return True
time.sleep(poll)
finish_shutdown()
return False
record = read_record(SHUTDOWN_REQUEST)
return None if record is None else str(record.get('reason', ''))
def finish_shutdown() -> None:
jetlink_repo/scripts/comma/jetlink-root.sh → jetlink_repo/jetlink/comma/jetlink-root.sh
+162 -65
View File
@@ -6,23 +6,25 @@
# Everything jetlink does as root on the comma, in one place. The owner runs it
# through jetlink/comma/root.py, which is sudo -n and a timeout:
#
# sudo scripts/comma/jetlink-root.sh gadget # the gadget for a Jetson or a Mac
# sudo scripts/comma/jetlink-root.sh gadget --ios # the gadget for an iPhone
# sudo scripts/comma/jetlink-root.sh net # after a bind, iOS only: address, DHCP
# sudo scripts/comma/jetlink-root.sh check # what is there now
# sudo scripts/comma/jetlink-root.sh teardown
# sudo scripts/comma/jetlink-root.sh port hold|off # the USB-C port held as the device, or let go
# sudo scripts/comma/jetlink-root.sh port device|reset # ask the far end to host over USB PD, or reset PD
# sudo scripts/comma/jetlink-root.sh port source # ask the far end for the source role: the comma charges it
# sudo scripts/comma/jetlink-root.sh udc apply|restore # the USB device controller kept a device, or stock
# sudo scripts/comma/jetlink-root.sh udc start|stop # its device side turned on, or off
# sudo scripts/comma/jetlink-root.sh vm apply|restore # the link's VM tuning, or the stock values
# sudo scripts/comma/jetlink-root.sh draw off|on # no current drawn from the port (iOS), or the charger's own
# sudo jetlink/comma/jetlink-root.sh gadget # the gadget for a Jetson or a Mac
# sudo jetlink/comma/jetlink-root.sh gadget --ios # the gadget for an iPhone
# sudo jetlink/comma/jetlink-root.sh net # after a bind, iOS only: address, DHCP
# sudo jetlink/comma/jetlink-root.sh rps # the cable's receive work onto core 6, read back
# sudo jetlink/comma/jetlink-root.sh check # what is there now
# sudo jetlink/comma/jetlink-root.sh teardown
# sudo jetlink/comma/jetlink-root.sh port hold|off # the USB-C port held as the device, or let go
# sudo jetlink/comma/jetlink-root.sh port device|reset # ask the far end to host over USB PD, or reset PD
# sudo jetlink/comma/jetlink-root.sh port source # ask the far end for the source role: the comma charges it
# sudo jetlink/comma/jetlink-root.sh udc apply|restore # the USB device controller kept a device and its debug log off, or stock
# sudo jetlink/comma/jetlink-root.sh udc start|stop # its device side turned on, or off
# sudo jetlink/comma/jetlink-root.sh vm apply|restore # the link's VM tuning, or the stock values
# sudo jetlink/comma/jetlink-root.sh draw off|on # no current drawn from the port, or the charger's own
# sudo jetlink/comma/jetlink-root.sh boot SETTING # at boot: draw off while the Jetlink setting's file is not 0
#
# For the comma four and the comma 3X only. Both are SDM845 on the same AGNOS
# kernel (4.9, dwc3 at a600000.dwc3), with configfs, FunctionFS and NCM built
# in and no modules to load.
# The 3X is assumed from the kernel it shares with the four; only the four has been on the bench.
# The 3X is assumed from the kernel it shares with the four; only the four is tested.
#
# gadget: the USB gadget, chosen by the comma's Jetlink setting. The
# comma is the USB device and the host runs the model; docs/transport.md says
@@ -44,7 +46,7 @@
# $NET_STATUS_FILE; it never fails the gadget.
set -euo pipefail
GADGET=/sys/kernel/config/usb_gadget/jetlink
GADGET=${JETLINK_GADGET:-/sys/kernel/config/usb_gadget/jetlink}
FFS_MOUNT=/dev/ffs-jetlink
FFS_NAME=jetlink
CONFIGFS=/sys/kernel/config
@@ -68,6 +70,25 @@ DHCP_RANGE=192.168.60.2,192.168.60.254,10m
DNSMASQ_PID=/dev/shm/jetlink-dnsmasq.pid
DNSMASQ_IF=/dev/shm/jetlink-dnsmasq.if
DNSMASQ_LEASES=/dev/shm/jetlink-usb0.leases
SYS_NET=${JETLINK_SYS_NET:-/sys/class/net}
# The cable's receive work (RPS): core 6, the one big core openpilot runs no
# real-time thread on. It clocks out the rest of each frame and takes the
# phone's reply, and under load 4.9 hands it to ksoftirqd, which waits behind
# any SCHED_FIFO thread on its core. With f0 (cores 4-7) each connection hashed
# to one of them: 4 is card/controlsd/selfdrived, 5 the ui, 87 and 77 % busy on
# a drive. A ui-like FIFO 53 load on the receive core took the link from 7.6 to
# 12.4 ms (p90 36 to 47 ms round trip); on core 6, 7.7.
# The kernel keeps only the cores online at the write, so with the big cores
# parked it reads 00: the owner steers again once a drive starts.
RPS_CPUS=40
# No connection tracking on the cable. AGNOS builds conntrack and the legacy
# tables in, so every packet to and from the phone would be tracked and walk four
# empty tables: NOTRACK in raw for the cable's netdev takes 0.37 ms off a TCP
# frame's send, and covers the TCP that frame
# datagrams fall back to. The rules carry a comment and are replaced as a set:
# the netdev's name moves between binds, and usb0 is the modem's when it has one.
IPTABLES=${JETLINK_IPTABLES:-iptables-legacy}
TRACK_TAG=jetlink
# port: the charger's DISABLE_POWER_ROLE_SWITCH voter on the PMI8998, the one
# role lever that holds across plugs, and the policy engine's own USB PD
@@ -85,11 +106,18 @@ DUAL_ROLE=${JETLINK_DUAL_ROLE:-/sys/class/dual_role_usb/otg_default}
# its data lines only after USB PD, and at boot no gadget is bound for 10 s.
UDC_GLUE=${JETLINK_UDC_GLUE:-/sys/devices/platform/soc/a600000.ssusb}
PE_PARAMS=${JETLINK_PE_PARAMS:-/sys/module/policy_engine/parameters}
# And Qualcomm's FunctionFS logs four lines to an IPC debug ring for every USB
# request, inside io_submit on modeld's frame thread: about 200 a frame, into
# a buffer only debugfs reads. Turned off while recording, the frame's
# io_submit goes from 1.20 to 0.88 ms and frames from 21.14 to 20.86 ms at
# p50. 1 turns it off; stock is 0.
FFS_LOG_OFF=${JETLINK_FFS_LOG_OFF:-/sys/kernel/debug/ipc_logging/f_fs/log_disable}
USB_PSY=${JETLINK_USB_PSY:-/sys/class/power_supply/usb}
# draw: the charger's USB_ICL voter, the input current limit on the port. As
# the sink the charger draws up to 900 mA from whatever powers the port, though
# the comma runs on its own supply. An iPhone powers what it plugs into with
# the comma runs on its own supply. A 3X browns out the moment it draws from a
# laptop (PMIC power-off reason UVLO). An iPhone powers what it plugs into with
# little current; under that draw its supply sags, the charger's weak-charger
# and reverse-boost handlers cut and restore the input in a loop, and the phone
# never sees a device. Forced to 0 the input is suspended: the port still reads
@@ -99,21 +127,24 @@ USB_ICL_VOTER=${JETLINK_USB_ICL_VOTER:-/sys/kernel/debug/pmic-votable/USB_ICL}
# vm: loggerd's dirty pages pile up until the kernel reclaims them
# synchronously, right while a FunctionFS transfer allocates its buffer: gadget
# reads stalled 200-350 ms and the big model fell back. Capping dirty memory
# reads stall 200-350 ms and the big model falls back. Capping dirty memory
# keeps that reclaim cheap. System-wide, since the gadget read shares the
# kernel with every writer.
# Not vm.min_free_kbytes: a 128 MB floor takes about three times that out of
# MemAvailable, 360 MB on a 3.6 GB comma, and openpilot's LOW MEMORY alert
# reads MemTotal-MemAvailable against 90 %. Drives at 80 % showed 90 and
# alerted. The caps alone are not yet re-measured against the stall.
# reads MemTotal-MemAvailable against 90 %, so a drive at 80 % would alert. vm.extra_free_kbytes (Android's) instead has kswapd keep that much
# more free ahead of the allocators, leaving the direct-reclaim floor where it
# is: 32 MB took 3 minutes of recording from 49 direct-reclaim stalls (the
# longest 148 ms) to none, for 2.7 points on that gauge, as the kernel takes
# it out of MemAvailable about 2.5 times; 64 MB adds nothing but costs 4.5.
# A drive sits near 71 %.
# And the socket buffer caps, for the iPhone's TCP link: transport/tcp.py asks
# for 4 MB, which turns autotuning off, and the kernel clamps it to these, 224
# KB on AGNOS and so 448 KB of buffer. A 393 KB frame in 1448-byte segments does
# not fit, so the send waited on the phone's ACKs: 6 to 50 ms, following its
# round trip. With 4 MB the frame goes in one write and the send took ~4 ms
# (a tester's comma, 2026-10-04). Only a socket that asks gets more; one set up
# not fit, so the send waits on the phone's ACKs: 6 to 50 ms, following its
# round trip. With 4 MB the frame goes in one write and the send takes ~4 ms. Only a socket that asks gets more; one set up
# before the apply keeps the old cap until the phone reconnects.
VM_SYSCTLS=(vm.dirty_bytes=16777216 vm.dirty_background_bytes=8388608
VM_SYSCTLS=(vm.dirty_bytes=16777216 vm.dirty_background_bytes=8388608 vm.extra_free_kbytes=32768
net.core.wmem_max=4194304 net.core.rmem_max=4194304)
PROC_SYS=${JETLINK_PROC_SYS:-/proc/sys}
# the stock values to write back, one key=value a line, recorded by the first
@@ -121,7 +152,7 @@ PROC_SYS=${JETLINK_PROC_SYS:-/proc/sys}
SYSCTL_PREV=${JETLINK_SYSCTL_PREV:-/dev/shm/jetlink-sysctl-prev}
usage() {
echo "usage: $0 gadget [--ios] | net | check | teardown | port hold|off|device|reset|source|sink | udc apply|restore|start|stop | vm apply|restore | draw off|on" >&2
echo "usage: $0 gadget [--ios] | net | rps | check | teardown | port hold|off|device|reset|source|sink | udc apply|restore|start|stop | vm apply|restore | draw off|on | boot SETTING" >&2
exit 2
}
@@ -160,10 +191,45 @@ dnsmasq_alive() {
net_ifname() {
local name
name=$(cat "$GADGET/functions/$NET_FN/ifname" 2>/dev/null || true)
[[ -n "$name" && -d "/sys/class/net/$name" ]] || return 1
[[ -n "$name" && -d "$SYS_NET/$name" ]] || return 1
echo "$name"
}
# The cable's receive work onto RPS_CPUS; 0 only when the mask reads it back.
steer() {
local dev=$1 mask got
mask="$SYS_NET/$dev/queues/rx-0/rps_cpus"
[[ -w "$mask" ]] || { echo "jetlink: $dev has no receive steering to set" >&2; return 1; }
put "$RPS_CPUS" "$mask" "could not steer $dev's receive work" || return 1
got=$(cat "$mask" 2>/dev/null || true)
if [[ -z "$got" ]] || (( 16#$got != 16#$RPS_CPUS )); then
echo "jetlink: $dev's receive steering reads ${got:-nothing}, not $RPS_CPUS (core 6 offline?)" >&2
return 1
fi
}
# Every raw-table rule of ours gone, each found by its comment. Never fatal;
# fails only without the tool, which untrack reads as nothing to do.
retrack() {
local rule
command -v "$IPTABLES" >/dev/null 2>&1 || return 1
while read -r rule; do
[[ -n "$rule" ]] || continue
# word splitting is the point: one rule's arguments
# shellcheck disable=SC2086
"$IPTABLES" -w -t raw $rule 2>/dev/null || true
done < <("$IPTABLES" -w -t raw -S 2>/dev/null | sed -n "/--comment $TRACK_TAG /s/^-A /-D /p")
return 0
}
# The cable's traffic, both ways, out of conntrack: ours replaced by these two.
untrack() {
local dev=$1
retrack || return 0
"$IPTABLES" -w -t raw -I PREROUTING -i "$dev" -m comment --comment "$TRACK_TAG" -j CT --notrack 2>/dev/null || true
"$IPTABLES" -w -t raw -I OUTPUT -o "$dev" -m comment --comment "$TRACK_TAG" -j CT --notrack 2>/dev/null || true
}
# The comma's end of the cable network. Idempotent, and never fatal. The owner
# runs it through net once it has bound the UDC and the interface has appeared.
net_up() {
@@ -185,11 +251,9 @@ net_up() {
net_status "error: $why"; echo "jetlink: $why" >&2
return 0
fi
# Steer receive processing onto the big cores: the comma's little cores add
# milliseconds to a 460 KB frame.
if [[ -w "/sys/class/net/$dev/queues/rx-0/rps_cpus" ]]; then
echo f0 > "/sys/class/net/$dev/queues/rx-0/rps_cpus" 2>/dev/null || true
fi
# parked big cores read back 00 here; the owner steers again onroad
steer "$dev" 2>/dev/null || true
untrack "$dev"
# DHCP for the phone, and only that: no router (option 3) and no DNS (option 6),
# so the phone keeps its default route over Wi-Fi. No DNS service (--port=0).
# dnsmasq binds the interface by name, so a netdev that came back under a new
@@ -222,6 +286,7 @@ net_down() {
kill "$pid" 2>/dev/null || true
fi
rm -f "$DNSMASQ_PID" 2>/dev/null || true
retrack || true
}
# The network function and its DHCP server, gone: for teardown, and for a USB
@@ -297,7 +362,7 @@ cmd_gadget() {
echo 0x02 > bDeviceSubClass
echo 0x01 > bDeviceProtocol
else
# the plain gadget, as it has always been: class per interface
# the plain gadget: class per interface
echo 0x0100 > bcdDevice
echo 0x00 > bDeviceClass
echo 0x00 > bDeviceSubClass
@@ -381,6 +446,13 @@ cmd_net() {
net_up
}
# Not fail(): that is the gadget's record.
cmd_rps() {
local dev
dev=$(net_ifname) || { echo "jetlink: no cable network interface to steer" >&2; exit 1; }
steer "$dev" || exit 1
}
cmd_teardown() {
local f linked=0
net_remove
@@ -499,20 +571,22 @@ cmd_port() {
# it off again once that host is gone.
cmd_udc() {
case "${1:-}" in
apply) udc_compliance Y || exit 1 ;;
restore) udc_compliance N || exit 1 ;;
apply) udc_set Y 1 || exit 1 ;;
restore) udc_set N 0 || exit 1 ;;
start) put peripheral "$UDC_GLUE/mode" "could not start the USB device controller ($UDC_GLUE/mode)" || exit 1 ;;
stop) put none "$UDC_GLUE/mode" "could not stop the USB device controller ($UDC_GLUE/mode)" || exit 1 ;;
*) usage ;;
esac
}
# Both knobs, each on its own, so one the kernel lacks does not keep the other.
udc_compliance() {
# The compliance modes to $1 and FunctionFS's debug log switch to $2, each on
# its own, so one the kernel lacks does not keep the others.
udc_set() {
local failed=0 knob
for knob in "$PE_PARAMS/usb_compliance_mode" "$UDC_GLUE/usb_compliance_mode"; do
put "$1" "$knob" "could not set $knob" || failed=1
done
put "$2" "$FFS_LOG_OFF" "could not set $FFS_LOG_OFF" || failed=1
return $failed
}
@@ -538,41 +612,50 @@ release_voter() {
{ echo "jetlink: could not release $1" >&2; return 1; }
}
# A key's value, or nothing when it cannot be read.
sysctl_read() {
local value=""
{ read -r value < "$PROC_SYS/${1//.//}"; } 2>/dev/null || true
echo "$value"
}
# One key per write, so a value the kernel rejects does not take the rest with it.
sysctl_write() {
put "$2" "$PROC_SYS/${1//.//}" "could not set $1=$2"
}
# Applied while the link is on, so a device with the link off runs stock
# values. Only restore undoes them, never an exit: a process that restored on
# exit and stopped at ignition would hand every drive the stock values. A
# reboot resets them.
# apply records the stock values, once a boot, and writes the link's; restore
# writes the record back and drops it, and without one does nothing. Either
# is safe to repeat. A reboot resets them.
vm_apply() {
local pair key value rec="" failed=0
if [[ ! -e "$SYSCTL_PREV" ]]; then
for pair in "${VM_SYSCTLS[@]}"; do
key=${pair%%=*}
value=""
{ read -r value < "$PROC_SYS/${key//.//}"; } 2>/dev/null || true
if [[ "$value" == 0 && "$key" == *_bytes ]]; then
# Stock AGNOS runs the dirty limits in ratio mode, so the *_bytes keys
# read 0, and the kernel silently drops a 0 written back to one.
# Writing the ratio key is what zeroes the bytes key, so that is the
# one to put back.
key=${key%_bytes}_ratio
value=""
{ read -r value < "$PROC_SYS/${key//.//}"; } 2>/dev/null || true
fi
if [[ -n "$value" ]]; then
rec+="$key=$value"$'\n'
else
echo "jetlink: could not read $key" >&2
fi
done
if [[ -n "$rec" ]]; then
{ printf '%s' "$rec" > "$SYSCTL_PREV" && chmod 0644 "$SYSCTL_PREV"; } 2>/dev/null ||
echo "jetlink: could not record the previous sysctls in $SYSCTL_PREV" >&2
local pair key value had="" rec="" failed=0
# A key already in the record keeps its first value: a second apply must
# not record our own. One missing from it (added by a newer jetlink
# since this boot's first apply) is recorded now, while it is stock.
[[ -e "$SYSCTL_PREV" ]] && had=$(cat "$SYSCTL_PREV" 2>/dev/null)
for pair in "${VM_SYSCTLS[@]}"; do
key=${pair%%=*}
if [[ $'\n'"$had" == *$'\n'"$key="* || $'\n'"$had" == *$'\n'"${key%_bytes}_ratio="* ]]; then
continue
fi
value=$(sysctl_read "$key")
if [[ "$value" == 0 && "$key" == *_bytes ]]; then
# Stock AGNOS runs the dirty limits in ratio mode, so the *_bytes keys
# read 0, and the kernel silently drops a 0 written back to one.
# Writing the ratio key is what zeroes the bytes key, so that is the
# one to put back.
key=${key%_bytes}_ratio
value=$(sysctl_read "$key")
fi
if [[ -n "$value" ]]; then
rec+="$key=$value"$'\n'
else
echo "jetlink: could not read $key" >&2
fi
done
if [[ -n "$rec" ]]; then
{ printf '%s' "$rec" >> "$SYSCTL_PREV" && chmod 0644 "$SYSCTL_PREV"; } 2>/dev/null ||
echo "jetlink: could not record the previous sysctls in $SYSCTL_PREV" >&2
fi
for pair in "${VM_SYSCTLS[@]}"; do
sysctl_write "${pair%%=*}" "${pair#*=}" || failed=1
@@ -605,9 +688,9 @@ cmd_vm() {
esac
}
# The port's input current while the link is iOS; see USB_ICL_VOTER. off forces
# the limit to 0 and on gives the charger its own limit back. Like udc and vm,
# undone only by on, never at an exit; a reboot clears it.
# The port's input current while the link is on; see USB_ICL_VOTER. off forces
# the limit to 0 and on gives the charger its own limit back; either is safe
# to repeat. A reboot clears it.
cmd_draw() {
case "${1:-}" in
off) force_voter "$USB_ICL_VOTER" 0 "to stop drawing from the port" || exit 1 ;;
@@ -616,16 +699,30 @@ cmd_draw() {
esac
}
# The fork's launch script, before manager: with the link on, draw nothing from
# the port from boot rather than from the owner's first step, ~20 s later. The
# comma runs off its own power port. SETTING is the Jetlink setting's params
# file; anything but 0 is on, a missing file is off.
cmd_boot() {
[[ $# -eq 1 ]] || usage
local mode
mode=$(cat "$1" 2>/dev/null) || return 0
[[ -n "$mode" && "$mode" != 0 ]] || return 0
cmd_draw off
}
cmd=${1:-}
if [[ $# -gt 0 ]]; then shift; fi
case "$cmd" in
gadget) cmd_gadget "$@" ;;
net) cmd_net ;;
rps) cmd_rps ;;
check) cmd_check ;;
teardown) cmd_teardown ;;
port) cmd_port "$@" ;;
udc) cmd_udc "$@" ;;
vm) cmd_vm "$@" ;;
draw) cmd_draw "$@" ;;
boot) cmd_boot "$@" ;;
*) usage ;;
esac
jetlink_repo/jetlink/comma/lending.py
+6 -6
View File
@@ -7,8 +7,8 @@ See the LICENSE file in the root directory for more details.
Who may do endpoint IO on the gadget, while one process owns it throughout.
The owner (owner.py) holds ep0 for as long as the link is on, and nothing else
ever does: when two processes took turns at it, every change of owner was an
unplug, a fresh libusb open and a fresh server session as the Jetson saw it.
ever does: to the Jetson, a change of who holds ep0 is an unplug, a fresh
libusb open and a fresh server session.
What still has to change hands is the right to read the endpoint files.
FunctionFS keeps a queued read queued until something completes it, so a second
@@ -119,10 +119,10 @@ class Loan:
def bounce(self) -> bool:
"""Ask the owner to take the gadget down and put it back up.
The only thing that dequeues a FunctionFS write nobody is reading is the
unbind, and the unbind belongs to whoever holds ep0. Called from the write
watchdog on a link that is already 15 s stuck, so the re-enumeration it
costs is not the expensive part.
The only thing that completes a FunctionFS write nobody is reading is the
unbind, and the unbind belongs to whoever holds ep0. Called on writes the
host has stopped taking (FfsTransport._abort_write), so the re-enumeration
it costs is not the expensive part.
"""
with self._lock:
if self._closed:
jetlink_repo/jetlink/comma/owner.py
+128 -35
View File
@@ -71,6 +71,8 @@ GADGET_SETUP_BACKOFF = 60.0
GADGET_HELD_RETRY = 2.0
# between attempts to bring the gadget's network interface up after a bind
NET_BACKOFF = 5.0
# jetlink-root.sh rps attempts a bind before the owner stops asking (ensure_steered)
STEER_TRIES = 6
# between attempts to listen for borrowers after one failed
LENDER_BACKOFF = 30.0
# between runs of the worker that found nothing to do. It costs a couple of
@@ -100,8 +102,8 @@ LOG_BYTES = 1 << 20
def logger(path: Path) -> logging.Logger:
"""The owner's log, to stderr and `path`: swaglog costs 28 MB, so the owner
keeps its own. The worker's lines go to the drive; these are for a bench
session, which wants them after the drive."""
keeps its own. The worker's lines go to the drive's log; these stay for
reading afterwards."""
log = logging.getLogger('jetlink.owner')
log.setLevel(logging.INFO)
handlers: list[logging.Handler] = [logging.StreamHandler()]
@@ -174,11 +176,17 @@ class Owner:
self.dormant = False
# the link mode the root tuning was applied for, None while stock
self.tuned: str | None = None
# the tuning put back since the link was last on: None not yet asked,
# False a restore failed (release_tuning)
self.released: bool | None = None
# the draw from the USB-C port: cut by this owner, and whether it was last
# settled for the link on (steer_draw)
self.cut = False
self.draw_for: bool | None = None
# when there was last something to do. The hold runs from here, not from
# process start: this daemon is not restarted at ignition any more, so a
# hold measured from its birth expires once and never applies again, and
# every wake released the gadget a second later with the jetson still
# coming up
# process start: this daemon lives across ignitions, so a hold measured
# from its birth would expire once and then release the gadget on every
# wake with the jetson still coming up
self.idle_since = time.monotonic()
self.next_attempt = 0.0
self.next_gadget_attempt = 0.0
@@ -194,6 +202,9 @@ class Owner:
self._peer: str | None = None # the phone published as dialed in
self.net_ready = False # usb0 configured for this bind
self.next_net_attempt = 0.0
self.steered = False # usb0's receive work read back on core 6, since the drive started
self.next_steer = 0.0
self.steer_tries = 0 # this bind's, up to STEER_TRIES
self.lender_failed = False # said once, until it listens again
self.next_lender = 0.0
self.worker: subprocess.Popen | None = None
@@ -355,6 +366,26 @@ class Owner:
return
self.next_net_attempt = now + NET_BACKOFF
self.net_ready = gadget.net_up()
self.steered, self.next_steer, self.steer_tries = False, 0.0, 0
def ensure_steered(self, offroad: bool) -> None:
"""The cable's receive work onto core 6 (jetlink-root.sh rps), once a
drive has started: net's own steer, at the bind, reads back 00 while
hardwared has the big cores parked, and the kernel never adds them later.
Retried on NET_BACKOFF, STEER_TRIES times a bind: a core 6 that stays
offline is no reason to run sudo every few seconds for a whole drive."""
if offroad or self.steered or not self.net_ready or self.steer_tries >= STEER_TRIES:
return
now = time.monotonic()
if now < self.next_steer:
return
self.next_steer = now + NET_BACKOFF
self.steer_tries += 1
self.steered = root.run('rps')
if self.steered:
gadget.log.warning("jetlink: the cable's receive work is on core 6")
elif self.steer_tries == STEER_TRIES:
gadget.log.warning("jetlink: the cable's receive work stays where the kernel puts it until the next bind")
def settle(self) -> None:
"""Put the gadget back to bound with nothing open on it.
@@ -444,8 +475,8 @@ class Owner:
# the far end may still be waking; give it the hold before letting go
self.idle_since = time.monotonic()
# after the run, not before it: a run writes JetlinkSpec itself, so a mark
# taken at spawn always differs by the time it exits and every successful
# provision started a second one
# taken at spawn would always differ by the time it exits and every
# successful provision would start a second one
self.note_marks()
return False
@@ -602,35 +633,88 @@ class Owner:
self.close_link()
self.stop_worker()
self.wake()
if self.tuned:
root.run('vm', 'restore')
root.run('udc', 'restore')
if self.tuned == 'ios':
root.run('draw', 'on', timeout=root.PORT_TIMEOUT)
self.tuned = None
self.port.off()
self.release_tuning()
return
# before the gadget: a host is never drawn from while the link comes up
self.steer_draw(True)
self.link_step(mode == 'ios')
self.released = None
if self.tuned != mode:
# jetlink-root.sh vm: the recording VM tuning the gadget's reads need;
# udc: the USB device side kept on for a host that powers the port
# (port.py); draw, iOS only: no current drawn from the port, which an
# iPhone cannot supply. While the link is on. After the step, so the
# first gadget does not wait on them. Put back only when the link is
# turned off or leaves iOS, never on exit: manager stops this at
# ignition, just as the contention starts, and a phone plugged in before
# it is back should still come up. A start assumes the draw stock, so a
# cut a dead iOS owner left lasts until the reboot that clears it
# (port.py). While the link is on. After the step, so the first gadget
# does not wait on them. Put back only when the link is not wanted
# (release_tuning)
if self.tuned is None:
root.run('vm', 'apply')
root.run('udc', 'apply')
if mode == 'ios':
if root.run('draw', 'off', timeout=root.PORT_TIMEOUT):
gadget.log.info("jetlink: drawing no current from the USB-C port, for an iPhone")
elif self.tuned == 'ios':
root.run('draw', 'on', timeout=root.PORT_TIMEOUT)
self.tuned = mode
def link_wanted(self) -> bool:
"""Is the link still wanted at a stop: set on, and no chestnut on the
port? Settings.mode() reads an unreadable param as 'off', as manager
does, so that is not wanted. The except only guards a bug: anything
raised in the stop's finally would skip the teardown after it. manager's
own chestnut test is chestnut_present(), cached 2 s, and this walks the
USB ids itself, so a chestnut mid-re-enumeration can read as absent here
and keep the tuning for that drive; accepted."""
try:
return self.settings.mode() != 'off' and not port.chestnut_attached(self.port.chestnut_ids)
except Exception:
gadget.log.exception("jetlink: could not tell whether the link is still wanted; keeping its tuning")
return True
def release_tuning(self, final: bool = False) -> bool:
"""The link is not wanted, turned off or a chestnut on the port: the
port, its draw and the root tuning go back as AGNOS boots them, at a step
and at the stop manager makes of it, which often comes before a step has
seen the setting. Any other stop keeps them for the owner manager starts
next, which applies them again; a reboot clears them.
The draw first, while a hold still keeps the far end on the port: letting
go of one can drop it for a DRP toggle, and an empty port read in that
moment would give the draw back with a host still in. It is given back
only on an empty port, so it stays cut with a chestnut on, which the
comma powers and where the cut does nothing. The tuning is restored
whatever `tuned` says, since an owner before this one may have applied
it: once when the link goes, and again at the stop (final) if that
failed. On the port's short timeout, since at a stop it comes before the
FunctionFS close, inside manager's 5 s. True when the draw is still cut."""
self.steer_draw(False)
self.port.off()
if self.released is None or (final and not self.released):
vm = root.run('vm', 'restore', timeout=root.PORT_TIMEOUT)
self.released = root.run('udc', 'restore', timeout=root.PORT_TIMEOUT) and vm
if self.released:
self.tuned = None
return self.cut
def steer_draw(self, want: bool) -> None:
"""jetlink-root.sh draw: with the link on, no current drawn from the USB-C
port, whatever is on it. The comma runs off its own power port, and drawing
from the host's only costs it: a 3X browns out the instant it draws from a
laptop (PMIC power-off reason UVLO), and an iPhone sags under the draw and
never sees a device. The launch script cuts it at boot (jetlink-root.sh
boot); this keeps it cut. A forced voter, so it holds for every plug after.
Given back only on an empty port, so the link turned off with a host still
in never lets the charger draw from it. Kept on a stop with the link
wanted, like the tuning (release_tuning). A start assumes the draw stock,
so the link off gives it back on an empty port even uncut here: an owner
before this one may have cut it, and letting go is safe to repeat."""
if want == self.draw_for:
return
if want:
self.cut = root.run('draw', 'off', timeout=root.PORT_TIMEOUT)
if self.cut:
gadget.log.info("jetlink: drawing no current from the USB-C port")
else:
if port.power_role() != 'none':
return # a host is still in; looked at again next step
root.run('draw', 'on', timeout=root.PORT_TIMEOUT)
self.cut = False
# settled, or failed, which is not retried every step
self.draw_for = want
def ensure_lender(self) -> None:
"""Listen for borrowers, or say why not and try again later.
@@ -670,6 +754,7 @@ class Owner:
return
self.attached = gadget.host_attached()
self.watch_the_port()
self.ensure_steered(offroad)
# before the worker gate: hardwared waits 25 s for this and a build in
# flight takes minutes, so a shutdown request cannot queue behind one
@@ -677,9 +762,9 @@ class Owner:
if reason is not None and not self.lender.lent:
if self.worker_running():
if self.shutting_down:
# the run asking the jetson. Stopping it here restarted it every
# step, half a second, less than it takes to start: on the bench
# nothing ever asked, and hardwared gave up after its 25 s
# the run asking the jetson. Stopping it here would restart it every
# half-second step, less than it takes to start, so nothing would
# ever ask and hardwared would give up after its 25 s
return
self.stop_worker()
if time.monotonic() < self.next_shutdown_run:
@@ -861,7 +946,7 @@ class Owner:
A power-off request ends it: hardwared waits 25 s for the owner to take
one, and the Jetson on its own supply stays on otherwise."""
self.backing_off = (f"the service keeps stopping ({died} times in {CRASH_WINDOW / 60:.0f} min), "
self.backing_off = (f"{gadget.BACKING_OFF} ({died} times in {CRASH_WINDOW / 60:.0f} min), "
f"waiting {until - time.monotonic():.0f} s before starting it again")
gadget.log.warning("jetlink: %s", self.backing_off)
self.publish_status()
@@ -881,7 +966,7 @@ class Owner:
still holds the gadget) would clear that owner's records, overwrite its
status record and release its port hold. The kernel lets an flock go with
the process however it dies, and the fd is not inherited by a run.
Without a /dev/shm to lock in, the owner runs as it did before."""
Without a /dev/shm to lock in, the owner runs unlocked."""
try:
gadget.OWNER_LOCK.parent.mkdir(parents=True, exist_ok=True)
fd = os.open(gadget.OWNER_LOCK, os.O_RDWR | os.O_CREAT, 0o644)
@@ -937,9 +1022,17 @@ class Owner:
time.sleep(max(0.0, POLL - (time.monotonic() - started)))
finally:
# first, inside manager's 5 s: it stops this when a chestnut turns up,
# and the comma has to host that. The sysctls stay: a stop here is where
# a drive begins
self.port.off()
# and the comma has to host that. The tuning and the draw go with the
# port when the link is not wanted (release_tuning)
if self.link_wanted():
self.port.off()
elif self.release_tuning(final=True):
if port.power_role() == 'source':
gadget.log.info("jetlink: the comma powers what is on the USB-C port; its draw stays cut, "
"which does nothing there, until a reboot")
else:
gadget.log.warning("jetlink: a host is still on the USB-C port; the comma draws nothing from it "
"until a reboot")
self.lender.stop()
# about this process's lender; with no owner there is nothing to borrow
# anyway, and a chestnut may be why it stopped
jetlink_repo/jetlink/comma/port.py
+2 -2
View File
@@ -11,8 +11,8 @@ host. An A-to-C cable settles which by construction: the A end only pulls CC
up, so the comma can only be the sink and the device. A C-to-C cable does not.
Both ends are dual role, and the comma can come out the source and the host:
facing a Mac, nothing enumerates; facing an iPhone or a Jetson's own USB-C
port, the far end enumerates as a device. Carrot's Jetson on C-to-C read
"Powered cable w/ sink" and enumerated as 0955:7020.
port, the far end enumerates as a device (a Jetson on C-to-C reads
"Powered cable w/ sink" and enumerates as 0955:7020).
While the link is on, a chestnut is the only thing the comma should host on
this port. So once the comma has been the source for a few seconds with no
jetlink_repo/jetlink/comma/root.py
+4 -4
View File
@@ -4,7 +4,7 @@ Copyright (c) 2026-, Zeph Leggett.
This file is part of jetlink and is licensed under the MIT License.
See the LICENSE file in the root directory for more details.
Root on the comma: scripts/comma/jetlink-root.sh under sudo -n, and nothing
Root on the comma: jetlink/comma/jetlink-root.sh under sudo -n, and nothing
else. The script leaves its own record of the gadget and its network in
/dev/shm (jetlink-gadget, jetlink-net), so a False here only says the call
failed; the reason is in the log line and, for the gadget, in those files.
@@ -27,9 +27,9 @@ log = logging.getLogger('jetlink.comma')
AGNOS = os.path.isfile('/AGNOS')
# Resolved, so that <openpilot>/jetlink, a symlink into jetlink_repo, still
# leads to the checkout's scripts.
SCRIPT = Path(__file__).resolve().parents[2] / 'scripts' / 'comma' / 'jetlink-root.sh'
# Next to this file, so the package alone is all the comma runs. A path
# through the <openpilot>/jetlink symlink is fine: bash only opens it.
SCRIPT = Path(__file__).with_name('jetlink-root.sh')
# sudo, configfs, and for iOS nmcli and dnsmasq
TIMEOUT = 30.0
jetlink_repo/jetlink/openpilot/__init__.py
+100 -41
View File
@@ -11,35 +11,42 @@ The fork implements Openpilot (interface.py) once, in an adapter module that
holds every openpilot import, binds it here (bind) and calls the methods of
the Jetlink that comes back. That is the whole contract between the two repos:
the names in __all__, the Jetlink methods and Status, which
tests/openpilot/test_api.py and test_interface.py pin. API names its version:
the adapter checks it exactly and treats any other value as jetlink being
absent, an offroad alert and no link rather than a crash. Everything else in
this package is jetlink's own and changes without notice.
tests/openpilot/test_api.py and test_interface.py pin, and the facade the
fork's hook lines call them through (hooks.py) and the kit its CI runs
(testing.py). API names its version: an adapter declares the one it was
written to, hooks.Hooks refuses one it does not accept, and the fork treats
that as jetlink being absent, an offroad alert and no link rather than a
crash. Everything else in this package is internal to jetlink and may change
at any time.
API changes only for a breaking change to the contract. A new Status field, a
new Jetlink method, or a new interface member that jetlink reads with getattr
and a fallback, is additive and keeps it.
API changes only for a breaking change to the contract: removing or renaming
anything in it, changing what one of its members means, or making an optional
member required. A new Status field, a new Jetlink method, or a new interface
member that jetlink reads with getattr and a fallback, is additive and keeps
it. A bump keeps the previous API working for one minor release, so a fork can
move its pin and then its adapter. Each release that changes the contract
says how in its CHANGELOG section (Forks, docs/publishing.md).
Imported by the resident gadget owner, so nothing heavy at module level: the
parts a heavy process uses are imported when it first uses them.
"""
from __future__ import annotations
import threading
import time
from jetlink.comma import gadget
from jetlink.openpilot.events import Flags, SwapEvents
from jetlink.openpilot.interface import MODES, STATES, Keys, ModelFace, Openpilot, OwnerConfig, conformance
from jetlink.openpilot.parts import Parts, for_this_process
from jetlink.openpilot.status import Status
API = 1
API = 3
__all__ = ['API', 'MODES', 'STATES', 'Jetlink', 'Keys', 'ModelFace', 'Openpilot', 'OwnerConfig', 'Status', 'bind',
'conformance']
'conformance', 'SHUTDOWN_TIMEOUT', 'Flags', 'SwapEvents']
# how long a caller waits for the owner's run to shut the Jetson down. Wake
# from suspend is ~8 s to a server
# how long a power-off request waits for the owner's run to shut the Jetson
# down before it is withdrawn. Wake from suspend is ~8 s to a server
SHUTDOWN_TIMEOUT = 25.0
@@ -62,6 +69,11 @@ class Jetlink:
self._failures: dict[str, str] = {}
# when this process asked for the Jetson to power off, until it was taken
self._shutdown_asked: float | None = None
# shutdown_step: has it run in this process, and has it said yes
self._stepped = False
self._may_power_off = False
# the joining model's handovers as handed_over() last saw them
self._handovers = 0
def _read(self, what: str, read, fallback):
"""read(), or fallback(error) when it raises. The readers never raise:
@@ -97,6 +109,21 @@ class Jetlink:
mode = self._mode()
return self._read('the status', lambda: status.read(self._parts, mode), lambda error: status.failed(error, mode))
def runtime(self):
"""What modeld runs (status.Runtime), the part of status() that changes
while driving, for a loop: the setting, the chestnut test and modeld's
record, nothing else. The chestnut test may walk the USB devices in sysfs,
once per parts.CHESTNUT_TTL (2 s), as status() does. None where
status().running is None for want of the link: off, or beside a chestnut."""
from jetlink.openpilot import status
return status.runtime() if self.enabled() else None
def model_state(self, ref: str) -> str | None:
"""One catalog model for the picker's list: 'ready' (built on the Jetson),
'downloaded' (on the comma) or None. Asked when a list opens."""
from jetlink.openpilot import status
return self._read('a model\'s state', lambda: status.model_state(self._parts, ref), lambda error: None)
def reason(self) -> str | None:
"""Why the link the user asked for cannot run: hardwared's offroad alert.
None with the link off. The files the gadget and the build leave, nothing
@@ -137,6 +164,10 @@ class Jetlink:
def attach(self, small, cam_w: int, cam_h: int):
"""Join the link to modeld, once the camera is up and `small` is built:
the joining model, which drives as `small` until the Jetson is there.
Before every run() it asks the adapter's in_control() (is openpilot or
MADS in control? it swaps only while not), and modeld writes onto it
`frame_drop_ratio` (its share of dropped camera frames, which hands a
lagging large model back).
None unless prepare() said yes in this process: without it the GPU's
thread would start on modeld's realtime core. If the joining model cannot
@@ -151,6 +182,23 @@ class Jetlink:
self._log.exception("jetlink load failed")
return small
def handed_over(self, model) -> bool:
"""modeld, after every run(): did it change which model drives? The
joining model counts its swaps and hand-backs (JoinedModel.handovers),
and the stall of one is not lag, nor are the drops of its frame. Any
other model has no count and never hands over. Since the last call in
this process, which runs one modeld loop."""
handovers = getattr(model, 'handovers', 0)
changed, self._handovers = handovers != self._handovers, handovers
return changed
def state(self, model) -> str:
"""What modeld publishes as the big model's state: the model's
big_model_state as it is, one of STATES on a joining model, and 'none' for
a model without one. A fork's capnp enum holds exactly STATES, so a value
outside it would raise on the frame thread."""
return getattr(model, 'big_model_state', 'none')
def request_shutdown(self, reason: str = '') -> bool:
"""The device is powering off for good: ask for the Jetson to go down
with it, and return at once. True when the request now waits for the
@@ -178,9 +226,47 @@ class Jetlink:
the Jetson powered off later, when nobody wants it off, and the log says
the request went unanswered."""
try:
pending = gadget.SHUTDOWN_REQUEST.exists()
return self._pending()
except Exception:
return False
def shutdown_step(self, should_shutdown: bool, reason: str = '') -> bool:
"""hardwared, every loop: may the device power off now? True once it may
(put DoShutdown), and from then on. False while should_shutdown is False
and nothing has been asked. The first True asks once (request_shutdown):
True at once when there was nothing to ask, otherwise once the owner has
taken the request, or SHUTDOWN_TIMEOUT after asking, when it is withdrawn.
Once asked, should_shutdown is not read again, and the caller keeps no
timer. A stat a call. Never raises: a failure once a power-off is wanted
is a yes, since a jetlink bug must not keep a comma on to drain the car's
battery, and one before then is a no."""
if self._may_power_off:
return True
try:
first, self._stepped = not self._stepped, True
# a request this process did not write: a partial one, or one from a
# hardwared that has since restarted. Left, the owner would power the
# Jetson off whenever it next saw one, maybe a drive later
if first and self._shutdown_asked is None:
gadget.finish_shutdown()
if self._shutdown_asked is None and not should_shutdown:
return False
asked = self._shutdown_asked is not None or self.request_shutdown(reason)
self._may_power_off = not asked or not self._pending()
except Exception:
self._log.exception("jetlink: the shutdown step failed")
if self._shutdown_asked is not None:
# nobody follows the request any more, so it must not stay for the owner
try:
gadget.finish_shutdown()
except Exception:
pass
self._may_power_off = should_shutdown or self._shutdown_asked is not None
return self._may_power_off
def _pending(self) -> bool:
"""shutdown_pending(), raising."""
pending = gadget.SHUTDOWN_REQUEST.exists()
asked, now = self._shutdown_asked, time.monotonic()
if asked is None:
return pending
@@ -195,39 +281,12 @@ class Jetlink:
self._shutdown_asked = None
return pending
def shutdown(self, reason: str = '', timeout: float = SHUTDOWN_TIMEOUT) -> None:
"""request_shutdown(), then wait up to `timeout` for the owner to take it,
on a thread abandoned at the deadline: for a caller with nothing to do
meanwhile. hardwared keeps publishing, and uses the two above."""
if not self.enabled():
return
def request():
try:
self._request_shutdown(reason, timeout)
except Exception:
self._log.exception("jetlink: shutdown request failed")
t = threading.Thread(target=request, name='accelerator-shutdown', daemon=True)
t.start()
t.join(timeout)
if t.is_alive():
self._log.warning("jetlink: shutdown request still pending after %.0f s, going on without it", timeout)
def _request_shutdown(self, reason: str, timeout: float) -> None:
if not self._ask_for_power_off(reason):
return
if gadget.await_shutdown(timeout):
self._log.warning("jetlink: shutdown request handed to the jetson")
else:
self._log.warning("jetlink: nobody took the shutdown request within %.0f s", timeout)
def _ask_for_power_off(self, reason: str) -> bool:
"""Hand the request to the owner: nothing else can touch the link, and
the request is a file (gadget.SHUTDOWN_REQUEST) the owner's step looks
for before anything else. Skipped when no Jetson is known to be there
(dormant counts as there). A run busy in a long provision is stopped for
it; the caller's timeout covers a Jetson that never answers."""
it; shutdown_pending()'s deadline covers a Jetson that never answers."""
from jetlink.openpilot.status import Presence
if not self.enabled():
return False
jetlink_repo/jetlink/openpilot/events.py
+115
View File
@@ -0,0 +1,115 @@
"""
Copyright (c) 2026-, Zeph Leggett.
This file is part of jetlink and is licensed under the MIT License.
See the LICENSE file in the root directory for more details.
The onroad events of a big model that joins mid-drive, as flags. A native big
model loads before the first modelV2; this one joins onto a modelV2 the small
model already publishes, and can leave and come back. Each fork's selfdrived
calls SwapEvents.update once a tick and maps the flags onto its own events.
It swaps in only while nothing is in control, so the driver is told once when
it is ready (offer) and re-engages to use it. For a second after a swap nothing
engages (no_entry) while the big model builds its history and proves it keeps
up (the joining model hands back on a held frame in that second), and the chime
at the end of it says the driver can. When it leaves, the small model drives on
and the driver is told (link_lost); nothing disengages.
The fork's own chime on a rising modelV2.big, a native big model's, must not
fire for a joined one, or it says engage now at the start of the no-entry
second: gate it on the state being 'none', or at least drop it on no_entry
(zoompilot drops it on no_entry or chime, then adds it on chime).
Either way the switch costs modeld a frame or two, and a frame the camera
dropped is an invalid pose from the model and a localizer soft disable on the
next tick (TAKE CONTROL IMMEDIATELY). So a switch is `settling` for a second, which selfdrived must
treat as it does a native big model's load: hold the comm and localization
errors that frame raises.
"""
from __future__ import annotations
from typing import NamedTuple
# how long each lasts, in seconds; SwapEvents counts them in the caller's ticks.
# The link-lost warning covers about the time the small model takes to refill
# its history, the no-entry about 20 frames of the big model's. An enable inside
# that second is refused, and pressed again after the chime
OFFER_S = 3.
HANDBACK_S = 5.
SWITCHING_S = 1.
class Flags(NamedTuple):
offer: bool # ready while something is in control: re-engage to switch
link_lost: bool # it fell while in control; the small model drives on
no_entry: bool # the second after a swap: refuse an engagement
chime: bool # the end of that second: the big model drives, engage now
settling: bool # the second after a switch either way: forgive its dropped frame
def _tick(left: int) -> tuple[bool, int]:
return left > 0, max(left - 1, 0)
class SwapEvents:
def __init__(self, tick_s: float):
# checked once here: update() runs in selfdrived's loop and never raises
if not tick_s > 0 or round(SWITCHING_S / tick_s) < 1:
raise ValueError(f"SwapEvents needs the loop period in seconds, short enough to count a {SWITCHING_S:g} s switch: {tick_s}")
self.offer_ticks = round(OFFER_S / tick_s)
self.handback_ticks = round(HANDBACK_S / tick_s)
self.switching_ticks = round(SWITCHING_S / tick_s)
self.offered = False
self.running = False
# the state as last seen fresh: a gap in the fork's status or message holds
# it, never reads as the link going
self._state = 'none'
# ticks left of each event, and of the settling after a switch either way
self._offer = self._handback = self._switching = self._settle = 0
@property
def settling(self) -> bool:
return self._settle > 0
def update(self, fresh: bool, big: bool, in_control: bool, state: str) -> Flags:
"""One tick of the caller's loop, in hooks.SwapTracker.update's order.
`fresh`: modelV2 and the fork's state message are seen, alive and valid
this tick. `big`: modelV2.big, from a modelV2 alive and valid this tick.
`in_control`: the fork's meaning of something steering or able to
(zoompilot: openpilot or MADS, MADS even paused), the same rule the swap
waits on (the adapter's engaged()). `state`: the joining model's
big_model_state, its STATES name (str() of a capnp enum, never its .raw
number, which matches no state)."""
# a stale state neither offers the switch nor rearms the offer
if fresh:
self._state = state
if self._state != 'ready' or big:
# ended by the swap as well as by the link going: an offer still on
# screen after the switch reads as if it had not happened
self.offered = False
self._offer = 0
elif in_control and not self.offered and self._handback == 0:
# with nothing in control it swaps at once and the chime says so. Once
# per readiness, not at every stop, and after the link-lost warning,
# which would hide it
self.offered = True
self._offer = self.offer_ticks
# a chestnut dropping modelV2.big is its own fork's failure event, not this
running = bool(big) and self._state != 'none'
if running != self.running:
self._settle = self.switching_ticks + 1 # counted down below, this tick included
self._switching = self.switching_ticks + 1 if running else 0 # the no-entry, then the chime
if not running and in_control:
self._handback = self.handback_ticks
self.running = running
self._settle = max(self._settle - 1, 0)
if not in_control:
self._handback = 0
offer, self._offer = _tick(self._offer)
link_lost, self._handback = _tick(self._handback)
switching, self._switching = _tick(self._switching)
return Flags(offer, link_lost, switching and self._switching > 0, switching and self._switching == 0, self.settling)
jetlink_repo/jetlink/openpilot/hooks.py
+292
View File
@@ -0,0 +1,292 @@
"""
Copyright (c) 2026-, Zeph Leggett.
This file is part of jetlink and is licensed under the MIT License.
See the LICENSE file in the root directory for more details.
What a fork's hook lines call: Jetlink's methods, behind a net that never
raises.
manager, hardwared, the UI and the model manager call these on every device,
link on or off, and modeld on every drive: whatever jetlink does wrong turns
the link off and is logged, and never takes one of them down. Each hook has
the answer it gets with the link off, and gives it on any failure. They are
not all False: attach() keeps modeld's model, extend_catalog() the catalog,
and shutdown_step() says yes to a power-off that is wanted, since a broken
jetlink must never keep a comma on to drain the car's battery. selfdrived's
swap events (swap_events()) answer no event at all.
The fork's guard, which builds Hooks or else Off inside a function, and its
own fallback for no jetlink at all: docs/porting.md, "The adapter module".
Off is for a jetlink that imports but cannot be used (another API); link_on()
is the fork's stdlib read of the setting, so the reason a link cannot run is
told only to someone who turned it on.
Imported by every process that imports the fork's hook module, the resident
gadget owner among them: the standard library and jetlink's light half only.
Construction checks the API and nothing more; the adapter is built and bound
on the first hook that needs it, once per process. handed_over() and state()
read the model alone and never bind. With link_on() saying the link is off,
the only hooks that bind are BINDS_WITH_LINK_OFF's; every other one answers
as the link off, modeld's prepare() among them.
"""
from __future__ import annotations
import threading
from collections.abc import Callable
from jetlink.openpilot import API, Flags, Jetlink, SwapEvents, bind
# the APIs a fork's adapter may be written to: today's, and after a bump the
# previous one too, for one minor release (jetlink/openpilot/__init__.py)
COMPATIBLE = frozenset({API})
# the hooks that ask jetlink with link_on() saying off. status(): the UI shows
# the big model's name from the model library. shutdown_step(): hardwared
# clears, from boot, a power-off request it did not write; and the request
# and its wait, for a link turned off while one is pending. The model
# manager's: the catalog is hardware and the library, not the setting
BINDS_WITH_LINK_OFF = frozenset({'status', 'shutdown_step', 'request_shutdown', 'shutdown_pending', 'model_state',
'should_extend_catalog', 'extend_catalog'})
# what selfdrived is told when the swap events cannot run: none at all
NO_EVENTS = Flags(False, False, False, False, False)
# how often the swap events read modeld's runtime record without a state of the fork's own
RUNTIME_POLL_S = 0.2
def _text(error: BaseException) -> str:
"""An error as a log line says it, from an error whose str() may raise too."""
try:
return f"{type(error).__name__}: {error}"
except Exception:
return type(error).__name__
class _Hooks:
"""The hooks and the answer each gets with the link off. _call(name,
default, *args) is Jetlink.<name>(*args), or default with the link off;
_run() the same without asking link_on()."""
def __init__(self, reason: str = '', link_on: Callable[[], bool] | None = None):
self._why = reason
self._link_on = link_on
# the joining model's handovers as handed_over() last saw them, in this process
self._handovers = 0
self._prepared = False
def _link(self) -> bool | None:
"""link_on()'s answer, or None without one or for one that raises, which
counts as on: the hook asks jetlink, as without link_on()."""
try:
return None if self._link_on is None else bool(self._link_on())
except Exception:
return None
def _call(self, name: str, default, *args):
if name not in BINDS_WITH_LINK_OFF and self._link() is False:
return default
return self._run(name, default, *args)
def _run(self, name: str, default, *args):
return default
def should_run(self, started, params, CP) -> bool:
"""manager's should_run for the gadget owner."""
return self._call('enabled', False)
def status(self):
return self._call('status', None)
def reason(self) -> str | None:
"""Why jetlink cannot run here, told to someone who turned the link on,
else Jetlink.reason's. The call first: a bind that fails sets the why.
Without link_on() the why is told; with one that raises, nobody is."""
on = True if self._link_on is None else self._link()
if on is False:
return None
answer = self._run('reason', None)
return self._why if self._why and on else answer
def prepare(self, chestnut: bool) -> bool:
"""modeld, before config_realtime_process. A chestnut runs the large
model itself and never asks jetlink."""
self._prepared = bool(not chestnut and self._call('prepare', False))
return self._prepared
def attach(self, model, small, cam_w: int, cam_h: int):
"""modeld, once `small` is built: the model to run, `model` unless the link
joined. Binds nothing unless prepare() said yes in this process."""
if not self._prepared:
return model
# past prepare()'s yes: the setting has had its say for this modeld
joined = self._run('attach', None, small, cam_w, cam_h)
return model if joined is None else joined
# Jetlink's own two, which read the model and self._handovers alone: modeld
# asks every frame, so they never bind
def handed_over(self, model) -> bool:
try:
return bool(Jetlink.handed_over(self, model))
except Exception:
return False
def state(self, model) -> str:
try:
return Jetlink.state(self, model)
except Exception:
return 'none'
def runtime(self):
"""What modeld runs, from its runtime record (Jetlink.runtime): a
status.Runtime, or None with the link off or unusable, or a chestnut
fitted. Cheap enough for selfdrived's loop: two small files, and the
chestnut test's USB walk once per parts.CHESTNUT_TTL, as status()."""
return self._call('runtime', None)
def shutdown_step(self, should_shutdown: bool, reason: str = '') -> bool:
return self._call('shutdown_step', should_shutdown, should_shutdown, reason)
def request_shutdown(self, reason: str = '') -> bool:
return self._call('request_shutdown', False, reason)
def shutdown_pending(self) -> bool:
return self._call('shutdown_pending', False)
def model_state(self, ref: str) -> str | None:
return self._call('model_state', None, ref)
def should_extend_catalog(self) -> bool:
return self._call('should_extend_catalog', False)
def extend_catalog(self, catalog: dict) -> dict:
return self._call('extend_catalog', catalog, catalog)
def swap_events(self, tick_s: float) -> SwapTracker:
"""selfdrived's swap-event tracker, updated every `tick_s` (events.SwapEvents)."""
return SwapTracker(None, None, tick_s)
class Off(_Hooks):
"""The link off on every hook; reason() says why, when the link is on."""
class Hooks(_Hooks):
"""Jetlink over the fork's adapter, for an adapter written to `api`.
adapter_factory() makes it, on first use. Raises at once for an API this
jetlink does not accept. `log` is a logger, or a function that returns one,
for what fails before the adapter's log is there."""
def __init__(self, api: int, adapter_factory: Callable[[], object], *, link_on: Callable[[], bool] | None = None,
log=None):
if api not in COMPATIBLE:
raise ValueError(f"jetlink package API {API}, this build expects {api}")
super().__init__('', link_on)
self._factory = adapter_factory
# the adapter's once bound; until then the fork's, or the standard library's
self._log = log
self._bound = None
self._binding = threading.Lock()
# hook -> the failure last logged for it, cleared by a call that works
self._failed: dict[str, str] = {}
def _jetlink(self):
"""Bound once, the failure included: one Jetlink per process, so that
prepare() and attach() reach the same one."""
if self._bound is None:
with self._binding:
if self._bound is None:
self._bound = self._bind()
return self._bound or None
def _bind(self):
try:
op = self._factory()
self._log = getattr(op, 'log', None) or self._log
return bind(op)
except Exception as e:
self._why = f"jetlink failed to start: {_text(e)}"
self._report("jetlink: %s", self._why)
return False
def _report(self, msg: str, *args) -> None:
"""A failure to the log, in the except block that caught it. A log that
fails is let go: nothing a hook does may raise."""
try:
if not hasattr(self._log, 'exception'):
try:
log = self._log() # a factory; None or a logger raises here too
except Exception:
log = None
if not hasattr(log, 'exception'):
import logging
log = logging.getLogger('jetlink')
self._log = log
self._log.exception(msg, *args)
except Exception:
pass
def _net(self, name: str, default, fn, *args):
"""fn(*args), or default when it raises. Logged once per distinct error
(the UI would log a failing status five times a second), again after a
call that works."""
try:
result = fn(*args)
except Exception as e:
text = _text(e)
if self._failed.get(name) != text:
self._failed[name] = text
self._report("jetlink: %s() failed: %s", name, text)
return default
self._failed.pop(name, None)
return result
def _run(self, name: str, default, *args):
def call():
jetlink = self._jetlink()
return default if jetlink is None else getattr(jetlink, name)(*args)
return self._net(name, default, call)
def swap_events(self, tick_s: float) -> SwapTracker:
"""Bookkeeping alone, so it binds nothing: selfdrived builds it once,
link on or off."""
return SwapTracker(self._net('swap_events', None, SwapEvents, tick_s), self, tick_s)
class SwapTracker:
"""events.SwapEvents behind the net: update() is no event at all when it
fails, and settling False; the failure is logged as the hooks' are. Inert
without a tracker."""
def __init__(self, events: SwapEvents | None, hooks: Hooks | None, tick_s: float):
self._events = events
self._hooks = hooks
self._ok = events is not None
# runtime() a few times a second: modeld rewrites its record once a second
self._every = max(round(RUNTIME_POLL_S / tick_s), 1) if events is not None else 1
self._ticks = 0
self._runtime = None
@property
def settling(self) -> bool:
try:
return self._ok and bool(self._events.settling)
except Exception:
return False
def update(self, fresh: bool, big: bool, in_control: bool, state: str | None = None) -> Flags:
"""SwapEvents.update. `state` is the fork's own message's, where it has
one (zoompilot: modelDataV2SP.acceleratorState); without it the tracker
reads modeld's runtime record (hooks.runtime()), and a tick without a
live one is not fresh."""
if self._events is None:
return NO_EVENTS
if state is None:
if self._ticks == 0:
self._runtime = self._hooks.runtime()
self._ticks = (self._ticks + 1) % self._every
state = getattr(self._runtime, 'state', None)
fresh = fresh and state is not None
flags = self._hooks._net('swap_events.update', None, self._events.update, fresh, big, in_control, state or 'none')
self._ok = flags is not None
return NO_EVENTS if flags is None else flags
jetlink_repo/jetlink/openpilot/interface.py
+83 -20
View File
@@ -17,7 +17,7 @@ The interface is split by process. The resident gadget owner gets data only
(OwnerConfig), no callbacks, so it stays at the standard library and about
10 MB. The heavy processes get the adapter object and each uses its side:
StatusSide in manager, the UI, hardwared and the model manager, WorkerSide in
the provisioning run, ModelSide in modeld, BuildSide in the warp build.
the provisioning run, ModelSide in modeld.
The standard library only: the owner imports this module.
"""
@@ -82,7 +82,8 @@ class ModelFace:
# The sides are typing Protocols so a type checker can hold the adapter to
# them; conformance() is the check both repos run. A parameter jetlink passes
# by keyword is keyword-only here, and only those names must match.
# by keyword is keyword-only here, and only those names must match. The
# members in OPTIONAL are ones jetlink reads with getattr and a fallback.
class StatusSide(Protocol):
"""Every reader: manager, the UI, hardwared, the model manager."""
@@ -103,10 +104,11 @@ class StatusSide(Protocol):
"""Is comma's chestnut fitted? A USB walk; jetlink caches the answer."""
def camera(self) -> tuple[int, int, int, int]:
"""This device's (cam_w, cam_h, model_w, model_h): the warp modeld asks for."""
"""This device's (cam_w, cam_h, model_w, model_h): the warp modeld loads."""
def warp_path(self, cam_w: int, cam_h: int, model_w: int, model_h: int) -> Path:
"""Where the build puts the warp for this geometry."""
"""Where modeld's warp for this geometry is: openpilot's driving warp
pickle, which warp.Warps.load reads."""
class WorkerSide(StatusSide, Protocol):
@@ -127,39 +129,98 @@ class WorkerSide(StatusSide, Protocol):
class ModelSide(WorkerSide, Protocol):
"""modeld: comma's model face, the engagement a swap waits out, and the
structured log line the link's telemetry goes to."""
"""modeld: comma's model face, whether anything is in control, and the
structured log line the link's telemetry goes to. What modeld writes onto
the model every frame is Jetlink.attach's."""
def model_face(self) -> ModelFace:
"""What a ModelState for comma's large model has to carry."""
def engagement(self) -> Callable[[int], bool]:
"""A poller, made on the thread that calls it: (timeout_ms) -> engaged. Waits
up to timeout_ms for news and answers True when controls are engaged, or when
that is not known."""
def in_control(self) -> bool:
"""Is openpilot or MADS in control? The joining model asks on modeld's
frame thread before every frame and swaps the large model in only while
not. True when it cannot tell; never raises."""
def event(self, name: str, **fields: Any) -> None:
"""A structured log line (cloudlog.event)."""
class BuildSide(Protocol):
"""scons: comma's warp graph."""
def make_warp(self, cam_w: int, cam_h: int, model_w: int, model_h: int) -> tuple[Callable[..., Any], int]:
"""The warp graph for this geometry, and the size of the NV12 frame it
reads. Imports comma's graph module before anything of tinygrad's."""
def small_reset(self, small: Any) -> Callable[[], None] | None:
"""Optional (OPTIONAL): the reset of the fork's small model for a hand-back, or None
for jetlink's own (warp.prepare_reset, which knows stock modeld's,
modeld_v2's and a ModelState with GPU queues and numpy inputs). Asked once
on modeld's main thread before the frame loop. The reset runs on the
frame thread at a hand-back, so it zeroes the history in place and
allocates and compiles nothing; warp.reset_queues makes one."""
class Openpilot(ModelSide, BuildSide, Protocol):
class Openpilot(ModelSide, Protocol):
"""The whole adapter: one object implements every side."""
# What modeld hands Jetlink.attach and gets back: not the adapter's, pinned so
# a fork knows what its ModelState needs and what it may read off the result.
class SmallModel(Protocol):
"""The fork's own ModelState, whatever its shape. jetlink calls run()
positionally and writes attributes onto it (lat_delay, PLANPLUS_CONTROL);
the rest is optional, read with getattr: the frame's desire key and numpy
inputs (modeld_v2), the lateral delay a joining large model starts on, and
sunnypilot's planplus gain."""
desire_key: str
numpy_inputs: Any
lat_delay: float
PLANPLUS_CONTROL: float
def run(self, bufs: Any, transforms: Any, inputs: Any, after_enqueue: Callable[[], None] | None = None) -> Any:
"""One frame's outputs, as modeld's ModelState.run."""
class JoinedModel(Protocol):
"""What attach returns and modeld reads off and writes onto every frame.
frame_drop_ratio is written, lat_delay read and written. lat_delay and the
face (get_action_from_model through LONG_SMOOTH_SECONDS, modeld_v2's reads)
follow the model that drives, like any name it does not define, so over a
stock modeld's small model they are there only while the large one drives."""
handovers: int # moves on every swap and hand-back; modeld forgives that frame's drops
big_model_state: str # one of STATES
chestnut: bool # modelV2.big
get_action_from_model: Callable[..., Any]
constants: Any
LAT_SMOOTH_SECONDS: float
LONG_SMOOTH_SECONDS: float
frame_drop_ratio: float # modeld's share of dropped camera frames, before every run()
lat_delay: float
def run(self, bufs: Any, transforms: Any, inputs: Any, after_enqueue: Callable[[], None] | None = None) -> Any:
"""One frame's outputs, from whichever model drives."""
def close(self) -> None:
"""Let go of the link at modeld's exit."""
# The members of each protocol an implementation may lack: jetlink reads them
# with getattr and a fallback. conformance() passes an object without one and
# holds one with it to its signature, and adding one keeps API. Kept out of
# the protocols, where a name would be a member itself; a type checker still
# asks for these, as typing has no optional member
OPTIONAL: dict[type, tuple[str, ...]] = {
ModelSide: ('small_reset',),
SmallModel: ('desire_key', 'numpy_inputs', 'lat_delay', 'PLANPLUS_CONTROL'),
JoinedModel: ('get_action_from_model', 'constants', 'LAT_SMOOTH_SECONDS', 'LONG_SMOOTH_SECONDS', 'lat_delay'),
}
def optional(protocol: type) -> frozenset[str]:
"""The members of `protocol` an implementation may lack, inherited ones included."""
return frozenset(name for cls in protocol.__mro__ for name in OPTIONAL.get(cls, ()))
def load_adapter(module: str) -> Openpilot:
"""The adapter an adapter module makes. An adapter module is the fork's
module that holds every openpilot import jetlink needs: its top level
imports only the standard library, and it has adapter() -> Openpilot and
owner_config() -> OwnerConfig. The entry points that run as their own
process (the provisioning run, the warp build) are told its name."""
process (the provisioning run) are told its name."""
return importlib.import_module(module).adapter()
@@ -201,9 +262,11 @@ def conformance(obj: object, protocol: type) -> list[str]:
jetlink pins the interface with it, and the fork's tests run it against the
real adapter."""
problems = []
may_lack = optional(protocol)
for name, expected in members(protocol).items():
if not hasattr(obj, name):
problems.append(f"{name}: missing")
if name not in may_lack:
problems.append(f"{name}: missing")
continue
if expected is None:
continue
jetlink_repo/jetlink/openpilot/joining.py
+301 -282
View File
@@ -15,36 +15,41 @@ So modeld is handed the small model it already loaded, and the Jetson is
swapped in underneath once the link, the engine and the warp are all there.
modeld re-reads `model` every frame, and `modelV2.big` keeps its meaning.
From the join on, every frame goes to the Jetson whether or not it drives:
the small model drives on, and the large model's history and hidden state are
current and warm when the swap comes, so its first driving frame costs what
every frame costs. Those shadow frames are also the proof: the large model
is offered only once the host has answered the last PROVING_FRAMES of them
in time (model_state.keeping_up), so a host cold from a reconnect, or one
that is simply too slow, proves itself with the small model driving and not
at the wheel. The swap itself changes which model's output is published and
nothing else. While it drives, a reply that is not back in time is not
waited for: the previous frame's output is published again (a held frame,
Until the swap the comma runs its own model and nothing else; the link idles
on a keepalive ping. The large model is built on the swap frame and starts
from a reset history, as at every modeld start. The second after the swap is
its proof: the fork holds engagement off for it, and a large model that holds
a frame in it (model_state.PROVING_FRAMES) hands back before anyone can
engage. While it drives, a reply that is not back in time is not waited for:
the previous frame's output is published again (a held frame,
model_state.HOLD_FRAME) and the camera frame is not dropped. Too many holds
hand the drive back to the small model, as a lost link does.
No frames go to the host while the small model drives. Sending them costs
the comma ~9 ms of every frame for as long as the window stays shut (a whole
drive with MADS on), which starves the driver monitoring model under
selfdrived's frequency floor (commIssue).
Two rules the swap keeps:
- tinygrad work happens on modeld's thread. The joining thread does link IO
only; building the JetlinkModelState unpickles a TinyJit, and doing that
next to the small model running frames on the same device is not safe.
- never swap while the plan is steering. The two models disagree by ~195 m of
planned path, and the swap costs a frame or two. Require fresh, fully
disengaged controls; standstill alone is not enough, longitudinal control
may still hold the brake.
planned path, and the swap costs a frame or two. The adapter says before
every frame whether anything is in control (in_control), from openpilot's
own messages; standstill alone is not enough, longitudinal control may still
hold the brake.
"""
from __future__ import annotations
import os
import threading
import time
from jetlink import protocol as P
from jetlink.comma import gadget
from jetlink.openpilot.model_state import JetlinkModelState
from jetlink.transport.priority import background_thread
# after a join fails or the large model dies mid-drive. Long enough not to
@@ -56,39 +61,25 @@ REJOIN_DELAY = 5.0
REJOIN_DELAY_MAX = 60.0
STABLE_SECONDS = 60.0
# after a join that held: a link that ran for minutes and then went is a USB
# drop, and the host re-enumerates a rebound gadget in under a second. The
# rest of the old delay was the driver's time, not the Jetson's
# drop, and the host re-enumerates a rebound gadget in under a second
REJOIN_DELAY_QUICK = 1.0
# failures in a streak retried that quickly before the doubling starts. A
# demote is mostly one late frame, a tail of the link or the host and not a
# dead one: on a phone's cable each one cost 5 to 60 s on the small model,
# five times in nine minutes on an iPad (2026-10-03). A link that fails on
# its heels this many times is backed off as before
# dead one, and a full backoff costs 5 to 60 s on the small model. A link
# that fails on its heels this many times is backed off as usual
QUICK_RETRIES = 3
# after a link lost while it only shadowed: no swap, demote or alert to pay
# for, so no doubling. A phone that dropped its first frames again and again
# pulsed the icon for 1 to 60 s between tries (2026-10-04)
SHADOW_REJOIN_DELAY = 2.0
# how often a link that is ready but has nowhere to land gets checked, and
# how long the check may take; both off the frame loop
KEEPALIVE_PERIOD = 10.0
PING_TIMEOUT = 2.0
# how often a backoff looks at the gadget. A host that configures it again
# after it went away is a replug, which the backoff is not for: one waited 16 s
# for a Jetson that was back in 0.4 (2026-09-29)
# after it went away is a replug, which the backoff is not for: a replugged
# Jetson is back in under half a second
REPLUG_POLL = 0.25
# link drops in one drive before the status names the cable. One is weather;
# the 2026-09-07 evening drive had six in twelve minutes, every one the
# USB-C port losing its host, and nothing the driver could see said so
# link drops in one drive before the status names the cable. One is chance;
# repeated drops are almost always the USB-C port losing its host, and nothing
# else the driver can see says so
DROPS_TO_BLAME_CABLE = 2
ENGAGEMENT_POLL_MS = 100
ENGAGEMENT_MAX_AGE = 0.25
# a large-model frame is ~30 ms and the worst seen on the current stack ~55.
# One that took LATE_FRAME, or a second past SLOW_FRAME within LAG_WINDOW of
# the last, is a fault and is handled as a loss: modeld would otherwise count
# the dropped camera frames into modeldLagging with the slow model still
# steering. With a late reply held rather than waited for (model_state), a
# frame this long is the comma's own stall, a warp or a send, or a frame with
# nothing to hold yet. A frame past link.INFERENCE_TIMEOUT never returns; it fails
LATE_FRAME = 0.1
SLOW_FRAME = 0.075
LAG_WINDOW = 10.0
# A host slower than the camera but never that slow (an iPhone that has warmed
# up: 55 ms a frame drops one camera frame in ten) still has modeld skipping
# frames, and selfdrived soft-disables on modeldLagging once modeld's filtered
@@ -98,54 +89,58 @@ LAG_WINDOW = 10.0
# a second within about 6.5 s is not. modeld forgives the frame of a handover
# too, so the drops that decided it never reach selfdrived
DROP_LIMIT = 0.0075
# the first frames after every swap are never counted as slow: the first after
# a join carries the history reset, and a Mac's is ~100 ms of CoreML warm-up
SETTLING_FRAMES = 3
# the small model drives the first frames of every modeld start, even with the
# large model ready: its first run in a process costs ~1.3 s, which the first
# fallback frame paid (25 dropped frames, commIssue) when the large model had
# driven from frame one. Nothing is in control at a modeld start
# fallback frame would pay (25 dropped frames, commIssue) if the large model
# drove from frame one. Nothing is in control at a modeld start
SMALL_WARMUP_FRAMES = 3
# shadowing this long without keeping up, the panel says the link is too slow
# to drive rather than 'checking' for the drive. The proof goes on: a host
# that warms up late still gets offered
SLOW_LINK_AFTER = 10.0
# the panel's line while the large model shadows: once it is built, once it
# keeps up, and once it has not for SLOW_LINK_AFTER
CHECKING = 'checking the link keeps up'
PROVEN = 're-engage to switch'
SLOW = 'too slow to drive'
# the panel's line while a ready link waits for the window
READY = 're-engage to switch'
# why a demote happened, as the log says it
LOST = 'lost jetlink'
BEHIND = 'jetlink fell behind'
# a NaN or an inf in an output: a host that answers, wrongly. A lag and not a
# drop, the link is kept, and the next swap starts the host from a reset
BAD_OUTPUT = 'jetlink output not finite'
# what modeld_v2 builds the frame's inputs from: the small bundle's desire
# name and optional slots, read off the small model whichever drives
SMALL_READS = ('desire_key', 'numpy_inputs')
# why a built model was let go before it drove
NOT_UP = 'could not bring up the large model'
# and as the leave says it to the server (protocol.Msg.LEAVE)
LEAVING = {LOST: P.LEAVE_LOST, BEHIND: P.LEAVE_BEHIND}
# and as the panel's one line says it while the join tries again
RECONNECTING = {LOST: 'reconnecting', BEHIND: 'fell behind, reconnecting', NOT_UP: 'load failed, reconnecting'}
# the demotes that count as lag, which says nothing about the cable
LAGS = (BEHIND, BAD_OUTPUT)
# and as the panel's one line says it while the join tries again, with the
# `why` of its progress code (status.PROGRESS_CODES 'reconnecting')
RECONNECTING = {LOST: ('reconnecting', 'lost'), BEHIND: ('fell behind, reconnecting', 'behind'),
NOT_UP: ('load failed, reconnecting', 'load_failed'),
BAD_OUTPUT: ('bad output, reconnecting', 'bad_output')}
# how long a model name the catalog had not resolved stays the sha's
NAME_RETRY = 30.0
class JoiningModelState:
"""Duck-types openpilot's modeld ModelState, with a second one inside.
`progress` is where the join says what it is waiting on (status.Progress),
`engagement` makes the poller the swap window reads (the adapter's), and
`log` is cloudlog on a comma.
`log` is cloudlog on a comma, and `in_control` the adapter's, asked before
every frame; without one, the in_control last written stands. `name` is
the catalog's name for a model's sha256, for the runtime record.
"""
def __init__(self, small, connect, build, prepare=None, reset_small=None, *, progress, engagement, log):
def __init__(self, small, connect, build, prepare=None, reset_small=None, *, progress, log, in_control=None,
name=None):
self._small = small
self._active = small
self._connect = connect
self._build = build
self._reset_small = reset_small
self._progress = progress
self._engagement = engagement
self._log = log
# whatever the swap would otherwise do on the frame loop, done now on
# modeld's main thread before the frame loop exists; at the swap it cost
# modeld's main thread before the frame loop exists; at the swap it costs
# a 1.5 s frame
if prepare is not None:
try:
@@ -156,14 +151,17 @@ class JoiningModelState:
log.exception("jetlink: could not prepare the large model ahead of the swap")
raise
# handed over by the joining thread, consumed by the next frame, which
# handed over by the joining thread, consumed by the swap frame, which
# builds the large model state from it. Only ever assigned under the lock
self._joined: tuple[object, object] | None = None
# the large model state once built: fed every frame, driving when it is
# _active. Only the frame thread assigns it
# a link is ready and waiting for the window. Kept true while a keepalive
# ping has _joined out for a moment
self._available = False
# the large model state, from the swap until it hands back: always _active.
# Only the frame thread assigns it
self._big = None
# what the frame thread let go of and why (LOST, BEHIND, or a build that
# failed), for the join thread to say goodbye to and close or keep.
# what the frame thread let go of and why (LOST, BEHIND, BAD_OUTPUT, or a
# build that failed), for the join thread to say goodbye to and close or keep.
# Assigned under the lock
self._retired: tuple[object, str] | None = None
self._lock = threading.Lock()
@@ -182,29 +180,24 @@ class JoiningModelState:
# changes of the model that drives, and decisions to change it: modeld
# compares it across run() (handovers)
self._handovers = 0
# set on a frame the large model fell behind on; the next frame demotes.
# _slow_at is when the last slow frame was, for the second strike
# set on a frame the large model fell behind on; the next frame demotes
self._lagging = False
self._slow_at: float | None = None
# modeld's share of dropped camera frames, for the frame about to run
self._frame_drop_ratio = 0.0
# frames each model has run: the small one's since start, the large one's
# since it swapped in
# set by the large model's first frame: the join thread clears the
# progress, which is params IO and not the frame thread's
self._announce = False
# frames the small model has run since start
self._small_frames = 0
self._big_frames = 0
# the panel's last line about the large model shadowing (CHECKING, PROVEN
# or SLOW). A report is a param write on the frame thread, so only a
# change is reported
self._proving = CHECKING
# whether the host had let go of the gadget when the last link was lost,
# and whether this failure streak has already skipped a backoff for a replug
self._host_left = False
self._replugged = False
# assume engaged and moving until a message says otherwise, so a swap can
# never happen on no information
self._engaged = True
self._engagement_updated = 0.0
# in control until the adapter says otherwise, so a swap can never happen
# on no information
self._in_control = True
self._ask_in_control = in_control
self._stop = threading.Event()
# whether the large model has produced a frame; a first inference that
@@ -212,10 +205,20 @@ class JoiningModelState:
# modelDataV2SP, not a param: a chestnut's load is over once, this never is
self._loading = True
self._threads = [threading.Thread(target=self._join_loop, daemon=True),
threading.Thread(target=self._watch_engagement, daemon=True)]
for t in self._threads:
t.start()
# the runtime record's: when the large model's last answer (not a held
# frame) came, and a wake for its writer on a hand-back and the first
# answer. The frame thread assigns them and nothing else; the writer
# thread reads the rest itself. The path is taken now, so a test's stays
# its own
self._output_at = 0.0
self._record = threading.Event()
self._record_path = gadget.MODELD
self._name = name
self._thread = threading.Thread(target=self._join_loop, daemon=True)
self._thread.start()
self._writer = threading.Thread(target=self._write_loop, daemon=True)
self._writer.start()
# -- what modeld reads ------------------------------------------------------
# A read not defined here follows the model that is driving (__getattr__):
@@ -223,21 +226,16 @@ class JoiningModelState:
# function, which sunnypilot keeps on the ModelState so a custom small bundle
# can carry its own. Swaps and demotes happen inside run(), so what is read
# after it belongs to the model whose output it is. What the loop writes
# lands on both. The frame's inputs are built for the small bundle (its
# desire name, its optional slots); the large model reads the desire under
# any name and the loop always supplies what it needs, so a swap can land on
# any frame
# lands on both, but for what only this reads. The frame's inputs are built
# for the small bundle (SMALL_READS, off the small model whichever drives);
# the large model reads the desire under any name and the loop always
# supplies what it needs, so a swap can land on any frame
@property
def big_model_available(self) -> bool:
"""Built, keeping up with the frames it shadows, and waiting to switch:
what big_model_state calls ready, and what the driver is offered."""
return not self._stop.is_set() and self._shadowing and self._big.keeping_up
@property
def _shadowing(self) -> bool:
# a large model built and fed every frame, with the small model driving
return self._big is not None and self._active is self._small
"""Connected and waiting to switch: what big_model_state calls ready, and
what the driver is offered."""
return not self._stop.is_set() and self._available and self._active is self._small
@property
def chestnut(self) -> bool:
@@ -271,14 +269,6 @@ class JoiningModelState:
# read by the status publisher on every send, so the telemetry follows the link
return getattr(self._active, 'client', None)
@property
def desire_key(self) -> str:
return self._small.desire_key
@property
def numpy_inputs(self):
return self._small.numpy_inputs
@property
def lat_delay(self):
return self._active.lat_delay
@@ -308,25 +298,36 @@ class JoiningModelState:
@frame_drop_ratio.setter
def frame_drop_ratio(self, value):
# modeld's frameDropPerc / 100, written before every run(). Read here, by
# the lag rule; it lands on the small model too, as every write does
# the lag rule; no small model reads it
self._frame_drop_ratio = value
self._small.frame_drop_ratio = value
@property
def in_control(self) -> bool:
return self._in_control
@in_control.setter
def in_control(self, value):
# openpilot or MADS in control, by openpilot's own messages, asked before
# every run(): the swap waits for False. Not a thread polling selfdrived:
# that wakes ~300 times a second and takes 6 to 11 % of a core inside modeld
self._in_control = value
def __getattr__(self, name):
# Only for names this class does not define. Without it, a comma or
# sunnypilot sync that adds one read was an AttributeError on the frame
# thread, which modeld re-raises: modeld dead for the drive, on jetlink
# devices only. Reads only; a new write lands here and nowhere else, so
# each one modeld makes has its own setter above
# Only for names this class does not define, so a read that a comma or
# sunnypilot sync adds is not an AttributeError on the frame thread (modeld
# re-raises it and is dead for the drive). Reads only; a write lands on
# this object alone, so each one modeld makes has its own setter above
if name.startswith('_'):
# also what keeps __init__ from recursing before _active exists
raise AttributeError(name)
return getattr(self._active, name)
# a small model without one of SMALL_READS has none, whichever drives
return getattr(self._small if name in SMALL_READS else self._active, name)
# -- the frame path ---------------------------------------------------------
def run(self, bufs, transforms, inputs, after_enqueue=None):
self._adopt()
if self._ask_in_control is not None:
self.in_control = self._ask_in_control()
if self._lagging:
# the last frame was the large model's last, published as it came
self._lagging = False
@@ -336,132 +337,71 @@ class JoiningModelState:
self._log.warning("jetlink: modeld dropped %.2f %% of camera frames behind the large model, "
"the small model drives from this one", self._frame_drop_ratio * 100)
self._demote(BEHIND)
self._note_proving()
self._maybe_swap()
if self._active is self._small:
self._small_frames += 1
return self._run_small(bufs, transforms, inputs, after_enqueue)
return self._small.run(bufs, transforms, inputs, after_enqueue)
return self._run_big(bufs, transforms, inputs, after_enqueue)
def _run_small(self, bufs, transforms, inputs, after_enqueue):
"""The small model's frame, and the large model's too when there is one:
warped now, sent once the small model's own work is on the GPU (its
after_enqueue), so the send overlaps it. A link that fails here is lost
before it drove: let go and reopened, with the small model's frame
unharmed."""
big = self._big
if big is None:
return self._small.run(bufs, transforms, inputs, after_enqueue)
try:
big.prepare(bufs, transforms, inputs)
except Exception:
self._log.exception("jetlink: the large model's link failed with the small model driving, reopening")
self._retire(LOST)
return self._small.run(bufs, transforms, inputs, after_enqueue)
failed: list[BaseException] = []
def enqueued():
try:
big.send()
except Exception as e: # never through the small model's run: modeld would make it fatal
failed.append(e)
if after_enqueue is not None:
after_enqueue()
result = self._small.run(bufs, transforms, inputs, enqueued)
if not failed and not big.sent:
# a small model that did not call back; send now
try:
big.send()
except Exception as e:
failed.append(e)
if not failed:
# the proof: would this frame have been held? (model_state.check_shadow)
try:
big.check_shadow()
except Exception as e:
failed.append(e)
if failed:
self._log.error("jetlink: the large model's link failed with the small model driving, reopening",
exc_info=failed[0])
self._retire(LOST)
return result
def _run_big(self, bufs, transforms, inputs, after_enqueue):
big = self._big
started = time.monotonic()
try:
result = big.run(bufs, transforms, inputs, after_enqueue)
except Exception:
except Exception as e:
failed = time.monotonic()
self._log.exception("jetlink: large model failed mid-drive, back to the small model")
self._demote(LOST)
self._demote(BAD_OUTPUT if isinstance(e, P.NotFinite) else LOST)
demoted = time.monotonic()
# re-run the frame rather than propagate: modeld's fallback is permanent,
# this one is retryable. after_enqueue is dropped, the large model may
# already have called it
result = self._small.run(bufs, transforms, inputs, None)
# all of it is one modeld frame, 42 to 107 ms on the 2026-09-29 drives
# with the small model warm. The dropped camera frames are forgiven
# all of it is one modeld frame, 42 to 107 ms measured with the small
# model warm. The dropped camera frames are forgiven
# (modeld sees the model change), but the log should say which part it was
done = time.monotonic()
self._log.warning("jetlink: fallback frame %.0f ms: link %.0f, demote %.0f, small model %.0f",
(done - started) * 1e3, (failed - started) * 1e3,
(demoted - failed) * 1e3, (done - demoted) * 1e3)
return result
took = time.monotonic() - started
self._big_frames += 1
if not getattr(big, 'held', False):
self._output_at = time.monotonic()
if self._loading:
# a connected engine can still fail its first inference; only announce
# readiness after a frame the caller can publish
self._loading = False
self._progress.clear()
self._announce = True
self._rejoin.set()
self._record.set()
self._log.warning("jetlink: large model joined mid-drive, modelV2.big is now true")
if self._big_frames > SETTLING_FRAMES:
why = getattr(big, 'behind', None) or (f'took {took * 1e3:.0f} ms' if self._fell_behind(took) else None)
if why:
# this frame's output is published as it came; its stall is forgiven now
self._lagging = True
self._handovers += 1
self._log.warning("jetlink: large model %s, the small model drives from the next", why)
if big.behind:
# this frame's output is published as it came; its stall is forgiven now
self._lagging = True
self._handed_over()
self._log.warning("jetlink: large model %s, the small model drives from the next", big.behind)
return result
def _fell_behind(self, took: float) -> bool:
if took > LATE_FRAME:
return True
if took <= SLOW_FRAME:
return False
now = time.monotonic()
second = self._slow_at is not None and now - self._slow_at <= LAG_WINDOW
self._slow_at = now
return second
@property
def _window_open(self) -> bool:
# standstill does not make an active longitudinal controller safe to swap
fresh = 0 <= time.monotonic() - self._engagement_updated < ENGAGEMENT_MAX_AGE
return fresh and not self._engaged
def _adopt(self) -> None:
"""Build the large model state from a join that has landed, on this
thread, which is where everything tinygrad touches must happen. From the
next frame on it is fed every frame (_run_small) and swapped in when the
window opens. A build that fails is backed off like a demote, or one that
fails the same way every time is a connect and a build per second for
the drive."""
if self._joined is None:
def _maybe_swap(self) -> None:
"""Build the large model state from a join that has landed and swap it in,
on this thread, which is where everything tinygrad touches must happen.
Its first frame resets the host's history. A build that fails is backed
off like a demote, or one that fails the same way every time is a connect
and a build per second for the drive."""
if self._joined is None or self._in_control or self._small_frames < SMALL_WARMUP_FRAMES:
return
with self._lock:
joined, self._joined = self._joined, None
if joined is not None:
self._available = False
if joined is None:
return
client, spec = joined
try:
t0 = time.monotonic()
big = self._build(client, spec)
big.lat_delay = self._small.lat_delay
self._log.warning("jetlink: built the large model state in %.0f ms; it runs every frame from here",
(time.monotonic() - t0) * 1000)
big.lat_delay = getattr(self._small, 'lat_delay', big.lat_delay)
self._log.warning("jetlink: built the large model state in %.0f ms", (time.monotonic() - t0) * 1000)
except Exception:
self._log.exception("jetlink: could not bring up the large model, staying small")
with self._lock:
@@ -469,63 +409,33 @@ class JoiningModelState:
self._back_off()
return
self._big = big
self._proving = CHECKING # the join thread said so at link ready
# the link is held from here, shadowing or driving: a shadow that ran for
# minutes and then went starts a new streak, as a drive that did
self._joined_at = time.monotonic()
def _note_proving(self) -> None:
"""The panel's line while the large model shadows, on a change only:
PROVEN once it keeps up, SLOW once it has not for SLOW_LINK_AFTER, and
CHECKING again after a hand-back. The first driven frame clears it."""
if not self._shadowing:
return
if self._big.keeping_up:
line = PROVEN
elif time.monotonic() - self._joined_at > SLOW_LINK_AFTER:
line = SLOW
else:
line = CHECKING
if line != self._proving:
self._proving = line
self._log.warning("jetlink: large model %s", line)
self._report('connect', line)
def _maybe_swap(self) -> None:
if not self._shadowing:
return
if not self._big.keeping_up or not self._window_open or self._small_frames < SMALL_WARMUP_FRAMES:
return
# nothing to build and nothing to reset (the module docstring says why)
self._big_frames = 0
self._slow_at = None
# no wake: the record says the swap with the first answer, which wakes it
self._handovers += 1
self._active = self._big
self._active = big
def _demote(self, why: str) -> None:
"""Back to the small model, from a reset history. On the frame thread, so
nothing here waits: the join thread reads the port, reports and closes
the link, moments later."""
self._retire(why) # while _shadowing still tells a drive from a shadow
self._active = self._small
self._handovers += 1
self._loading = True
if self._reset_small is not None:
self._reset_small()
def _retire(self, why: str) -> None:
"""Let the large model go, driving or not, for the join thread to say
goodbye to and close or keep (_close_retired). Counted as a lost link or
a hand-back for lag, and backed off."""
shadow = self._shadowing
the link, moments later. Counted as a lost link or a hand-back for lag,
and backed off."""
big, self._big = self._big, None
if why == BEHIND:
if why in LAGS:
self._lags += 1
else:
self._drops += 1
with self._lock:
self._retired = (big, why)
self._back_off(shadow)
self._back_off()
self._active = self._small
self._handed_over()
self._loading = True
if self._reset_small is not None:
self._reset_small()
def _handed_over(self) -> None:
self._handovers += 1
self._record.set()
def _close_retired(self, retired, why: str) -> None:
"""Say goodbye over the retired large model's link, then close it if it
@@ -557,14 +467,13 @@ class JoiningModelState:
except Exception:
self._log.exception("jetlink: could not say why the link is left")
def _back_off(self, shadow: bool = False) -> None:
def _back_off(self) -> None:
"""Push the next attempt out, further each time one fails on its heels.
Each failed cycle is a swap frame, a demote frame and the alerts that go
with them. The first QUICK_RETRIES failures of a streak are retried in
REJOIN_DELAY_QUICK; the ones after double from REJOIN_DELAY. A join that
held for STABLE_SECONDS starts a new streak. A link lost while it only
shadowed waits SHADOW_REJOIN_DELAY.
held for STABLE_SECONDS starts a new streak.
"""
held = time.monotonic() - self._joined_at if self._joined_at else 0.0
stable = bool(self._joined_at) and held > STABLE_SECONDS
@@ -572,9 +481,7 @@ class JoiningModelState:
if stable:
self._replugged = False # a new streak may skip a backoff for a replug again
self._joined_at = 0.0
if shadow:
delay = SHADOW_REJOIN_DELAY
elif self._failures <= QUICK_RETRIES:
if self._failures <= QUICK_RETRIES:
delay = REJOIN_DELAY_QUICK
else:
delay = min(REJOIN_DELAY * 2 ** (self._failures - QUICK_RETRIES - 1), REJOIN_DELAY_MAX)
@@ -585,17 +492,18 @@ class JoiningModelState:
# -- background -------------------------------------------------------------
def _report(self, stage: str, msg: str) -> None:
def _report(self, stage: str, msg: str, code: str | None, args: dict | None = None) -> None:
"""Tell the UI what the join is waiting on.
Offroad the record carries a provisioning run's progress; without this, a
Jetson that is not plugged in looked like one six seconds from loading. No
fraction to give, and the panel does not invent one.
Jetson that is not plugged in looks like one about to load. No fraction to
give, and the panel does not invent one.
"""
# the count is the diagnosis. A link that runs for minutes and then goes,
# again and again, is the cable, and the cable is the one thing the
# driver can do something about
self._progress.report(stage, 0.0, msg, drops=self._drops if self._drops >= DROPS_TO_BLAME_CABLE else 0)
self._progress.report(stage, 0.0, msg, drops=self._drops if self._drops >= DROPS_TO_BLAME_CABLE else 0,
code=code, args=args)
def _note_link_loss(self, why: str) -> None:
"""Off the frame thread: what the comma's USB-C port sees now, next to the
@@ -610,7 +518,8 @@ class JoiningModelState:
# has still been replugged
self._host_left = not gadget.host_attached()
self._log.warning("jetlink: %s, %s; drop %d, lag %d this drive", why, port, self._drops, self._lags)
self._report('connect', RECONNECTING[why])
msg, code = RECONNECTING[why]
self._report('connect', msg, 'reconnecting', {'why': code})
def _join_loop(self) -> None:
"""Open the link and get the engine ready. No tinygrad in here."""
@@ -625,6 +534,16 @@ class JoiningModelState:
# a dead link, and close() sets this. An idle wake per second is not
# free on modeld's core
self._rejoin.wait()
if self._announce:
# the large model's first frame is out. Before the retired one below,
# whose reconnecting line is newer; woken for this alone, wait again.
# A demote or close sets its own flag before the event, so clearing
# the event and then looking misses neither
self._announce = False
self._progress.clear()
self._rejoin.clear()
if self._retired is None and not self._stop.is_set():
continue
with self._lock:
retired, self._retired = self._retired, None
if retired is not None:
@@ -641,16 +560,20 @@ class JoiningModelState:
self._rejoin.clear()
if self._wait_out_back_off():
return
self._report('connect', 'waiting for jetlink')
self._report('connect', 'waiting for jetlink', 'waiting_for_host')
try:
# the connect can take minutes when the picked model still has to be
# built, so it is handed the flag close() sets rather than polled
client, spec = self._connect(self._stop.is_set)
except Exception as e:
# expected while the Jetson boots. Not exception(): a stack trace every
# 5 s for the first minute of every drive is noise
self._log.warning("jetlink: not joined yet (%s), retrying in %.0fs", e, REJOIN_DELAY)
if self._stop.wait(REJOIN_DELAY):
# expected while the Jetson boots, so no stack trace. A failure that
# knows when it is worth asking again says so (link.ModelMissing), and
# what the panel should show meanwhile
delay = getattr(e, 'retry_after', REJOIN_DELAY)
if waiting := getattr(e, 'waiting', None):
self._report('connect', waiting, getattr(e, 'code', None))
self._log.warning("jetlink: not joined yet (%s), retrying in %.0fs", e, delay)
if self._stop.wait(delay):
return
self._rejoin.set()
continue
@@ -658,9 +581,11 @@ class JoiningModelState:
if self._stop.is_set():
client.close()
return
self._available = True
self._joined = (client, spec)
self._log.warning("jetlink: link ready, the large model shadows from the next frame")
self._report('connect', CHECKING)
self._log.warning("jetlink: link ready, waiting for a window to swap")
self._report('connect', READY, 'ready_waiting_window')
self._keep_alive()
def _wait_out_back_off(self) -> bool:
"""Until the next attempt is due, or a host configures the gadget again
@@ -679,21 +604,117 @@ class JoiningModelState:
host_left = host_left or not attached
return self._stop.is_set()
def _watch_engagement(self) -> None:
background_thread() # off modeld's realtime core; see _join_loop
# made on this thread: the poller's sockets (a SubMaster) belong to the
# thread that made them
engaged = self._engagement()
def _keep_alive(self) -> None:
"""Ping a link that is waiting for a swap window.
On a drive with no stop and no disengage that is the whole drive, and a
host that reboots in there would otherwise be found at the swap: a build
on a dead link, a demote and the backoff, all on modeld's thread.
The client is taken out of _joined for the ping and put back after, so the
frame loop sees a whole one or none, and never waits on the lock.
"""
while not self._stop.is_set():
# rechecked on every poll, including ones with no news: that is when
# alive turns false. The frame thread expires the answer too
self._engaged = engaged(ENGAGEMENT_POLL_MS)
self._engagement_updated = time.monotonic()
if self._rejoin.wait(KEEPALIVE_PERIOD):
return
with self._lock:
joined, self._joined = self._joined, None
if joined is None:
return # swapped in on a frame, or closed under us
try:
joined[0].ping(timeout=PING_TIMEOUT)
except Exception as e:
self._available = False
self._log.warning("jetlink: the link died before it could be used (%s), reopening", e)
self._note_link_loss(LOST)
try:
joined[0].close()
except Exception:
self._log.exception('jetlink: closing the dead link')
self._back_off()
return
with self._lock:
stopped = self._stop.is_set()
if not stopped:
self._joined = joined
if stopped:
# close() ran during the ping and found nothing to close
self._say_leaving(joined[0], P.LEAVE_STOPPED)
joined[0].close()
return
def _write_loop(self) -> None:
"""The runtime record (gadget.MODELD), once a second, on a hand-back and on
the first answer after a swap: tmpfs IO, the link record and a catalog
read for the name, so never on the frame thread, and off modeld's realtime
core first (_join_loop). The last one, after close(), is stopping.
status.runtime reads pid, at, stopping, state, active and last_output_at;
handovers, held, sha256, name, checkpoint, transport, drops and lags have
no reader in jetlink yet, they are for forks (StarPilot's sha and
checkpoint, a drive's counts)."""
background_thread()
pid, names, kinds = os.getpid(), {}, {}
failing = False
while True:
# before the record is built: a wake from here on is for the next one
self._record.clear()
stopping = self._stop.is_set()
try:
gadget.write_record(self._record_path, self._runtime(pid, names, kinds, stopping))
failing = False
except Exception:
if not failing: # once, not every second
self._log.exception("jetlink: could not write the runtime record")
failing = True
if stopping:
return
self._record.wait(gadget.RECORD_PERIOD)
def _runtime(self, pid: int, names: dict, kinds: dict, stopping: bool) -> dict:
big = self._big
state = self.big_model_state
swapped_at = self._joined_at # after the state: a swap sets it before _active
if state == 'running' and self._output_at < swapped_at:
# swapped in and not answered yet: the swap is still landing, not a
# model that stopped answering
state = 'joining'
spec = getattr(big, 'spec', None)
sha = getattr(spec, 'sha256', None)
if big is not None and swapped_at not in kinds:
# once per swap: the gadget the link runs on does not change under it
kinds.clear()
kinds[swapped_at] = gadget.link_kind()
return {'pid': pid, 'at': time.monotonic(), 'stopping': stopping, 'state': state,
'handovers': self._handovers, 'active': big is not None and self._active is big,
'last_output_at': self._output_at, 'held': bool(getattr(big, 'held', False)),
'sha256': sha, 'name': self._name_of(sha, names) if sha else None,
'checkpoint': getattr(spec, 'checkpoint', None),
'transport': kinds.get(swapped_at) if big is not None else None, 'drops': self._drops,
'lags': self._lags}
def _name_of(self, sha: str, names: dict) -> str:
"""The catalog's name, kept once it is one. The sha's stand-in (a row not
resolved yet, or a lookup that failed) is asked again after NAME_RETRY."""
name, retry_at = names.get(sha, (None, 0.0))
if time.monotonic() >= retry_at:
try:
name = self._name(sha) if self._name is not None else None
except Exception:
name = None
real = bool(name) and name != sha[:16]
name = name if real else sha[:16]
names[sha] = (name, float('inf') if real else time.monotonic() + NAME_RETRY)
return name
def close(self) -> None:
first = not self._stop.is_set() # the leave is said once; cleanups close twice
self._stop.set()
self._available = False
self._rejoin.set()
# after the stop: the writer's next record is its last, and says stopping
self._record.set()
self._writer.join(1.0)
self._progress.clear()
with self._lock:
joined, self._joined = self._joined, None
@@ -718,19 +739,16 @@ def join(parts, cam_w: int, cam_h: int, small) -> JoiningModelState:
what the link will hand back; another geometry is rejected.
"""
from jetlink.openpilot import link as links
# here rather than in build(): the import then costs modeld's main thread
# before the frame loop, not the frame the swap lands on
from jetlink.openpilot.model_state import JetlinkModelState
op = parts.op
face = op.model_face()
ready: dict = {}
link = links.Link(parts.log)
def prepare():
from jetlink.openpilot.warp import prepare_reset, warm
from jetlink.openpilot.warp import Warp, prepare_reset
# the gadget first, so the Jetson enumerates while the warp loads. Left to
# the join thread the bind landed ~3 s later, behind the small model's
# first frame, and one ignition had a 655 ms frame during the bind
# the join thread the bind lands ~3 s later, behind the small model's
# first frame, and can stall a frame by hundreds of ms
links.present_early(link, background_thread)
cached = parts.spec.load()
if cached is not None:
@@ -739,9 +757,9 @@ def join(parts, cam_w: int, cam_h: int, small) -> JoiningModelState:
else:
geometry = parts.warps.geometry()[2:]
try:
ready['reset_small'] = prepare_reset(small)
warp = parts.warps.load(cam_w, cam_h, *geometry)
warm(warp, face.frame_size(cam_w, cam_h))
# the fork's own reset of its small model, else jetlink's from its shape
ready['reset_small'] = getattr(op, 'small_reset', lambda small: None)(small) or prepare_reset(small)
warp = Warp(parts.warps.load(cam_w, cam_h, *geometry), face.frame_size(cam_w, cam_h))
except Exception:
link.close()
raise
@@ -752,10 +770,11 @@ def join(parts, cam_w: int, cam_h: int, small) -> JoiningModelState:
warp = ready.get('warp') if ready.get('geometry') == (img_w * 2, img_h * 2) else None
if warp is None:
raise RuntimeError('no prepared warp for the server model geometry')
return JetlinkModelState(cam_w, cam_h, client, spec, warp, face=face, log=parts.log, event=op.event)
return JetlinkModelState(client, spec, warp, face=face, log=parts.log, event=op.event)
def connect(should_stop=None):
return links.open_link(parts, link, should_stop)
return JoiningModelState(small, connect, build, prepare, reset_small=lambda: ready['reset_small'](),
progress=parts.progress, engagement=op.engagement, log=parts.log)
progress=parts.progress, log=parts.log, in_control=op.in_control,
name=parts.models.name_for)
jetlink_repo/jetlink/openpilot/link.py
+81 -20
View File
@@ -31,9 +31,11 @@ CONNECT_TIMEOUT = 45.0
CONNECT_DELAY = 0.5
# four frame periods, and four times the slowest frame seen. A host that goes
# silent with no USB edge (a hung server, a phone's cable) costs one frame this
# long plus the small model's, inside modelV2's 0.5 s alive limit; 0.5 s here
# flashed commIssue on top of the fallback
# long plus the small model's, inside modelV2's 0.5 s alive limit; at 0.5 s
# here commIssue flashes on top of the fallback
INFERENCE_TIMEOUT = 0.2
# between joins while neither end has the picked model and nothing else is built
MODEL_WAIT = 60.0
# how long the load may wait for the early gadget bind; a provisioning run may
# still be letting go of the endpoints
PRESENT_TIMEOUT = 5.0
@@ -43,6 +45,16 @@ PRESENT_TIMEOUT = 5.0
BUILD_TIMEOUT = 1800.0
class ModelMissing(RuntimeError):
"""Neither end has the picked model and nothing else is built: only a
provisioning run's download changes that, so the join asks again after
`retry_after` rather than every few seconds, and the panel says `waiting`
(joining._join_loop)."""
retry_after = MODEL_WAIT
waiting = 'big model not downloaded yet'
code = 'model_missing' # waiting's progress code (status.PROGRESS_CODES)
def _left(deadline: float | None, default: float) -> float:
"""What is left of an absolute `deadline`, or `default` without one."""
return default if deadline is None else max(0.0, deadline - time.monotonic())
@@ -93,7 +105,7 @@ class Link:
def open(self, deadline: float | None = None):
"""The client, opening one if we have not got one yet, or if the one held
is dead: a big model retired after a link loss closes its client, and
reusing it failed the next attempt with EBADF, 5 s after every loss."""
reusing it fails the next attempt with EBADF."""
if self.client is not None and self.client.dead:
self.close()
if self.client is None:
@@ -218,7 +230,7 @@ def identity(parts, entry: dict) -> tuple[str, int]:
sha256, nbytes = entry.get('oid'), entry.get('size')
if sha256 and nbytes:
return sha256, int(nbytes)
parts.progress.report('connect', 0.0, 'finding model')
parts.progress.report('connect', 0.0, 'finding model', code='finding_model')
return parts.models.resolve_pointer(entry['ref'])
@@ -229,6 +241,16 @@ def identity(parts, entry: dict) -> tuple[str, int]:
_hashed: dict[tuple[str, int, int], str] = {}
def _hash_key(path: Path, st) -> tuple[str, int, int]:
return str(path), st.st_size, st.st_mtime_ns
def remember_hash(path: Path, sha256: str) -> None:
"""A file whose hash is already proven, as a download's is on the way in:
its upload need not read the gigabyte again (verified_upload)."""
_hashed[_hash_key(path, path.stat())] = sha256
def verified_upload(log, model_path: Path | None, sha256: str, nbytes: int) -> Path | None:
"""The file to upload, once its hash is proven to match the registry: under
a sha the bytes do not have, the Jetson's plan would lie about its contents."""
@@ -240,7 +262,7 @@ def verified_upload(log, model_path: Path | None, sha256: str, nbytes: int) -> P
log.error("jetlink: %s is %d bytes, the registry says %d; not uploading it",
model_path.name, st.st_size, nbytes)
return None
key = (str(model_path), st.st_size, st.st_mtime_ns)
key = _hash_key(model_path, st)
have = _hashed.get(key)
if have is None:
from jetlink.spec import sha256_file
@@ -275,13 +297,41 @@ def ensure(parts, client, sha256: str, nbytes: int, model_path: Path | None, *,
return spec
def stand_in(parts, hello: dict, pick: str, built) -> tuple[str, int] | None:
"""The model to ask for in place of the pick, as (sha256, nbytes), or None
for the pick itself. `built` is the spec record's ready spec, or None.
The server's hello says what it has: the model it has loaded and the ones it
has built. The pick wins whenever the server has it. Otherwise the model it
has loaded drives, so a phone set to its own model serves that rather than
whatever the comma last built on some other server; then the comma's own
record of the last model built, if this server has it too. A server that lists nothing has only the record
to go on."""
cached = hello.get('cached_models')
have = None
if isinstance(cached, list):
loaded = hello.get('loaded')
have = set(cached) | ({loaded} if loaded else set())
if pick in have:
return None
if loaded:
# its size names it to the server, and only a catalog model resolved
# here, or the record, has one
nbytes = built.nbytes if built is not None and built.sha256 == loaded else parts.models.size_for(loaded)
if nbytes:
return loaded, nbytes
if built is not None and built.sha256 != pick and (have is None or built.sha256 in have):
return built.sha256, built.nbytes
return None
def open_link(parts, link: Link, should_stop=None):
"""Get a client and a spec. Link IO only, so it is safe off modeld's thread;
everything that touches tinygrad stays in the joining state's build.
The picked model is built here if the Jetson has not got it, minutes with
the small model driving: a provisioning run works offroad only, so a model
picked in the driveway cost the whole drive otherwise. Only reached with
picked in the driveway would otherwise cost the whole drive. Only reached with
the small model driving (the join loop stops asking once joined), so a build
here never unloads an engine that is steering. Whatever this attempt cannot
use stays on `link`, still open, for the next one.
@@ -302,23 +352,34 @@ def open_link(parts, link: Link, should_stop=None):
hello.get('engine_state'), str(hello.get('loaded'))[:16])
link.note_server(hello)
sha256, nbytes = identity(parts, selected)
if not parts.spec.engine_ready_for(sha256):
path = parts.models.shipped_model_path()
built = parts.spec.ready_spec()
standin = stand_in(parts, hello, sha256, built)
if standin is not None:
# by design, a model already built drives until the
# pick is fetched and built, which a provisioning run does parked, rather
# than the small model. Building here would unload it, and uploading
# would share the link with its frames
parts.log.warning("jetlink: %s is not ready yet, %s drives until it is", selected.get('name', sha256[:16]),
parts.models.name_for(standin[0]))
(sha256, nbytes), path = standin, None
elif built is None or built.sha256 != sha256:
parts.log.warning("jetlink: %s is not built yet, building it with the small model driving",
selected.get('name', sha256[:16]))
try:
# normally one round trip, since the provisioning run left the engine loaded. A
# server that restarted reloads from the plan cache, 13 to 25 s; one
# that has never seen this model builds it, 102 to 294 s
spec = ensure(parts, client, sha256, nbytes, parts.models.shipped_model_path(),
progress=parts.progress.report_with_eta, should_stop=should_stop)
except EngineMissing:
# neither end has the bytes. Fetching them needs the internet and a
# gigabyte of it, which is a parked job; clear the record so the next
# parked period provisions again
parts.spec.clear_ready()
raise
selected.get('name', sha256[:16]))
# normally one round trip, since the provisioning run left the engine loaded. A
# server that restarted reloads from the plan cache, 13 to 25 s; one
# that has never seen this model builds it, 102 to 294 s
spec = ensure(parts, client, sha256, nbytes, path, progress=parts.progress.report_with_eta,
should_stop=should_stop)
client.deadline = INFERENCE_TIMEOUT
return client, spec
except EngineMissing as e:
# neither end has the bytes. Fetching them is a provisioning run's job;
# clear the record if it named this model, so the next one does. The link
# is fine and stays on `link`: closing it leaves a read queued that the
# next attempt's open has to bounce the gadget for
parts.spec.clear_ready(sha256)
raise ModelMissing(str(e)) from e
except BaseException:
link.close()
raise
jetlink_repo/jetlink/openpilot/model_state.py
+112 -146
View File
@@ -13,8 +13,8 @@ frames and context in, 18452 floats out, and holds no control state.
The split is at `run_policy`. The warp stays on the comma's GPU: its input is a
2 MB camera buffer already there and its output the 393 KB the link carries
anyway. The history queues live on the Jetson, shipping them would cost ~10 MB
a frame instead of ~0.5 MB. Upstream fused warp and policy into one JIT, so
the fork's build compiles a standalone warp; see warp.
a frame instead of ~0.5 MB. modeld runs its warp as a JIT of its own, and
the link runs the same one; see warp.
A model that keeps its own history (openpilot #38916, Cinque Terre V3 on)
takes the same warped frame and the same scalars; its hidden state never
@@ -22,8 +22,7 @@ leaves the Jetson, so there is no prev_feat to send back. The spec says which.
What openpilot's modeld reads off a ModelState comes from the fork's adapter
(ModelFace): comma's constants, parser, smoothing and action function.
tinygrad is imported when a model is built, never at module level: only
modeld and the build have it.
tinygrad is warp.Warp's to import: only modeld has it.
"""
from __future__ import annotations
@@ -35,7 +34,7 @@ from dataclasses import dataclass
import numpy as np
from jetlink.openpilot.warp import call_warp
from jetlink.protocol import NotFinite
SEND_RAW_PRED = os.getenv('SEND_RAW_PRED')
SLOW_FRAME = 0.05 # the full 20 Hz budget, not just the largest outliers
@@ -44,7 +43,7 @@ SLOW_FRAME = 0.05 # the full 20 Hz budget, not just the largest outliers
# would be dropped: modeld's own work around run() takes the rest of the 50 ms.
# A held frame is a plan one frame old, which the 10 s plan and the actuator
# delay both dwarf, where a dropped frame is the same stale plan plus a count
# toward selfdrived's modeldLagging. None waits the client's deadline, as before
# toward selfdrived's modeldLagging. None waits the client's full deadline
HOLD_FRAME = 0.046
# Held frames the large model may make before it is behind rather than
# tailing: this many in a row, or more than HOLDS_ALLOWED within HOLD_WINDOW.
@@ -57,12 +56,16 @@ HOLD_FRAME = 0.046
HOLDS_IN_A_ROW = 5
HOLDS_ALLOWED = 20
HOLD_WINDOW = 10.0
# How long a host may answer none of its shadow frames before the link is
# abandoned, where a driving one gets the client's deadline
# (link.INFERENCE_TIMEOUT, 0.2 s). The small model drives meanwhile, and a
# phone's first frames after a join are Neural Engine warm-up: at 0.2 s an
# iPhone 18 Pro lost 11 of 18 joins inside 0.33 s (2026-10-04)
SHADOW_SILENCE = 1.0
# The first frames after a swap, which the fork holds engagement off for (a
# second at 20 Hz): a single held frame among them is behind. A host cold from
# a reconnect proves itself here, before anyone can engage, rather than at the
# wheel, where a slow host's return is a soft disable within a second
PROVING_FRAMES = 20
# the first of them never hand back, held or not: the first after a join
# carries the history reset, and a Mac's is ~100 ms of CoreML warm-up. A
# single slow frame later is the comma's own stall, a warp or a send, which
# handing back would not fix; only held frames say the host is behind
SETTLING_FRAMES = 3
# frames between asks for the server's telemetry, which rides on the response:
# every second one, as modeld sent a chestnut's state (20 Hz over 10 Hz)
TELEMETRY_EVERY = 2
@@ -70,12 +73,6 @@ TELEMETRY_EVERY = 2
TELEMETRY_PERIOD = 1.0
# frames Trips keeps the timings of: a minute at 20 Hz
TRIPS_KEPT = 1200
# shadow frames in a row that would not have been held (check_shadow) before
# the large model may drive (joining reads `keeping_up`): a second at 20 Hz.
# A host cold from a reconnect proves itself with the small model driving,
# not at the wheel, where an iPhone's every return was a soft disable
# within a second (2026-10-04)
PROVING_FRAMES = 20
class Trips:
@@ -86,7 +83,6 @@ class Trips:
budget it cannot measure. Bounded to the last TRIPS_KEPT frames."""
def __init__(self):
self.sent = 0 # frames sent; the ones never ended ran with the small model driving
self.frames = 0 # frames waited for and published
self.over = 0 # frames past SLOW_FRAME, the whole 20 Hz budget
self.held = 0 # frames whose reply was late and published the previous one
@@ -102,9 +98,9 @@ class Trips:
self._server_ms.append(server_us / 1e3)
def summary(self) -> dict:
"""frames, over, held, shadowed, span_s, and over the frames kept: p50_ms,
p99_ms, max_ms of the whole frame and server_ms, the server's mean total."""
out = {'frames': self.frames, 'over': self.over, 'held': self.held, 'shadowed': self.sent - self.frames,
"""frames, over, held, span_s, and over the frames kept: p50_ms, p99_ms,
max_ms of the whole frame and server_ms, the server's mean total."""
out = {'frames': self.frames, 'over': self.over, 'held': self.held,
'span_s': round(time.monotonic() - self.started, 1)}
if self._whole_ms:
whole = sorted(self._whole_ms)
@@ -115,12 +111,12 @@ class Trips:
@dataclass
class Frame:
"""One frame on its way to the host: when the warp and the readback
finished, the bytes the wire takes, then its seq and send time once sent."""
"""One frame on its way to the host: when its warp started, was launched
and finished, then its seq and send time once sent. The bytes are the
warp's output (Warp.output)."""
t0: float
t1: float
t2: float
data: object
seq: int | None = None
sent: float = 0.0
telemetry: bool = False
@@ -134,18 +130,14 @@ class JetlinkModelState:
action function off the ModelState; stock modeld has them as module
constants. This is comma's large model, so they are comma's.
A frame is prepare(), send() and end(). run() is all three, as modeld calls
it. The joining model (joining.JoiningModelState) prepares and sends every
frame the small model drives and never ends them, so the host's model is
warm and current at the swap; it reads `behind` after each driven frame.
A frame is prepare(), send() and end(); run() is all three, as modeld calls
it. The joining model (joining.JoiningModelState) reads `behind` after each
frame.
"""
prev_desire: np.ndarray # for tracking the rising edge of the pulse
def __init__(self, cam_w: int, cam_h: int, client, spec, warp, *, face, log, event):
from tinygrad.device import Device
from tinygrad.tensor import Tensor
self._tensor = Tensor
def __init__(self, client, spec, warp, *, face, log, event):
self._log = log
# (name, **fields): a structured log line, cloudlog.event on a comma
self._event = event
@@ -161,19 +153,13 @@ class JetlinkModelState:
self.client = client
self.spec = spec
# Over a phone's cable, hand the socket the warp's GPU mapping itself: the
# kernel copies it while the first segments are already on the wire, where
# copying it here first held the send back 2.5 ms. On the Mac stand-in the
# comma's side of a frame was 0.6 ms faster at p50 and 2 ms at p99. USB
# keeps the host copy it was measured with. The client's transport says
# which, as the owner lent it
self.send_from_gpu = client.t.link_info().get('kind') == 'cable'
# not chestnut hardware, but the same role: modelV2.big, the UI and the
# model manager key off this flag
self.chestnut = True
# a warm warp, loaded ahead: the first call costs ~2 s and this can run on
# modeld's frame thread (warp.warm)
# a warp.Warp, loaded and warmed ahead: its first call links the JIT, and
# this runs on modeld's frame thread. Frames go out of its output as they
# are, over either transport
self.warp = warp
self.input_shapes = spec.input_shapes
@@ -182,26 +168,16 @@ class JetlinkModelState:
# from the same field
self.frame_skip = spec.frame_skip
# the warp's own two inputs stay separate NPY tensors; everything past the
# warp is the server's. Writing the numpy array is what feeds the JIT
self.npy = {'tfm': np.zeros((3, 3), dtype=np.float32),
'big_tfm': np.zeros((3, 3), dtype=np.float32)}
self.warp_inputs = {k: Tensor(v, device='NPY').realize() for k, v in self.npy.items()}
# compile_modeld.make_input_queues' packed_npy_inputs, minus the GPU queues
# the server owns; for a stateful graph, minus prev_feat too
# compile_modeld.make_input_queues' packed_npy_inputs, minus the warp's
# inputs and the GPU queues the server owns; for a stateful graph, minus
# prev_feat too
self.packed = np.zeros(spec.packed_nelem, dtype=np.float32)
self.npy.update({k: self.packed[at].reshape(shape) for k, (at, shape) in spec.packed_layout.items()})
self.npy = {k: self.packed[at].reshape(shape) for k, (at, shape) in spec.packed_layout.items()}
# read once: it must be the device the cached JIT was compiled against
self.warp_dev = Device.DEFAULT
self.prev_desire = np.zeros(face.desire_len, dtype=np.float32)
self.parser = face.parser()
self.frame_size = face.frame_size(cam_w, cam_h)
# the camera buffers modeld hands over, whatever the graph calls its inputs
self.vision_input_names = ['img', 'big_img']
self.full_frames: dict = {}
self._blob_cache: dict = {}
self._need_reset = True
self._frame_id = 0
self._last_logged = 0.0
@@ -218,8 +194,8 @@ class JetlinkModelState:
self._holds: deque[float] = deque()
self._holds_in_a_row = 0
self.behind: str | None = None
# shadow frames in a row that would not have been held
self._kept_up = 0
# whether the last frame published the one before it: the runtime record's
self.held = False
def slice_outputs(self, model_outputs: np.ndarray, output_slices: dict[str, slice]) -> dict[str, np.ndarray]:
return {k: model_outputs[np.newaxis, v] for k, v in output_slices.items()}
@@ -236,28 +212,22 @@ class JetlinkModelState:
return
self._last_logged = now
self._event("jetlinkTelemetry", dead=bool(self.client.dead), **telemetry)
send = getattr(self.client.t, 'last_send', None)
if send:
self._event("jetlinkSend", nonce=self.client.nonce, frame=self._frame_id, totals=self.client.t.send_totals.copy(), **send)
def prepare(self, bufs: dict, transforms: dict[str, np.ndarray], inputs: dict[str, np.ndarray]) -> None:
"""Take this frame up: read what the host has answered so far, then warp
the frame for it. send() puts it on the link and end() waits for its
answer. A frame sent and never ended ran on the host with the small model
driving (a shadow frame); its timings reach the log here, at the next."""
frame = self._frame
if frame is not None and frame.seq is not None and (self._frame_id <= 3 or frame.sent - frame.t0 > SLOW_FRAME / 2):
# the frame before was sent and never ended: a shadow frame
self._log.warning("jetlink: shadow frame %d warp %.1f data %.1f send %.1f ms; %d unanswered", self._frame_id,
(frame.t1 - frame.t0) * 1e3, (frame.t2 - frame.t1) * 1e3, (frame.sent - frame.t2) * 1e3,
self.client.unanswered)
"""Take this frame up: start the warp, and while the GPU runs it, read
what the host has answered so far (a held frame's late reply) and pack
the rest of the inputs. send() puts it on the link and end() waits for
its answer."""
t0 = time.perf_counter()
self.warp.start(_address(bufs['img']), _address(bufs['big_img']), transforms['img'], transforms['big_img'])
t1 = time.perf_counter()
# the GPU is warping; nothing below is an input to it
self.client.drain()
self.log_telemetry()
for key in bufs.keys():
ptr = np.frombuffer(bufs[key].data, dtype=np.uint8).ctypes.data
cache_key = (key, ptr)
if cache_key not in self._blob_cache:
self._blob_cache[cache_key] = self._tensor.from_blob(ptr, (self.frame_size,), dtype='uint8', device=self.warp_dev)
self.full_frames[key] = self._blob_cache[cache_key]
# Model decides when action is completed, so desire input is just a pulse triggered on rising edge.
# Under whichever name the loop keyed it: stock modeld's desire_pulse, or a modeld_v2 bundle's own
desire = inputs[next(k for k in inputs if k.startswith('desire'))]
@@ -266,124 +236,105 @@ class JetlinkModelState:
self.prev_desire[:] = desire
self.npy['traffic_convention'][:] = inputs['traffic_convention']
self.npy['action_t'][:] = inputs['action_t']
self.npy['tfm'][:, :] = transforms['img'][:, :]
self.npy['big_tfm'][:, :] = transforms['big_img'][:, :]
t0 = time.perf_counter()
warped = call_warp(self.warp, **self.warp_inputs, frame=self.full_frames['img'], big_frame=self.full_frames['big_img'])
t1 = time.perf_counter()
# .data() rather than .numpy(): same ~2.5 ms mean (a write-combined GPU
# mapping read), but no per-frame allocation and no 52 ms outlier. The
# mapping is safe to send: infer_begin returns only once the socket has
# copied all of it, before the next warp can write it
if self.send_from_gpu:
data = warped._buffer().as_memoryview(allow_zero_copy=True)
else:
data = warped.data()
self._frame = Frame(t0, t1, time.perf_counter(), data)
# free again once infer_begin returns: the gadget's io_submit and the
# cable's sendmsg have copied it by then, before the next warp writes it
self.warp.wait()
self._frame = Frame(t0, t1, time.perf_counter())
def send(self, want_telemetry: bool = False, silence: float | None = SHADOW_SILENCE) -> None:
def send(self, want_telemetry: bool = False) -> None:
"""The prepared frame to the host. It asks for the server's telemetry
when the log is due for it, or when told to: modeld asks on the frames it
would have sent a chestnut's state on. Sent from outside, it is a shadow
frame and a quiet host has `silence`; run() sends with the client's
deadline."""
would have sent a chestnut's state on.
With an output to hold, a frame the link cannot take without waiting for
the host to drain earlier ones is not sent, and end() holds it: the same
plan one frame old a late reply publishes, where waiting stalls the frame
loop on a host that is behind."""
frame = self._frame
self._frame_id += 1
frame.telemetry = want_telemetry or time.monotonic() - self._last_logged >= TELEMETRY_PERIOD
frame.seq = self.client.infer_begin(frame.data, self.packed, self._frame_id, reset=self._need_reset,
want_state=frame.telemetry, silence=silence)
try:
frame.seq = self.client.infer_begin(self.warp.output, self.packed, self._frame_id, reset=self._need_reset,
want_state=frame.telemetry, skip_if_busy=self._can_hold)
except Exception:
self._log.warning("jetlink: frame %d send failed: %s", self._frame_id, getattr(self.client.t, 'last_send', {}))
raise
frame.sent = time.perf_counter()
frame.data = None
self._need_reset = False
self.trips.sent += 1
@property
def sent(self) -> bool:
"""Has the prepared frame gone out?"""
return self._frame is not None and self._frame.seq is not None
if frame.seq is not None:
self._need_reset = False
def end(self, after_enqueue: Callable[[], None] | None = None) -> dict[str, np.ndarray]:
"""The sent frame's output, as modeld blocks on a chestnut's, but not past
HOLD_FRAME once there is an output to publish again: a frame that long
is a dropped camera frame. Only a stall past the client's deadline raises,
into the joining state's demotion to the small model."""
is a dropped camera frame. Only a stall past the client's deadline, or an
output that is not finite, raises, into the joining state's demotion to
the small model."""
frame, self._frame = self._frame, None
# publish health while the Jetson works
if after_enqueue is not None:
after_enqueue()
waiting_from = time.perf_counter()
hold = None
if HOLD_FRAME and self.client.last_output is not None:
hold = self._hold_left(frame)
model_output = self.client.infer_end(frame.seq, hold=hold)
if frame.seq is None:
model_output = None # not sent (see send): held
else:
try:
model_output = self.client.infer_end(frame.seq, hold=self._hold_left(frame) if self._can_hold else None)
except NotFinite:
# answered, so the leave's summary counts it
self.trips.record(time.perf_counter() - frame.t0, self.client.last_timings[2])
raise
t4 = time.perf_counter()
held = model_output is None
held = self.held = model_output is None
self.behind = self._note_hold(held)
outputs = self._outputs(self.client.last_output if held else model_output)
self.trips.record(t4 - frame.t0, self.client.last_timings[2])
outputs = self._outputs(self.client.last_output if held else model_output)
# a frame past the budget is a dropped camera frame; send against reply
# says which end it was. Held frames are a steady few on a marginal host,
# so after the first they go to the log at TELEMETRY_PERIOD
log_hold = held and (self.trips.held <= 3 or t4 - self._last_hold_logged >= TELEMETRY_PERIOD)
if self._frame_id <= 3 or t4 - frame.t0 > SLOW_FRAME or log_hold:
note = ''
if held:
self._last_hold_logged = t4
note = f', held ({self.trips.held} so far{", not sent" if frame.seq is None else ""})'
# persisted on the comma so a drive can separate server execution from
# receive stalls once the Jetson is offline; server total excludes USB
gpu_us, queue_us, total_us = self.client.last_timings
receive = getattr(self.client.t, 'last_receive', {})
# over the cable: how frames go, how many never came back, and what the
# kernel dropped on the way in (softnet), since the link came up
drops = getattr(self.client.t, 'net_drops', lambda: None)()
cable = (f"; frames {'as datagrams' if self.client.t.datagrams else 'on the stream'}, "
f"{self.client.frames_lost} lost, net drops {drops}" if drops is not None else '')
self._log.warning("jetlink: frame %d warp %.1f data %.1f send %.1f wait %.1f ms%s; "
"server gpu %.1f queue %.1f total %.1f ms; ffs maxima prepare %.1f read_wait %.1f handoff %.1f ms",
"server gpu %.1f queue %.1f total %.1f ms; ffs maxima prepare %.1f read_wait %.1f handoff %.1f ms%s",
self._frame_id, (frame.t1 - frame.t0) * 1e3, (frame.t2 - frame.t1) * 1e3,
(frame.sent - frame.t2) * 1e3, (t4 - waiting_from) * 1e3,
f', held ({self.trips.held} so far)' if held else '',
note,
gpu_us / 1e3, queue_us / 1e3, total_us / 1e3, receive.get('prepare', 0.0) * 1e3,
receive.get('read_wait', 0.0) * 1e3, receive.get('handoff', 0.0) * 1e3)
receive.get('read_wait', 0.0) * 1e3, receive.get('handoff', 0.0) * 1e3, cable)
return outputs
def run(self, bufs: dict, transforms: dict[str, np.ndarray],
inputs: dict[str, np.ndarray], after_enqueue: Callable[[], None] | None = None) -> dict[str, np.ndarray]:
self.prepare(bufs, transforms, inputs)
self.send(after_enqueue is not None, silence=None)
self.send(after_enqueue is not None)
return self.end(after_enqueue)
@property
def keeping_up(self) -> bool:
"""Would the host have driven the last PROVING_FRAMES shadow frames
without a hold? What the joining model swaps on; nothing while it drives."""
return self._kept_up >= PROVING_FRAMES
def check_shadow(self) -> None:
"""Score the shadow frame just sent as end() would have, now that the
small model's own frame is done: kept up when its reply is back HOLD_FRAME
or less after its warp began, waited for until then as end() waits. The
joining model calls this once per shadow frame; a reply not back by then
is read by the next prepare(), and starts the proof over. The shadow
frame went out after the small model's work was queued, later than a
driven one, so a host that passes here does not hold at the wheel."""
frame = self._frame
if frame is None or frame.seq is None:
return
self.client.drain()
left = self._hold_left(frame)
if not self.client.unanswered:
kept = left >= 0
elif self.client.unanswered == 1 and left > 0:
kept = self.client.infer_end(frame.seq, hold=left) is not None
else:
# past the hold, or behind older frames: replies come in order, so
# this one cannot make it, and waiting would only hold the small model
kept = False
self._kept_up = self._kept_up + 1 if kept else 0
def _can_hold(self) -> bool:
"""Is there an output to publish again instead of waiting (HOLD_FRAME)?"""
return bool(HOLD_FRAME) and self.client.last_output is not None
@staticmethod
def _hold_left(frame) -> float:
"""What is left of the hold for `frame`, from its warp's start."""
return (HOLD_FRAME or SLOW_FRAME) - (time.perf_counter() - frame.t0)
return HOLD_FRAME - (time.perf_counter() - frame.t0)
def _note_hold(self, held: bool) -> str | None:
"""Why the large model should hand back after this frame, if it should:
any held frame among its first PROVING_FRAMES past SETTLING_FRAMES,
HOLDS_IN_A_ROW held frames running, or more than HOLDS_ALLOWED in
HOLD_WINDOW. None while it is keeping up."""
if not held:
@@ -395,6 +346,10 @@ class JetlinkModelState:
while now - self._holds[0] > HOLD_WINDOW:
self._holds.popleft()
self._holds_in_a_row += 1
if self._frame_id <= SETTLING_FRAMES:
return None
if self._frame_id <= PROVING_FRAMES:
return f'held frame {self._frame_id} of the first {PROVING_FRAMES}'
if self._holds_in_a_row >= HOLDS_IN_A_ROW:
return f'held {self._holds_in_a_row} frames in a row'
if len(self._holds) > HOLDS_ALLOWED:
@@ -406,10 +361,17 @@ class JetlinkModelState:
gives the same dict again: the parser's softmax works in place on it."""
if self._parsed is not None and self._parsed[0] is model_output:
return self._parsed[1]
# the non-finite check runs on the server (Status.NOT_FINITE -> LinkError),
# so modeld's big->small failover fires as it does for a chestnut
# the server checks first (Status.NOT_FINITE, which the client raises as
# this). This is for one that did not, or bytes the link changed: a raw
# inf is a corrupt output even where the parser would clip it finite. A
# sum allocates nothing, and any NaN or inf makes it one; these outputs
# are far from overflow
if not np.isfinite(model_output.sum()):
raise NotFinite(f"frame {self._frame_id}: the output is not finite")
outputs_dict = self.parser.parse_outputs(self.slice_outputs(model_output, self.output_slices))
self.spec.feed_back(self.packed, model_output)
# a finite output can still parse to inf, by a parser that does not clip
if not np.isfinite(sum(float(value.sum()) for value in outputs_dict.values())):
raise NotFinite(f"frame {self._frame_id}: the parsed output is not finite")
if SEND_RAW_PRED:
outputs_dict['raw_pred'] = model_output.copy()
self._parsed = (model_output, outputs_dict)
@@ -419,3 +381,7 @@ class JetlinkModelState:
"""Let go of the link; the joining state calls this on a model it retires."""
self.client.close()
def _address(buf) -> int:
"""Where a camera buffer modeld hands over (a VisionBuf) keeps its bytes."""
return np.frombuffer(buf.data, dtype=np.uint8).ctypes.data
jetlink_repo/jetlink/openpilot/models.py
+85 -14
View File
@@ -34,6 +34,11 @@ _TRAILING_DATE = re.compile(r' \([A-Za-z]+ \d{1,2}, \d{4}\)$')
# the index and the slot are JSON params, and the UI names the active model
# every frame; the status line can lag a new pick by this long
INDEX_TTL = 2.0
# A big model's download over a car's connection: the waits between the
# attempts of one provisioning run, one more attempt than waits. Each carries
# on from the .part the last left, so a stall costs the wait, not the bytes
# (lfs.lfs_download)
DOWNLOAD_RETRY_DELAYS = (5.0, 15.0, 30.0, 60.0, 120.0)
class Models:
@@ -77,11 +82,14 @@ class Models:
return known[ref]['oid'], int(known[ref]['size'])
oid, size = self.fetch_pointer(ref)
known[ref] = {'oid': oid, 'size': size}
self._keep_pointers(known)
self.op.log.warning("jetlink: %s is %s, %d MB", ref[:10], oid[:16], size >> 20)
return oid, size
def _keep_pointers(self, known: dict[str, dict]) -> None:
# blocking: the next lookup reads this back, and a put still in flight would be lost under it
self.op.put(self.op.keys.pointers, known, block=True)
self._index_cache = None
self.op.log.warning("jetlink: %s is %s, %d MB", ref[:10], oid[:16], size >> 20)
return oid, size
@staticmethod
def _row(name: str, ref: str, known: dict[str, dict]) -> dict:
@@ -120,7 +128,7 @@ class Models:
"""What runs with no pick: jetlink's default big model, else the newest the
catalog lists. Not the chestnut's default, which is the model in the tree.
A fork with no catalog at all (Keys.catalog None) gets jetlink's default by
its ref; one whose catalog has not been fetched yet waits for it, as ever."""
its ref; one whose catalog has not been fetched yet waits for it."""
if self.op.keys.catalog is None:
return self._row(DEFAULT_BIG_MODEL_NAME, DEFAULT_BIG_MODEL_REF, self._index()[1])
models = self.model_index()
@@ -154,17 +162,18 @@ class Models:
"""One file per model, so switching back does not re-download."""
return f"{model['oid'][:16]}.onnx"
def model_path(self, model: dict) -> Path:
"""Where a model's ONNX lives, whether or not it is there yet."""
return self.model_dir() / self.model_file_name(model)
def shipped_model_path(self) -> Path | None:
"""The chosen large model, if it has been fetched. Keyed on the oid, not
the in-tree pointer, which moves with upstream syncs; the size is the cheap
check that the file is the one we mean."""
model = self.selected_model()
if model is None or not model['oid']:
if model is None or not self.has_file(model):
return None
path = self.model_dir() / self.model_file_name(model)
if path.is_file() and path.stat().st_size == model['size']:
return path
return None
return self.model_path(model)
def lfs_endpoints(self) -> list[str]:
"""Where a model's bytes are asked for, nearest first: the LFS server this
@@ -182,19 +191,43 @@ class Models:
pass
return out + [e for e in LFS_ENDPOINTS if e not in out]
def fetch_shipped_model(self, progress=None, should_stop=None) -> Path | None:
def fetch_shipped_model(self, progress=None, should_stop=None, retrying=None) -> Path | None:
"""Download the chosen large model if it is not here yet; None when nothing
is chosen. The registry streams it to a .part file and hashes it on the way,
so only the whole model ever takes the name."""
so only the whole model ever takes the name.
A transfer that fails is tried again after each of DOWNLOAD_RETRY_DELAYS,
carrying on from the .part it left; once they are spent the .part outlives
the run for the next one. Each attempt asks the LFS server again:
the address it hands out expires. `retrying(error, delay)` hears of each
wait; a run that is stopped stops waiting."""
from jetlink.registry.catalog import NetworkError
from jetlink.registry.lfs import Pointer, lfs_download, lfs_resolve
from jetlink.registry.lfs import Pointer
model = self.selected_model()
if model is None or not model['oid']:
return None
dest = self.model_dir() / self.model_file_name(model)
if dest.is_file() and dest.stat().st_size == model['size']:
dest = self.model_path(model)
if self.has_file(model):
return dest
pointer = Pointer(model['oid'], int(model['size']))
for delay in (*DOWNLOAD_RETRY_DELAYS, None):
try:
return self._fetch_once(pointer, dest, progress, should_stop)
except NetworkError as e:
if delay is None:
raise
self.op.log.warning("jetlink: download of %s failed (%s), trying again in %.0f s", pointer.oid[:16], e, delay)
if retrying is not None:
retrying(e, delay)
end = time.monotonic() + delay
while time.monotonic() < end:
if should_stop is not None and should_stop():
raise
time.sleep(min(1.0, end - time.monotonic()))
def _fetch_once(self, pointer, dest: Path, progress, should_stop) -> Path:
from jetlink.registry.catalog import NetworkError
from jetlink.registry.lfs import lfs_download, lfs_resolve
for endpoint in self.lfs_endpoints():
href = lfs_resolve(endpoint, pointer)
if href is None:
@@ -203,6 +236,30 @@ class Models:
return lfs_download(href, pointer, dest, progress=progress, should_stop=should_stop)
raise NetworkError(f"no LFS server has {pointer.oid[:16]}")
def _row_for(self, oid: str) -> dict | None:
"""The catalog model with this oid, among those we have resolved."""
return next((m for m in self.model_index() if m['oid'] == oid), None)
def name_for(self, oid: str) -> str:
"""The catalog's name for the model with this oid, or the oid's first 16
characters when it lists none we have resolved."""
row = self._row_for(oid)
return row['name'] if row is not None else oid[:16]
def size_for(self, oid: str) -> int | None:
"""The byte count of the catalog model with this oid, once resolved here."""
row = self._row_for(oid)
return row['size'] if row is not None else None
def has_file(self, model: dict) -> bool:
"""Is this model's ONNX on the comma, whole? Its size is the cheap check."""
if not model.get('oid') or not model.get('size'):
return False
try:
return self.model_path(model).stat().st_size == model['size']
except OSError:
return False
# -- the model manager's catalog --------------------------------------------
def big_catalog(self, catalog: dict) -> dict:
@@ -212,7 +269,10 @@ class Models:
Never raises: a probe that fails keeps what the last one found."""
try:
from jetlink.registry.catalog import fetch_catalogs, merge_catalogs
merged = merge_catalogs([catalog, fetch_catalogs()], selector=self.op.catalog_selector)
fetched = fetch_catalogs()
self.keep_pins(fetched.get('pointers', {}))
merged = merge_catalogs([catalog, fetched], selector=self.op.catalog_selector)
merged.pop('pointers', None) # the pins are kept above, never in the model manager's catalog
added = len(merged.get('bundles', [])) - len(catalog.get('bundles', []))
if added <= 0:
return catalog
@@ -222,6 +282,17 @@ class Models:
self.op.log.exception("jetlink: could not check for newer catalogs")
return self._found_before(catalog)
def keep_pins(self, pins: dict[str, dict]) -> None:
"""Writes the list's pins over the kept pointers (see catalog.pinned_pointers)."""
known = self.pointers()
changed = {ref: pin for ref, pin in pins.items() if known.get(ref) != pin}
if not changed:
return
known.update(changed)
self._keep_pointers(known)
for ref, pin in changed.items():
self.op.log.warning("jetlink: %s is pinned to %s", ref[:10], pin['oid'][:16])
def _found_before(self, catalog: dict) -> dict:
"""The catalog with the models the last probe found folded in again, from the
model manager's cached copy: every entry there the fetched catalog does not
jetlink_repo/jetlink/openpilot/provision.py
+62 -17
View File
@@ -84,30 +84,66 @@ class ProvisioningRun:
# -- provisioning ---------------------------------------------------------
def fetch_model(self):
"""Download the pinned large model, once.
"""Download the picked large model, if the comma has not got it whole.
Minutes on a slow link, so it reports progress and stops when the owner
stops the run; it is the one call in a run that blocks for long.
stops the run; it is the one call in a run that blocks for long. A stall is
retried within the run and resumed from where it stopped, by this run or
the next (Models.fetch_shipped_model); a run gives up only once those
retries are spent, and the .part waits for the next one.
"""
if self.fetch_failed:
return None
parts = self.parts
model = parts.models.selected_model() or {}
size = model.get('size') or 0
got = 0.0
def progress(frac: float) -> None:
nonlocal got
got = frac
got_mb, total_mb = int(frac * size) >> 20, size >> 20
parts.progress.report('download', frac, f"downloading {got_mb} of {total_mb} MB" if size else 'downloading',
code='downloading', args={'got_mb': got_mb, 'total_mb': total_mb} if size else None)
def retrying(error, delay: float) -> None:
parts.progress.report('download', got, f"download stalled, retrying in {delay:.0f} s",
code='download_stalled', args={'retry_s': round(delay)})
try:
path = self.parts.models.fetch_shipped_model(
progress=lambda frac: self.parts.progress.report('download', frac, 'downloading'),
should_stop=lambda: self.stop,
)
path = parts.models.fetch_shipped_model(progress=progress, should_stop=lambda: self.stop, retrying=retrying)
except Exception:
self.log.exception("jetlink: could not fetch the large model")
self.parts.progress.report('failed', 1.0, 'could not download the large model')
# one attempt per run; retrying a gigabyte on a loop is worse than staying small
if self.stop:
self.log.warning("jetlink: download stopped, the next run carries on from where it got to")
else:
# the owner starts another run after its backoff, which resumes the .part
self.log.exception("jetlink: could not fetch the large model")
parts.progress.report('failed', 1.0, 'download failed, retrying later', code='download_failed')
self.fetch_failed = True
return None
if path is not None and model.get('oid'):
link.remember_hash(path, model['oid']) # the download proved it; the upload need not read it again
return path
def needs_download(self) -> bool:
"""Does the comma have to fetch the picked model before a Jetson can build
it? Only while its engine is not known to be built and the file is not here.
Asked without the Jetson: a download needs only the internet, and a car
whose Jetson goes off with the ignition has no Jetson while parked."""
parts = self.parts
entry = parts.models.selected_model()
if entry is None:
return False
sha256, _ = link.identity(parts, entry)
if parts.spec.engine_ready_for(sha256):
return False
return parts.models.shipped_model_path() is None
def provision(self) -> bool:
"""Make the Jetson ready for the selected model; a host is attached. The
file is fetched only when the server asks for the bytes: the Jetson keeps
its own copy of every ONNX and never prunes it."""
"""Make the Jetson ready for the selected model; a host is attached. run()
has fetched the file already unless the spec record said the model was
built; a server that lost it since asks for the bytes, and they are
fetched then."""
from jetlink.client import EngineMissing
parts = self.parts
entry = parts.models.selected_model()
@@ -124,7 +160,7 @@ class ProvisioningRun:
self.log.warning("jetlink: provisioning %s (%d MB, sha %s)",
entry.get('name', sha256[:16]), nbytes >> 20, sha256[:16])
parts.progress.report('connect', 0.0, 'connecting')
parts.progress.report('connect', 0.0, 'connecting', code='connecting')
def ensure(path):
return link.ensure(parts, self.client, sha256, nbytes, path, progress=parts.progress.report_with_eta,
@@ -137,15 +173,15 @@ class ProvisioningRun:
except EngineMissing:
# the server has nothing to build from, and neither have we: fetch the
# model and hand it over in this run. The owner lends the link for the
# whole run, so leaving after the download only put the build a
# WORKER_BACKOFF later, five minutes of a finished download doing nothing
# whole run; leaving after the download would put the build a
# WORKER_BACKOFF later
if model_path is not None or (model_path := self.fetch_model()) is None:
raise
# a download of minutes can outlast the session; a hello starts a new one
self.hello()
spec = ensure(model_path)
parts.progress.report('ready', 1.0, 'engine ready')
parts.progress.report('ready', 1.0, 'engine ready', code='engine_ready')
self.log.warning("jetlink: engine ready for %s", spec.sha256[:16])
return True
@@ -229,15 +265,24 @@ class ProvisioningRun:
finished = False
try:
# the bytes first while no host is on the cable: a parked car whose
# Jetson is switched with the ignition never has one to wait for. A host
# that is there is asked first, since one with the model built needs
# nothing from the comma; provision() fetches it if the host has not
if self.needs_download() and not gadget.host_attached() and self.fetch_model() is None:
return False
if not self.open_link():
return False
if not self.wait_for_jetson():
self.log.warning("jetlink: no jetson within %.0f s, leaving it for the next run", WAKE_TIMEOUT)
if self.parts.models.shipped_model_path() is not None:
self.parts.progress.report('waiting', 0.0, 'downloaded, builds when the jetson is on',
code='downloaded_waiting_host')
return False
finished = self.provision()
except Exception:
self.log.exception("jetlink: provisioning failed")
self.parts.progress.report('failed', 1.0, 'see the log')
self.parts.progress.report('failed', 1.0, 'see the log', code='failed')
finally:
self.close_link()
return finished
jetlink_repo/jetlink/openpilot/state.py
+25 -11
View File
@@ -4,7 +4,7 @@ Copyright (c) 2026-, Zeph Leggett.
This file is part of jetlink and is licensed under the MIT License.
See the LICENSE file in the root directory for more details.
The picked model's spec, and whether its engine is built, in one record.
The spec of the last model a server built (the pick, or a stand-in for it), and whether its engine is built, in one record.
Reading the shapes and output slices means parsing a 766 MB ONNX. The server
does that when it builds the engine and answers with the spec; provisioning
@@ -13,7 +13,7 @@ record also says whether the server has built the engine for the sha it names,
so the spec and the readiness can never name different models. It must
survive a reboot, or every ignition rebuilds a 160 s engine.
A param today (Keys.spec), through the adapter's get and put.
Stored as a param (Keys.spec), through the adapter's get and put.
"""
from __future__ import annotations
@@ -27,11 +27,12 @@ class SpecRecord:
value = self.op.get(self.op.keys.spec)
return value if isinstance(value, dict) else None
def load(self):
"""The recorded ModelSpec, or None if there is not a usable one."""
def load(self, d: dict | None = None):
"""The recorded ModelSpec, or None if there is not a usable one. `d` is
the record, when the caller has read it already."""
from jetlink.spec import ModelSpec
try:
d = self._raw()
d = self._raw() if d is None else d
return ModelSpec.from_dict(d) if d else None
except Exception:
self.op.log.exception("jetlink: cached spec is unreadable")
@@ -43,12 +44,25 @@ class SpecRecord:
def engine_ready_for(self, sha256: str | None) -> bool:
"""Has the server built the engine for this model? Params only."""
d = self._raw()
return bool(sha256) and d is not None and d.get('sha256') == sha256 and d.get('ready') is True
return bool(sha256) and self.built_sha() == sha256
def clear_ready(self) -> None:
"""The engine is no longer known to be built. The spec stays: it still
sizes the warp, and the next provisioning run asks again."""
def built_sha(self) -> str | None:
"""ready_spec()'s sha256, without making the spec of it."""
d = self._raw()
if d is not None and d.get('ready'):
return d.get('sha256') if d is not None and d.get('ready') is True else None
def ready_spec(self):
"""The last model whose engine the server built, whichever model is picked
now: what drives while a new pick is still being fetched or built. None
without one."""
d = self._raw()
return self.load(d) if d is not None and d.get('ready') is True else None
def clear_ready(self, sha256: str | None = None) -> None:
"""The engine is no longer known to be built. The spec stays: it still
sizes the warp, and the next provisioning run asks again. With `sha256`,
only if the record is that model's: a new pick the server has nothing of
says nothing about the last model it built, which still drives."""
d = self._raw()
if d is not None and d.get('ready') and (sha256 is None or d.get('sha256') == sha256):
self.op.put(self.op.keys.spec, {**d, 'ready': False})
jetlink_repo/jetlink/openpilot/status.py
+189 -24
View File
@@ -22,13 +22,24 @@ from jetlink.comma import gadget
# -- progress -----------------------------------------------------------------
# Written by the provisioning run and the joining state, read by the UI: a
# param today (Keys.progress), because the writer is another process.
# param (Keys.progress), because the writer is another process.
# Each report is a file write, and the UI reads it at 5 Hz. An upload reports
# once per 4 MB chunk, 440 of them for a 1.7 GB model, and onroad that is IO a
# recording would have to share the disk with.
PROGRESS_MIN_INTERVAL = 0.25
# what a progress record's `code` may be, for a fork that words the line
# itself; `msg` stays the English one. `args`, where a code has them:
# downloading and upload {got_mb, total_mb}, download_stalled {retry_s},
# reconnecting {why: lost, behind, load_failed or bad_output}, build {eta_s}.
# From upload on they are stage names, and the server's stages are its own:
# a server can add one, so a fork falls back to `msg` for a code it does not
# know. A server message with no stage has no code
PROGRESS_CODES = ('waiting_for_host', 'finding_model', 'model_missing', 'ready_waiting_window', 'reconnecting',
'connecting', 'downloading', 'download_stalled', 'download_failed', 'downloaded_waiting_host',
'engine_ready', 'failed', 'upload', 'patch', 'parse', 'convert', 'compile', 'build', 'load', 'warm')
def estimated_build_seconds(size: int) -> int:
"""Orin Nano Super, TensorRT 10.3: the 766 MB models built in 102 to 166 s,
@@ -57,21 +68,28 @@ class Progress:
return None
return value if isinstance(value, dict) else None
def report(self, stage: str, frac: float, msg: str = '', drops: int = 0) -> None:
def report(self, stage: str, frac: float, msg: str = '', drops: int = 0, code: str | None = None,
args: dict | None = None) -> None:
"""Never raises: called from except handlers. A fraction's ticks are held to
4 Hz within a stage; a report with no fraction (a join's state) and the end
of a stage always go through, so the panel's last word is never dropped. A
join says 'waiting for jetlink' and 'checking the link keeps up' 50 ms apart;
join says 'waiting for jetlink' and 're-engage to switch' moments apart;
holding the second left the first on the panel for the rest of the drive.
`msg` is the panel's one short line; `drops`, once the link has dropped often
enough to blame the cable, is the panel's to word."""
enough to blame the cable, is the panel's to word; `code` and `args` say
`msg` for a fork with its own words (PROGRESS_CODES)."""
last_stage, last_at = self._last
now = time.monotonic()
if 0.0 < frac < 1.0 and stage == last_stage and now - last_at < PROGRESS_MIN_INTERVAL:
return
self._last = (stage, now)
record = {'stage': stage, 'frac': round(frac, 4), 'msg': msg, 'drops': drops}
if code is not None:
record['code'] = code
if args:
record['args'] = args
try:
self.op.put(self.op.keys.progress, {'stage': stage, 'frac': round(frac, 4), 'msg': msg, 'drops': drops})
self.op.put(self.op.keys.progress, record)
except Exception:
self.op.log.exception("jetlink: could not report progress")
@@ -85,11 +103,14 @@ class Progress:
"""Progress, with how long the build still has to run, from the model's
size. Only the build: the upload reports MB of MB and a connect has nothing
to predict."""
if stage == 'build':
size = self._size() if self._size is not None else None
if size:
msg = _eta(estimated_build_seconds(size) * max(0.0, 1.0 - frac))
self.report(stage, frac, msg)
args = None
size = self._size() if self._size is not None and stage in ('build', 'upload') else None
if size and stage == 'build':
left = estimated_build_seconds(size) * max(0.0, 1.0 - frac)
msg, args = _eta(left), {'eta_s': round(left)}
elif size:
args = {'got_mb': int(frac * size) >> 20, 'total_mb': size >> 20}
self.report(stage, frac, msg, code=stage or None, args=args)
# -- what is on the other end -------------------------------------------------
@@ -168,8 +189,13 @@ def usb_port() -> str | None:
# -- whether the link can run --------------------------------------------------
# the offroad alert's text for a device whose build made no warp for its camera
# the offroad alert's text for a device whose checkout has no warp for its camera
NO_WARP = "no warp built for this camera"
# how a failed snapshot starts its reason (the owner's backoff: gadget.BACKING_OFF)
STATUS_FAILED = "jetlink status failed"
# Status.reason_code: the reason's kind, for a fork with its own words. Any
# reason that is none of the others is the gadget's, its text the detail
REASON_CODES = ('no_warp', 'service_stopped', 'backing_off', 'status_failed', 'gadget')
def unavailable(parts, record: dict | None) -> str | None:
@@ -187,21 +213,61 @@ def unavailable(parts, record: dict | None) -> str | None:
return None if parts.warps.built() else NO_WARP
def _built_for_the_pick(parts) -> bool:
spec = parts.spec.load()
selected = parts.models.selected_model()
return (spec is not None and selected is not None and spec.sha256 == selected['oid']
and parts.spec.engine_ready_for(spec.sha256))
def reason(parts, mode: str) -> str | None:
"""Why the link cannot run, for someone who asked for it only: with it off,
a device that cannot present the gadget simply does not offer the feature."""
return unavailable(parts, owner_record()[0]) if parts.enabled(mode) else None
# -- what modeld runs ------------------------------------------------------------
# a big model drives while its last answer was this recent when the record
# was written: five frames at 20 Hz, so a held frame or two still counts
OUTPUT_FRESH = 0.25
# and while the record itself is this recent: a modeld that froze stops
# reading as driving within it. Two writer periods, since the writer runs at
# background priority and a late write must not read as a hand-back; below the
# owner's HEARTBEAT_TIMEOUT, which `running` keeps
ACTIVE_FRESH = 2 * gadget.RECORD_PERIOD
class Runtime(NamedTuple):
state: str | None = None # the joining model's state (STATES)
active: bool = False # the big model drives modeld's frames now
sha256: str | None = None # and which model it is, while it does
checkpoint: str | None = None
def runtime() -> Runtime:
"""The joining model's state, whether the big model drives now and which
model it is, from modeld's runtime record (gadget.MODELD); Runtime() without
a live one: no modeld with the link since boot, or one that stopped, died
or stopped writing (gadget.HEARTBEAT_TIMEOUT, as the owner's). Never raises."""
record = gadget.read_record(gadget.MODELD)
try:
if record is None or record.get('stopping') or not gadget.owner_alive(record) \
or not gadget.pid_alive(record.get('pid')):
return Runtime()
# the answer's age when written: the writer wakes once a second, the frames 20 times
answered = record['at'] - record.get('last_output_at', 0.0) < OUTPUT_FRESH
active = bool(record.get('active')) and answered and time.monotonic() - record['at'] < ACTIVE_FRESH
def text(key: str, shown: bool = True) -> str | None:
value = record.get(key)
return value if shown and isinstance(value, str) else None
return Runtime(text('state'), active, text('sha256', active), text('checkpoint', active))
except (TypeError, KeyError):
return Runtime()
# -- the snapshot ----------------------------------------------------------------
# Status.phase, in the order it is decided (see there). Plain strings, and
# like the codes only ever added to: a fork maps one it does not know to its
# own default
PHASES = ('unavailable', 'off', 'active', 'waiting', 'reconnecting', 'failed', 'preparing', 'connecting', 'prepared')
class Status(NamedTuple):
"""Everything a reader shows, taken at once. Fields are only ever added."""
enabled: bool # the setting is on and no chestnut is fitted
@@ -214,11 +280,87 @@ class Status(NamedTuple):
progress: dict | None # {stage, frac, msg} while provisioning or joining
model: str | None # the big model it will run: the pick, else jetlink's default
default_model: str | None # jetlink's default, named like the chestnut's (no build date)
standin: str | None = None # the last model built, which drives while the pick is not ready
running: str | None = None # the joining model's state (STATES) in a live modeld, by its record (runtime)
active: bool = False # the big model drives modeld's frames now, by the same record
running_sha256: str | None = None # the model it drives with, while active
running_checkpoint: str | None = None
@property
def runnable(self) -> bool:
"""Is a big model built to run: the pick, or the stand-in for it?"""
return self.ready or self.standin is not None
@property
def active_model(self) -> str | None:
"""model, once it can run."""
return self.model if self.ready else None
"""The big model that runs: the pick once it is built, else the stand-in."""
return self.model if self.ready else self.standin
@property
def shown(self) -> bool:
"""Anything of the link's to show: it is on, a host is there, or progress."""
return self.enabled or self.present or self.progress is not None
@property
def holds_port(self) -> bool:
"""The link holds the comma's USB port, so ADB cannot have it."""
return self.enabled
@property
def port_state(self) -> str | None:
"""The port line's: 'connected' with a host on the link, else the CC
pin's 'empty' or 'device' (a cable with something live behind it, it
cannot say what); None where the kernel does not say."""
if self.present:
return 'connected'
if self.port is None:
return None
return 'empty' if self.port == 'empty' else 'device'
@property
def reason_code(self) -> str | None:
"""`reason` as one of REASON_CODES, None without one."""
reason = self.reason
if reason is None:
return None
if reason == NO_WARP:
return 'no_warp'
if reason == STOPPED:
return 'service_stopped'
if reason.startswith(gadget.BACKING_OFF):
return 'backing_off'
return 'status_failed' if reason.startswith(STATUS_FAILED) else 'gadget'
@property
def phase(self) -> str:
"""One of PHASES, what a panel's state chip says, decided in that order.
Why the link cannot run comes first (a failed snapshot has the reason
with enabled False), then the setting, then what a live modeld says it
runs, then the progress stage, then the records. Without a live record
the progress line decides, and it can be a drive old: a modeld that is
never closed leaves its join's last 'connect' line, which reads
connecting offroad, as the icon reads loading. A big model that stopped
answering while it drives is reconnecting: it is about to hand back. A
build inside a join is preparing; a connect is connecting, a join's or a
provisioning run's."""
if self.reason is not None:
return 'unavailable'
if not self.enabled:
return 'off'
if self.active:
return 'active'
if self.running == 'ready':
return 'waiting'
if self.running in ('retrying', 'running'):
return 'reconnecting'
stage = str((self.progress or {}).get('stage') or '')
if stage == 'failed':
return 'failed'
if stage not in ('', 'ready', 'connect'):
return 'preparing'
if self.running == 'joining' or stage == 'connect':
return 'connecting'
return 'prepared' if self.runnable else 'preparing'
def icon(self, started: bool, model_seen: bool, running_big: bool, state: str) -> str:
"""The chestnut icon's state for the link: a ChestnutState value.
@@ -234,7 +376,7 @@ class Status(NamedTuple):
return 'disconnected'
if stage and stage != 'ready':
return 'failed' if stage == 'failed' else 'loading'
return 'ready' if self.ready else 'uncompiled'
return 'ready' if self.runnable else 'uncompiled'
if model_seen and running_big:
return 'active'
@@ -247,7 +389,7 @@ class Status(NamedTuple):
# nothing is in control, so a driver who stays engaged keeps the small model
if state == 'ready':
return 'waiting'
if not self.ready:
if not self.runnable:
return 'uncompiled'
if state == 'running':
return 'active'
@@ -258,21 +400,44 @@ def read(parts, mode: str) -> Status:
"""Everything at once, over the link setting the caller read: each file once."""
record, live = owner_record()
enabled = parts.enabled(mode)
live_model = runtime() if enabled else Runtime()
reason = unavailable(parts, record) if enabled else None
# the last model the server built; the pick's, or the stand-in that drives
# until the pick is built (link.open_link)
built = parts.spec.built_sha() if enabled and reason is None else None
selected = parts.models.selected_model() if built is not None else None
ready = built is not None and selected is not None and built == selected['oid']
return Status(
enabled=enabled,
mode=mode,
transport=link_transport(mode, live),
present=bool(live.get('present')) if live is not None else parts.presence.present(),
port=usb_port(),
ready=enabled and reason is None and _built_for_the_pick(parts),
ready=ready,
reason=reason,
progress=parts.progress.read(),
model=parts.models.selected_model_name(),
default_model=parts.models.default_model_name(),
standin=parts.models.name_for(built) if built is not None and not ready else None,
running=live_model.state,
active=live_model.active,
running_sha256=live_model.sha256,
running_checkpoint=live_model.checkpoint,
)
def model_state(parts, ref: str) -> str | None:
"""One catalog model, for the picker's list: 'ready' when the Jetson has
built it (the spec record), 'downloaded' when the comma has its file, else
None. Records and one stat; a model never resolved has neither."""
model = next((m for m in parts.models.model_index() if m['ref'] == ref), None)
if model is None or not model.get('oid'):
return None
if parts.spec.engine_ready_for(model['oid']):
return 'ready'
return 'downloaded' if parts.models.has_file(model) else None
def failed(error: str, mode: str) -> Status:
"""What a reader gets when the snapshot itself failed: nothing to show, the
setting as it is, and, for someone who asked for the link, the failure where
@@ -282,4 +447,4 @@ def failed(error: str, mode: str) -> Status:
def failure(error: str, mode: str) -> str | None:
return f"jetlink status failed: {error}" if mode != 'off' else None
return f"{STATUS_FAILED}: {error}" if mode != 'off' else None
jetlink_repo/jetlink/openpilot/stock.py
+266
View File
@@ -0,0 +1,266 @@
"""
Copyright (c) 2026-, Zeph Leggett.
This file is part of jetlink and is licensed under the MIT License.
See the LICENSE file in the root directory for more details.
The Openpilot interface over stock openpilot's paths: the base a fork's
adapter subclasses, overriding only what its fork does differently.
The one jetlink module that imports openpilot, and only inside a method, where
it is used: the top level is the standard library and interface.py, because
the resident owner imports this module (through the fork's owner_config()).
Which method runs where matters. owner_config() imports only
openpilot.common.basedir (os alone); anything under common.hardware loads
capnp, swaglog loads zmq, and model_face() imports modeld itself, so it is for
modeld alone, where that module is already loaded.
Data are class attributes, behaviour small methods, each a fork may override;
one override point per value: a fork sets KEYS and CATALOG_SELECTOR, and the
interface's keys and catalog_selector read them. Both are jetlink's contract:
renaming or removing one, or changing what one means, is a breaking change
and bumps API. scripts/upstream (the upstream-paths CI job) checks the paths
and literals here against commaai/openpilot master weekly.
A fork's adapter module defines its subclass lazily, as its top level must
import without a jetlink checkout: docs/porting.md, "The adapter module".
Defined there, the class's module names the adapter module the provisioning
run is started with, so it needs no ADAPTER_MODULE.
"""
from __future__ import annotations
import os
import time
from functools import cached_property
from pathlib import Path
# root.AGNOS is openpilot's own test for a comma (common.hardware.AGNOS), which
# that package answers only after loading capnp; root is on the owner's path
from jetlink.comma import root
from jetlink.openpilot.interface import Keys, ModelFace, OwnerConfig
# how long a Params or SubMaster that failed to load is left before the next
# try: in_control() asks on every realtime frame
RETRY_S = 5.0
# what the base engaged() reads, by service: only those in IN_CONTROL_SERVICES
_ENGAGED = {'carControl': ('enabled', 'latActive', 'longActive'), 'selfdriveState': ('enabled', 'active')}
# what marks stock modeld's ModelState, which small_reset() zeroes (and its prev_desire)
_MODEL_STATE = ('input_queues', 'state_pairs', 'packed_input')
class Stock:
"""jetlink.openpilot.interface.Openpilot over stock openpilot (the 0.11 layout)."""
# the fork's adapter module, which the provisioning run imports; None for
# the module the subclass is defined in
ADAPTER_MODULE: str | None = None
# the canonical names, which a fork declares in params_keys.h. No model
# manager upstream: no big-model slot or catalog, so jetlink runs its default
KEYS = Keys(link='JetlinkLink', offroad='IsOffroad', progress='JetlinkProgress', spec='JetlinkSpec',
pointers='JetlinkModelPointers', charge_phone='JetlinkChargePhone')
# the model manager's catalog version; none upstream
CATALOG_SELECTOR = 0
# comma's chestnut, running and in its ROM (common.hardware.usb's
# CHESTNUT_USB_IDS + CHESTNUT_ROM_USB_IDS, which CI holds this to): written
# out, as the owner cannot import that package
CHESTNUT_IDS = frozenset({(0xADD1, 0x0001), (0x3801, 0x0001), (0x174C, 0x2464), (0x174C, 0x2463)})
# what in_control() reads; carState too, so a dead card counts as in control.
# It goes with engaged(): the base reads only the services listed here, so a
# fork that adds one (zoompilot's carControlSP) overrides engaged() to read it
IN_CONTROL_SERVICES = ('carState', 'carControl', 'selfdriveState')
# modeld's driving warp, which its SConscript compiles per camera
WARP_DIR = 'openpilot/selfdrive/modeld/models'
WARP_NAME = 'driving_warp_{cam_w}x{cam_h}_tinygrad.pkl'
OWNER_LOG = Path('/data/log/jetlink-owner.log')
def __init__(self):
# nothing imported here: owner_config() builds one in the owner
self._stores: dict[Path, object] = {}
self._params_after = 0.0
self._sm = None
self._sm_after = 0.0
self._in_control_error: str | None = None
# -- the readers --------------------------------------------------------------
# the interface's members, over the class attributes a fork sets
@property
def keys(self) -> Keys:
return self.KEYS
@property
def catalog_selector(self) -> int:
return self.CATALOG_SELECTOR
@cached_property
def log(self):
from openpilot.common.swaglog import cloudlog
return cloudlog
@cached_property
def basedir(self) -> Path:
# the checkout's root, which holds .lfsconfig and the openpilot package
from openpilot.common.basedir import BASEDIR
return Path(BASEDIR)
def params_dir(self) -> Path:
# params.cc and hw.h's rule: PARAMS_ROOT, else /data/params on a comma and
# $HOME/.comma<OPENPILOT_PREFIX>/params elsewhere, then /<OPENPILOT_PREFIX,
# or d>. HOME unset gives /.comma, as the C++ concatenation does. The
# environment only, per call, so it follows a prefix as Params does
prefix = os.environ.get('OPENPILOT_PREFIX', '')
store = os.environ.get('PARAMS_ROOT')
if store is None:
store = '/data/params' if root.AGNOS else os.environ.get('HOME', '') + '/.comma' + prefix + '/params'
return Path(store) / os.environ.get('OPENPILOT_PREFIX', 'd')
def _params(self):
# one Params per store: making one costs more than the read on a comma, and
# the UI reads several five times a second. By store, for a test's prefix
where = self.params_dir()
store = self._stores.get(where)
if store is None:
# one that failed to load is not tried again for RETRY_S: get() is on the UI's frame
if time.monotonic() < self._params_after:
raise RuntimeError("openpilot's Params failed to load, retrying shortly")
try:
from openpilot.common.params import Params
store = self._stores[where] = Params()
except Exception:
self._params_after = time.monotonic() + RETRY_S
raise
return store
def get(self, key: str):
# hardwared's and the UI's threads read here, which UnknownKeyName (a
# params library older than the key) must not take down
try:
return self._params().get(key)
except Exception:
return None
def chestnut_present(self) -> bool:
# any chestnut id, its ROM's included, the owner's walk: never the port
# beside one, even one modeld ignores until its firmware matches
from jetlink.comma.port import chestnut_attached
return chestnut_attached(self.CHESTNUT_IDS)
def camera(self) -> tuple[int, int, int, int]:
# modeld takes the size from camerad's buffer; a comma four's os04c10 is
# the 1344x760 one, every other device's the 1928x1208
from openpilot.common.hardware import HARDWARE
from openpilot.common.transformations.camera import _ar_ox_fisheye, _os_fisheye
from openpilot.common.transformations.model import MEDMODEL_INPUT_SIZE
camera = _os_fisheye if HARDWARE.get_device_type() == 'mici' else _ar_ox_fisheye
return camera.width, camera.height, *MEDMODEL_INPUT_SIZE
def warp_path(self, cam_w: int, cam_h: int, model_w: int, model_h: int) -> Path:
# scons builds it for the one model input size, which jetlink checks the warp makes
return self.basedir / self.WARP_DIR / self.WARP_NAME.format(cam_w=cam_w, cam_h=cam_h, model_w=model_w, model_h=model_h)
# -- the provisioning run --------------------------------------------------------
def put(self, key: str, value, *, block: bool = False) -> None:
self._params().put(key, value, block=block)
def remove(self, key: str) -> None:
self._params().remove(key)
def model_root(self) -> Path:
# sunnypilot's Paths.model_root(), where there is one; upstream has none,
# and the same place it would be
from openpilot.common.hardware.hw import Paths
if hasattr(Paths, 'model_root'):
return Path(Paths.model_root())
return Path('/data/media/0/models') if root.AGNOS else Path(Paths.comma_home()) / 'media' / '0' / 'models'
# -- modeld ---------------------------------------------------------------------
def parser(self):
"""A new Parser for comma's large model's outputs."""
from openpilot.selfdrive.modeld.parse_model_outputs import Parser
return Parser()
def model_constants(self):
"""The ModelConstants modeld reads off the model."""
from openpilot.selfdrive.modeld.constants import ModelConstants
return ModelConstants
def action_function(self):
"""modeld's get_action_from_model."""
from openpilot.selfdrive.modeld.modeld import get_action_from_model
return get_action_from_model
def model_face(self) -> ModelFace:
from openpilot.selfdrive.modeld.constants import ModelConstants
from openpilot.selfdrive.modeld.modeld import LAT_SMOOTH_SECONDS, LONG_SMOOTH_SECONDS
from openpilot.system.camerad.cameras.nv12_info import get_nv12_info
# desire_len is comma's large model's, which is stock modeld's
return ModelFace(parser=self.parser, frame_size=lambda w, h: get_nv12_info(w, h)[3],
desire_len=ModelConstants.DESIRE_LEN, constants=self.model_constants(),
lat_smooth_seconds=LAT_SMOOTH_SECONDS, long_smooth_seconds=LONG_SMOOTH_SECONDS,
get_action_from_model=self.action_function())
def engaged(self, sm) -> bool:
"""Is openpilot in control, off fresh and valid IN_CONTROL_SERVICES?
carControl's enabled or either axis, or selfdriveState's enabled or active,
of those services the fork lists."""
return any(getattr(sm[service], field) for service, fields in _ENGAGED.items()
if service in self.IN_CONTROL_SERVICES for field in fields)
def in_control(self) -> bool:
# on modeld's frame thread before every frame: a poll that never waits, and
# a service late or invalid, or any failure, holds the swap off, never modeld
try:
if self._sm is None:
# one try per RETRY_S, not one per frame
if time.monotonic() < self._sm_after:
return True
self._sm_after = time.monotonic() + RETRY_S
from openpilot.cereal import messaging
self._sm = messaging.SubMaster(list(self.IN_CONTROL_SERVICES))
services = self.IN_CONTROL_SERVICES
self._sm.update(0)
result = not (self._sm.all_alive(services) and self._sm.all_valid(services)) or self.engaged(self._sm)
except Exception as e:
# once per distinct error: this runs 20 times a second
if self._in_control_error != (text := f"{type(e).__name__}: {e}"):
self._in_control_error = text
try:
self.log.error("jetlink: in_control() failed: %s", text)
except Exception:
pass
return True
self._in_control_error = None
return bool(result)
def event(self, name: str, **fields) -> None:
self.log.event(name, **fields)
def small_reset(self, small):
"""Stock modeld's ModelState, as its warmup leaves it: the recurrent state
on the GPU, the packed host inputs and the desire edge, zeroed in place.
None for another shape, so jetlink's own reset runs."""
if not all(hasattr(small, name) for name in _MODEL_STATE):
return None
from jetlink.openpilot.warp import reset_for_stock
return reset_for_stock(small, getattr(small, 'prev_desire', None))
# -- the adapter module's, for jetlinkd --------------------------------------------
@classmethod
def owner_config(cls) -> OwnerConfig:
"""What jetlinkd needs, data only: openpilot.common.basedir is the one import."""
from openpilot.common.basedir import BASEDIR
op = cls()
# the provisioning run starts in the checkout with it on the path, where
# the launch script links jetlink in
return OwnerConfig(params_dir=op.params_dir(), keys=op.keys, chestnut_ids=cls.CHESTNUT_IDS,
adapter=cls.ADAPTER_MODULE or cls.__module__, cwd=Path(BASEDIR), env={'PYTHONPATH': BASEDIR},
log_file=cls.OWNER_LOG)
@classmethod
def main(cls) -> None:
"""jetlinkd: hold the USB gadget until manager stops this process."""
from jetlink.openpilot.owner import main
main(cls.owner_config())
jetlink_repo/jetlink/openpilot/testing.py
+452
View File
@@ -0,0 +1,452 @@
"""
Copyright (c) 2026-, Zeph Leggett.
This file is part of jetlink and is licensed under the MIT License.
See the LICENSE file in the root directory for more details.
A kit a fork runs in its own CI, against its real adapter and modelds, and
the fakes jetlink's own tests share.
check_adapter(module) the adapter module: every side of Openpilot, a top
level of the standard library alone, an owner_config()
that imports nothing heavy, an in_control() that
answers a bool at once
check_seam(path) a modeld's main(): prepare() before it goes realtime,
attach() once the small model is built and its
result run, handed_over() read
check_fallback(obj) the fork's own hooks for no usable jetlink: each
answers as Off does, over HOOK_CALLS
install_guard() an audit hook a conftest starts and stops around each
test, which records a reach for the machine's own
params, /dev/shm records, sysfs or sudo
FakeJoinedModel, FakeSubMaster, RecordingLog, and tinygrad over numpy
(fake_tinygrad)
The checks return their problems, [] when there are none, so one assertion
shows them all: assertEqual(check_adapter(adapter_module), []).
Not on the owner's path, and no test framework or numpy at module level: a
fork imports this from its tests only.
"""
from __future__ import annotations
import ast
import json
import os
import subprocess
import sys
import time
from collections.abc import Iterable
from pathlib import Path
from types import ModuleType
from jetlink.openpilot.interface import Openpilot, conformance
# jetlink's own heavy half, which the owner must not reach: numpy, and urllib
# with ssl and email behind it
HEAVY_JETLINK = ('jetlink.client', 'jetlink.spec', 'jetlink.queues', 'jetlink.registry')
# -- the adapter module ----------------------------------------------------------
_PROBE = '''
import importlib, json, sys
before = set(sys.modules)
module = importlib.import_module(sys.argv[1])
found = {'import': sorted(set(sys.modules) - before)}
before = set(sys.modules)
try:
config = module.owner_config()
found['owner_config'] = sorted(set(sys.modules) - before)
from jetlink.openpilot.interface import OwnerConfig
found['config'] = isinstance(config, OwnerConfig)
except Exception as e:
found['error'] = f'{type(e).__name__}: {e}'
print(json.dumps(found))
'''
def _foreign(names, own: str, heavy, allowed=()) -> list[str]:
"""What is neither the standard library nor jetlink's light half, nor of
the fork's package `own`, nor `allowed`: by the outermost module of each."""
found = {n for n in names if n not in allowed and (n.startswith((*heavy, *HEAVY_JETLINK)) or
n.split('.')[0] not in (*sys.stdlib_module_names, 'jetlink', own))}
return sorted(n for n in found if not any(n.startswith(f + '.') for f in found))
def check_adapter(module: ModuleType, *, heavy: tuple[str, ...] = ()) -> list[str]:
"""The adapter module's problems. Its top level imports the standard
library alone, as manager and the resident owner import it. owner_config()
may import more: anything of the fork's own package that pulls in no third
party, so 'openpilot.common.params' passes unless it is named in `heavy`. A
fork should name its heavy modules there (params, swaglog, cereal). Both
in a fresh interpreter on this one's path, so a module this process has
loaded already counts."""
name = module.__name__
problems = [f"{name}.{member}(): missing" for member in ('adapter', 'owner_config')
if not callable(getattr(module, member, None))]
if problems:
return problems
# on this interpreter's path, as a fork's CI has set it up
try:
run = subprocess.run([sys.executable, '-c', _PROBE, name], capture_output=True, text=True, timeout=120,
env={**os.environ, 'PYTHONPATH': os.pathsep.join(p for p in sys.path if p)})
# the probe's line is the last: the module may print as it imports
found = json.loads(run.stdout.strip().splitlines()[-1])
except subprocess.TimeoutExpired:
return [f"{name} did not import and build its owner_config() in a fresh interpreter within 120 s"]
except (IndexError, ValueError):
return [f"{name} does not import in a fresh interpreter: {run.stderr.strip()[-500:]}"]
# its own packages' __init__s, which run first
if top := _foreign(found['import'], '', heavy, {name.rsplit('.', i)[0] for i in range(name.count('.') + 1)}):
problems.append(f"{name}'s top level imports {top}: manager and the resident owner import it")
if 'error' in found:
problems.append(f"owner_config() raised {found['error']}")
elif owner := _foreign(found['owner_config'], name.split('.')[0], heavy):
problems.append(f"owner_config() imports {owner}: the resident owner calls it")
if found.get('config') is False:
problems.append("owner_config(): not an OwnerConfig")
try:
op = module.adapter()
except Exception as e:
return [*problems, f"adapter() raised {type(e).__name__}: {e}"]
problems += [f"adapter: {p}" for p in conformance(op, Openpilot)]
if not callable(getattr(op, 'in_control', None)):
return problems
try:
op.in_control() # the first may set up what it reads
start = time.monotonic()
answer, took = op.in_control(), time.monotonic() - start
except Exception as e:
return [*problems, f"in_control() raised {type(e).__name__}: {e}"]
if not isinstance(answer, bool):
problems.append(f"in_control() answered {answer!r}, not a bool")
if took > 0.05:
problems.append(f"in_control() took {took * 1e3:.0f} ms: modeld asks before every frame")
return problems
# -- the modeld seam ---------------------------------------------------------------
def check_seam(modeld_path: str | Path) -> list[str]:
"""The problems of a modeld's main() with the hooks, read off its source
(importing a modeld costs tinygrad and a vision stream). The hooks are
found as the receiver of attach(model, small, cam_w, cam_h): the fork's
module (jetlink_adapter.attach) or its Hooks (HOOKS.attach) alike. Calls
are ordered as the source reads, so one in a branch that never runs (an
`if False:`) counts as made. state() is the fork's to publish where it has
a message for it, so it is not asked for."""
path = Path(modeld_path)
main = next((n for n in ast.parse(path.read_text(), filename=str(path)).body
if isinstance(n, ast.FunctionDef) and n.name == 'main'), None)
if main is None:
return [f"{path.name}: no main()"]
def at(n: ast.AST) -> tuple[int, int]:
return n.lineno, n.col_offset
nodes = list(ast.walk(main))
calls = sorted((n for n in nodes if isinstance(n, ast.Call)), key=at)
def called(name: str, facade: str | None = None) -> list[tuple[int, int]]:
return [at(c) for c in calls if (c.func.attr if isinstance(c.func, ast.Attribute) else getattr(c.func, 'id', None)) == name
and (facade is None or isinstance(c.func, ast.Attribute) and ast.unparse(c.func.value) == facade)]
def stored(target: str) -> list[tuple[int, int]]:
return [at(n) for n in nodes if isinstance(getattr(n, 'ctx', None), ast.Store) and ast.unparse(n) == target]
attach = next((c for c in calls if isinstance(c.func, ast.Attribute) and c.func.attr == 'attach'
and len(c.args) + len(c.keywords) == 4), None)
if attach is None:
return [f"{path.name}: main() never calls attach(model, small, cam_w, cam_h)"]
facade = ast.unparse(attach.func.value)
problems, prepare, realtime = [], called('prepare', facade), called('config_realtime_process')
if not prepare:
problems.append(f"main() never calls {facade}.prepare()")
elif realtime and prepare[0] >= realtime[0]:
problems.append("prepare() comes after config_realtime_process(): the GPU's thread would inherit the realtime core")
elif prepare[0] >= at(attach):
problems.append("attach() comes before prepare(), so the link never joins")
if not realtime:
problems.append("main() never calls config_realtime_process()")
small = attach.args[1] if len(attach.args) > 1 else next((k.value for k in attach.keywords if k.arg == 'small'), None)
if isinstance(small, ast.Name) and not any(p < at(attach) for p in stored(small.id)):
problems.append(f"attach() comes before {small.id} is built")
# the name its result is bound to, which the frame loop must run
bound = next((t.id for n in nodes if isinstance(n, (ast.Assign, ast.AnnAssign)) and n.value is attach
for t in getattr(n, 'targets', [getattr(n, 'target', None)]) if isinstance(t, ast.Name)), None)
if bound is None:
problems.append("attach()'s result is dropped: modeld would never run the joined model")
elif not any(p > at(attach) for p in called('run', bound)):
problems.append(f"attach()'s result, {bound}, is never run: modeld would keep driving the model it had")
elif not any(p > at(attach) for p in stored(f'{bound}.frame_drop_ratio')):
problems.append(f"main() never writes {bound}.frame_drop_ratio: a host that drops camera frames would never hand back")
if not called('handed_over', facade):
problems.append(f"main() never calls {facade}.handed_over(model)")
return [f"{path.name}: {p}" for p in problems]
# -- the fork's fallback ------------------------------------------------------------------
_MODEL = object()
# every hook a fork's lines may call, and the arguments each is checked with:
# the one list of them (jetlink's tests hold Hooks to it)
HOOK_CALLS = {'should_run': [(False, None, None)], 'status': [()], 'reason': [()], 'prepare': [(False,), (True,)],
'attach': [(_MODEL, object(), 1928, 1208)], 'handed_over': [(_MODEL,)], 'state': [(_MODEL,)],
'shutdown_step': [(False,), (True, 'test')], 'request_shutdown': [('test',)], 'shutdown_pending': [()],
'model_state': [('f' * 40,)], 'should_extend_catalog': [()], 'extend_catalog': [({'bundles': []},)],
'runtime': [()], 'swap_events': [(0.01,)]}
def check_fallback(fallback, hooks: Iterable[str] | None = None, *, link_on=None) -> list[str]:
"""The problems of the fork's own hooks for a jetlink that is missing or
too old to have Off: each of `hooks`, the ones the fork's lines call (by
default every hook it has, so pass them to have a missing one named),
there and answering as Off does, the link off. reason() is the fork's to
word, a str or None, and None while `link_on()`, the fork's read of the
setting, says off: nobody who left the link off is nagged. swap_events()
may be None where the fork's lines check for it. link_on() reads the
params store, so the test sets PARAMS_ROOT or OPENPILOT_PREFIX first, or
it reads the machine's own."""
from jetlink.openpilot.events import Flags
from jetlink.openpilot.hooks import Off
def flags(tracker):
found = tracker.update(True, False, True, state='ready')
return tuple(bool(getattr(found, f)) for f in Flags._fields), bool(tracker.settling)
problems = []
for name in hooks if hooks is not None else [n for n in HOOK_CALLS if hasattr(fallback, n)]:
if name not in HOOK_CALLS or not callable(getattr(fallback, name, None)):
problems.append(f"{name}(): missing" if name in HOOK_CALLS else f"{name}(): not a hook")
continue
for args in HOOK_CALLS[name]:
try:
got, want = getattr(fallback, name)(*args), getattr(Off(), name)(*args)
if name == 'swap_events':
got, want = (want, want) if got is None else (flags(got), flags(want))
except Exception as e:
problems.append(f"{name}() raised {type(e).__name__}: {e}")
continue
if name == 'reason':
try:
off = link_on is not None and not link_on()
except Exception as e:
problems.append(f"link_on() raised {type(e).__name__}: {e}")
continue
same = got is None or (isinstance(got, str) and not off)
else:
same = got is want if any(want is a for a in args) else (type(got), got) == (type(want), want)
if not same:
problems.append(f"{name}() answered {got!r}, where the link off is {want!r}")
return problems
# -- the guard ------------------------------------------------------------------------
# the machine's own params store, jetlink's records, sysfs, FunctionFS and a
# PC's openpilot params
FORBIDDEN = ('/data/', '/dev/shm/', '/dev/ffs', '/sys/', str(Path.home() / '.comma') + os.sep)
# read by a library on import, never jetlink's own reach: tinygrad's CPU quota
ALLOWED = ('/sys/fs/cgroup/cpu.max',)
# the audit events Guard reads a path from, and the argument it is
_PATH_ARG = {'open': 0, 'os.listdir': 0, 'os.scandir': 0, 'socket.connect': 1}
class Guard:
"""Records, between start() and stop(), every open, listing, socket connect
or root-script run that reaches a forbidden path. Recorded, not refused:
many of the paths are read inside `except OSError`, so a refusal would only
hide the reach. A test that reaches the live params store can clear a
real setting such as JetlinkSpec."""
def __init__(self, forbidden: tuple[str, ...] = FORBIDDEN):
self.forbidden = tuple(forbidden)
self.touched: list[str] | None = None
def start(self) -> None:
self.touched = []
def stop(self) -> list[str]:
"""What was reached since start(), once each."""
touched, self.touched = self.touched or [], None
return sorted(set(touched))
def __call__(self, event: str, args) -> None:
touched = self.touched
if touched is None:
return
if event == 'subprocess.Popen':
# the root script is sudo on a comma
argv = args[1] if len(args) > 1 and isinstance(args[1], (list, tuple)) else []
if (hit := next((str(a) for a in argv if 'jetlink-root.sh' in str(a) or str(a) == 'sudo'), None)) is not None:
touched.append(f"{event} {hit}")
return
index = _PATH_ARG.get(event)
value = args[index] if index is not None and len(args) > index else None
if isinstance(value, (str, bytes, os.PathLike)):
path = os.path.abspath(os.fsdecode(value))
if (path + os.sep).startswith(self.forbidden) and path not in ALLOWED:
touched.append(f"{event} {path}")
def install_guard(forbidden: tuple[str, ...] = FORBIDDEN) -> Guard:
"""A Guard, installed as this process's audit hook (for good: Python has no
removing one). Install once, in a conftest, and start and stop it around
each test."""
guard = Guard(forbidden)
sys.addaudithook(guard)
return guard
# -- fakes ---------------------------------------------------------------------------
class FakeJoinedModel:
"""What attach() returns, for a fork's modeld tests: drives as `small`;
hand_over() is a swap or a hand-back."""
def __init__(self, small=None, *, state: str = 'joining', chestnut: bool = False, constants=None):
self.small = small
self.handovers = 0
self.big_model_state = state
self.chestnut = chestnut
self.constants = constants
self.LAT_SMOOTH_SECONDS = self.LONG_SMOOTH_SECONDS = self.frame_drop_ratio = self.lat_delay = 0.0
self.runs, self.closed = 0, False
def hand_over(self, state: str = 'running') -> None:
self.handovers += 1
self.big_model_state = state
def run(self, bufs, transforms, inputs, after_enqueue=None):
self.runs += 1
return self.small.run(bufs, transforms, inputs, after_enqueue) if self.small is not None else {}
def get_action_from_model(self, *args):
return ('action', args)
def close(self) -> None:
self.closed = True
class _Blank:
"""A message never received, as cereal's SubMaster holds one: every field
reads falsy, a nested one too."""
def __getattr__(self, name: str):
if name.startswith('__'):
raise AttributeError(name)
return self
def __bool__(self) -> bool:
return False
class FakeSubMaster:
"""cereal's SubMaster over fixed messages, for an adapter's engaged() and
in_control() tests: what Stock and StarPilot's adapter read of it. A
service given a message is seen, alive and valid; any other is none of
them until update() receives it, and reads as a blank message whose every
field is falsy. A test changes `data`, `alive` and `valid` as it goes.
update() keeps its timeouts: in_control() is on modeld's frame thread and
must never wait (all 0). In for the real one: mock.patch.object(messaging,
'SubMaster', lambda services: FakeSubMaster(services, **messages)). Unlike
cereal's: a service is seen from construction, not from its first receive;
a blank field is falsy but equals nothing (a capnp default equals 0 or
False); and there is no updated, frame, freq_ok or logMonoTime."""
def __init__(self, services: Iterable[str] = (), **messages):
self.services = list(dict.fromkeys([*services, *messages]))
self.data = dict(messages)
self.seen = {s: s in messages for s in self.services}
self.alive = dict(self.seen)
self.valid = dict(self.seen)
self.timeouts: list[float] = []
def update(self, timeout: float = 100) -> None:
self.timeouts.append(timeout)
for s in self.services:
self.seen[s] = self.seen[s] or (s in self.data and self.alive[s])
def all_alive(self, service_list=None) -> bool:
return all(self.alive[s] for s in service_list or self.services)
def all_valid(self, service_list=None) -> bool:
return all(self.valid[s] for s in service_list or self.services)
def __getitem__(self, service: str):
if service not in self.seen:
raise KeyError(service) # not subscribed, as cereal's
return self.data.get(service, _BLANK)
_BLANK = _Blank()
def _level(name: str):
def log(self, msg, *args, **kwargs):
try:
text = str(msg) % args if args else str(msg)
except (TypeError, ValueError):
text = f"{msg} {args}"
self.records.append((name, text))
return log
class RecordingLog:
"""cloudlog's face, keeping every line: (level, message with its arguments)."""
debug, info, warning, error, exception, critical = map(_level, ('debug', 'info', 'warning', 'error', 'exception', 'critical'))
def __init__(self):
self.records: list[tuple[str, str]] = []
def lines(self, level: str | None = None) -> list[str]:
return [text for lvl, text in self.records if level is None or lvl == level]
def has(self, fragment: str, level: str | None = None) -> bool:
return any(fragment in text for text in self.lines(level))
# The fork's tinygrad is not jetlink's to install. What jetlink calls of it is
# small, and these stand-ins do it with numpy; the fork's tests run the real one.
class FakeTensor:
def __init__(self, data=None, device=None, dtype=None):
import numpy as np
self.array = np.asarray(data) if data is not None else np.zeros(0)
self.device = device
def realize(self, *others):
return self
def assign(self, value):
self.array[...] = value
return self
def numpy(self):
return self.array
class FakeDevice:
DEFAULT = 'CPU'
def fake_jit(fn=None, prune=False):
"""TinyJit as a decorator or a wrapper: the function as it is."""
return (lambda f: f) if fn is None else fn
def fake_tinygrad() -> dict[str, ModuleType]:
"""sys.modules entries for what jetlink imports of tinygrad; patch them in
with mock.patch.dict(sys.modules, fake_tinygrad())."""
from types import SimpleNamespace
def module(name, **attrs):
m = ModuleType(name)
m.__dict__.update(attrs)
return m
return {
'tinygrad': module('tinygrad', Tensor=FakeTensor, TinyJit=fake_jit, Device=FakeDevice),
'tinygrad.tensor': module('tinygrad.tensor', Tensor=FakeTensor),
'tinygrad.device': module('tinygrad.device', Device=FakeDevice),
'tinygrad.helpers': module('tinygrad.helpers', PARALLEL=SimpleNamespace(value=8)),
'tinygrad.engine': module('tinygrad.engine'),
'tinygrad.engine.jit': module('tinygrad.engine.jit', TinyJit=fake_jit),
}
jetlink_repo/jetlink/openpilot/warp.py
+187 -167
View File
@@ -4,92 +4,82 @@ Copyright (c) 2026-, Zeph Leggett.
This file is part of jetlink and is licensed under the MIT License.
See the LICENSE file in the root directory for more details.
The comma-side warp JIT: how it is built, where it lives, what loads it, and
the small model's reset for a fallback.
The comma-side warp: openpilot's own, where it lives, what loads it, how the
frame loop runs it (Warp), and the small model's reset for a fallback.
The warp stays on the comma (see model_state), and upstream's fused run_model
JIT (openpilot #38684) has no warp to borrow, so comma's make_warp graph is
JIT-compiled as a build target: the fork's build runs
`python -m jetlink.openpilot.warp` once per camera it builds for. A source
build makes the one for its own camera, a prebuilt release one for every
camera it installs on. Nothing compiles one at runtime: in modeld the ~9 s
compile would hold back the first frame on every ignition, and in a
provisioning run, which only runs offroad, it was lost to ignition. A device
without one runs the small model.
The warp stays on the comma (see model_state). openpilot's modeld runs it as
a JIT of its own, driving_warp_{w}x{h}_tinygrad.pkl, one per camera, and the
link runs the same pickle: both camera frames in as one (2, frame size) uint8
input_frame, their transforms as one (2, 3, 3) M_inv, and out the
(2, 6, 128, 256) uint8 frame the link sends. The fork's adapter says where it
is. Nothing compiles one at runtime: a device without one runs the small
model.
load() is what stands between a bad pickle and the car.
tinygrad is the fork's, and only modeld and the build have it: every import of
it here is inside the function that needs it.
tinygrad is the fork's, and only modeld has it: every import of it here is
inside the function that needs it.
"""
from __future__ import annotations
import argparse
import ctypes
import pickle
import sys
from pathlib import Path
# what TinyJit records for the keyword call in call_warp: sorted(kwargs)
WARP_INPUT_NAMES = ['big_frame', 'big_tfm', 'frame', 'tfm']
def call_warp(warp, tfm, big_tfm, frame, big_frame):
"""Call a warp JIT. Every caller goes through here, capture included.
TinyJit names inputs from enumerate(args) plus sorted(kwargs) and refuses a
call whose names differ from the capture. A positional compile and a keyword
call raised JitError on the first frame of a drive.
"""
return warp(tfm=tfm, big_tfm=big_tfm, frame=frame, big_frame=big_frame)
# what the warp's TinyJit was captured with: sorted(kwargs) of modeld's call
WARP_INPUT_NAMES = ['M_inv', 'input_frame']
# KGSL allocation flags (msm_kgsl.h) for memory the GPU's accesses snoop the
# CPU's caches on, mapped write-back: KGSL_MEMFLAGS_IOCOHERENT, which
# tinygrad's kgsl bindings lack, and KGSL_CACHEMODE_WRITEBACK (3 << 26). See
# Warp
COHERENT_WRITEBACK = (1 << 31) | (3 << 26)
def init_device(log) -> None:
"""Bring the GPU up now, on the caller's thread.
"""Bring the GPU up now, on the caller's thread, and turn tinygrad's
compile pool off for the process.
tinygrad initialises the device on its first kernel run and spawns a libusb
event thread doing it. A thread created after config_realtime_process(7, 54)
inherits SCHED_FIFO 54 and the core-7 pin and preempts the frame loop; that
cost 5% of frames once. Failure is not a reason to refuse the accelerator.
inherits SCHED_FIFO 54 and the core-7 pin and preempts the frame loop,
enough to drop ~5% of frames. Failure is not a reason to refuse the accelerator.
"""
try:
# modeld never needs the pool (engine/worker.py), and its PARALLEL idle
# interpreters' handler threads would inherit FIFO 54 on core 7
from tinygrad.helpers import PARALLEL
PARALLEL.value = 0
except Exception:
log.exception("jetlink: could not turn tinygrad's compile pool off")
try:
from tinygrad.tensor import Tensor
Tensor([0.0]).realize()
except Exception:
log.exception("jetlink: could not bring the gpu up before modeld goes realtime")
# the same trap for tinygrad's compile pool (engine/worker.py): created on
# the first compile, after modeld goes realtime, its handler threads sat at
# SCHED_FIFO 54 on core 7. An older tinygrad or PARALLEL=0 is not a failure
try:
from tinygrad.engine.worker import get_worker_pool
get_worker_pool()
except Exception:
log.exception("jetlink: could not start tinygrad's compile pool before modeld goes realtime")
class Warps:
"""The warps the build made for this device, as the fork's adapter says where."""
"""The warps this device's checkout has, where the fork's adapter says."""
def __init__(self, op):
self.op = op
# nothing compiles a warp at runtime and the build runs before manager, so
# the answer holds for the life of the process; the UI asks at 5 Hz
# nothing makes a warp at runtime and the checkout's are there before
# manager, so the answer holds for the life of the process; the UI asks
# at 5 Hz
self._built: bool | None = None
def geometry(self) -> tuple[int, int, int, int]:
"""(cam_w, cam_h, model_w, model_h) for this device: the choice modeld's
build makes, so the warp built is the one modeld asks for. If they
disagree, load() raises and the drive is small-model."""
"""(cam_w, cam_h, model_w, model_h) for this device: the warp modeld
loads. If the link's model disagrees, load() raises and the drive is
small-model."""
return tuple(self.op.camera())
def path(self, cam_w: int, cam_h: int, model_w: int, model_h: int) -> Path:
return Path(self.op.warp_path(cam_w, cam_h, model_w, model_h))
def is_cached(self, cam_w: int, cam_h: int, model_w: int, model_h: int) -> bool:
"""Is there a warp for this geometry?
Presence only. Staleness is the build's job: the target depends on tinygrad
and the capture sources. A pickle from an incompatible tinygrad raises in load().
"""
"""Is there a warp for this geometry? Presence only: a pickle from an
incompatible tinygrad raises in load()."""
return self.path(cam_w, cam_h, model_w, model_h).is_file()
def built(self) -> bool:
@@ -99,155 +89,185 @@ class Warps:
self._built = self.is_cached(*self.geometry())
return self._built
def load(self, cam_w: int, cam_h: int, model_w: int, model_h: int):
"""The built warp JIT. Raises if it is not there or is stale.
def load(self, cam_w: int, cam_h: int, model_w: int, model_h: int) -> dict:
"""The warp as openpilot pickles it: {'run': its TinyJit, 'input_specs':
{name: (shape, dtype, device)}}. Raises if it is not there or is not one
the frame loop can run.
modeld's big-model load is wrapped in the fallback to the small model, and a
warp that cannot be trusted must not reach the car.
"""
if not self.is_cached(cam_w, cam_h, model_w, model_h):
raise RuntimeError(f"no warp built for {cam_w}x{cam_h} -> {model_w}x{model_h}; "
"the fork's build makes it (python -m jetlink.openpilot.warp)")
with open(self.path(cam_w, cam_h, model_w, model_h), 'rb') as f:
path = self.path(cam_w, cam_h, model_w, model_h)
if not path.is_file():
raise RuntimeError(f"no warp for {cam_w}x{cam_h} -> {model_w}x{model_h} at {path}")
with open(path, 'rb') as f:
warp = pickle.load(f)
if not isinstance(warp, dict) or not {'run', 'input_specs'} <= warp.keys():
raise RuntimeError(f"{path.name} is not a driving warp: {type(warp).__name__}")
# a JIT pickled before TinyJit captured loads fine and computes nothing; one
# captured with a different call convention raises JitError on the first
# frame of a drive. Both have happened
captured = getattr(warp, 'captured', None)
# captured under other names raises JitError on the first frame of a drive
captured = getattr(warp['run'], 'captured', None)
if captured is None:
raise RuntimeError("cached warp was pickled before it captured; it computes nothing")
raise RuntimeError("the warp was pickled before it captured; it computes nothing")
names = list(getattr(captured, 'expected_names', []))
if names != WARP_INPUT_NAMES:
raise RuntimeError(f"cached warp expects {names}, call_warp passes {WARP_INPUT_NAMES}")
raise RuntimeError(f"the warp expects {names}, the frame loop passes {WARP_INPUT_NAMES}")
specs = warp['input_specs']
frames, tfm = tuple(specs['input_frame'][0]), tuple(specs['M_inv'][0])
if len(frames) != 2 or frames[0] != 2 or tfm != (2, 3, 3):
raise RuntimeError(f"the warp takes frames {frames} and transforms {tfm}, not two of each")
made = tuple(getattr(captured.ret, 'shape', ()))
if made != (2, 6, model_h // 2, model_w // 2):
raise RuntimeError(f"the warp makes {made}, not a {model_w}x{model_h} model's input")
return warp
def warm(warp, frame_size: int) -> None:
"""Run a loaded warp JIT until it is cheap to call. `frame_size` is the
camera's NV12 buffer size (ModelFace.frame_size).
class Warp:
"""The warp as modeld's frame loop runs it: start() with two camera buffers
and their transforms, wait(), and `output` holds the warped frame, where the
link sends it from.
Measured: loading 0.3 s, the first call 1.9 s, the second 5 ms. Paid on
modeld's frame loop that was ~26 dropped frames and 16 s of modeldLagging
after every join.
openpilot's warp takes both frames as one buffer, so start() copies the
camera buffers in, as modeld does, and the transforms beside them. Two
things the JIT alone does not do, measured on the comma:
- Its output lives in GPU memory the CPU reads through its cache. tinygrad
maps a QCOM buffer write-combined, which the CPU reads uncached: copying
the 393 KB out takes 2.9 ms, the kernel's copy straight from that mapping
7.3 ms. The Adreno 630 is IO-coherent, so KGSL memory flagged IOCOHERENT
and write-back is right to read once the GPU is done: 0.22 ms for the
kernel's copy, the GPU's own time unchanged. It moves before the first
call, which links the JIT to its buffers' addresses.
- TinyJit prepares and checks every call's inputs, a graph rewrite per
input: 0.97 ms of a 1.93 ms call. Every frame's inputs are the same two
buffers, which start() writes into, so they are checked once, by the
warm-up's calls through TinyJit, and a frame replays the capture.
The warm-up, the link of the JIT and its first runs, is paid here rather
than on modeld's frame loop.
"""
import numpy as np
def __init__(self, warp: dict, frame_size: int):
import numpy as np
from tinygrad.device import Device
from tinygrad.tensor import Tensor
jit, specs = warp['run'], warp['input_specs']
(frames_shape, _, _), (tfm_shape, _, _) = specs['input_frame'], specs['M_inv']
# what it reads of each camera buffer: modeld's frame_copy_size, short of
# the buffer's end
self._frame_size = frames_shape[1]
if self._frame_size > frame_size:
raise RuntimeError(f"the warp reads {self._frame_size} bytes a frame, past a {frame_size} byte camera buffer")
out = jit.captured.ret.uop.base.buffer
self._device = out.device
if self._device.startswith('QCOM'):
_make_coherent(out)
# the two buffers every frame passes, written in place by start()
self._tensors = {name: Tensor(np.zeros(shape, dtype=dtype), device=device).realize()
for name, (shape, dtype, device) in specs.items()}
for _ in range(2):
jit(**self._tensors)
self.wait = Device[self._device].synchronize
self.wait()
self.output = out.as_memoryview(allow_zero_copy=True)
self._frames = np.frombuffer(self._view('input_frame'), dtype=np.uint8).reshape(frames_shape)
self._at = [self._frames[i].ctypes.data for i in range(2)]
self._tfm = np.frombuffer(self._view('M_inv'), dtype=np.float32).reshape(tfm_shape)
self._replay = jit.captured
self._inputs = [self._tensors[name].uop.base for name in WARP_INPUT_NAMES]
def _view(self, name: str) -> memoryview:
"""The CPU's view of an input buffer, not a copy of it."""
return self._tensors[name].uop.base.buffer.as_memoryview(allow_zero_copy=True)
def start(self, frame: int, big_frame: int, tfm, big_tfm) -> None:
"""Warp the camera buffers at these addresses under their transforms.
`output` is the frame once wait() returns, until the next start(). The
GPU is idle here: every start() is waited for before the next."""
ctypes.memmove(self._at[0], frame, self._frame_size)
ctypes.memmove(self._at[1], big_frame, self._frame_size)
self._tfm[0] = tfm
self._tfm[1] = big_tfm
self._replay(self._inputs, {})
def _make_coherent(out) -> None:
"""Move a QCOM buffer to KGSL memory flagged IOCOHERENT and write-back.
Raises if the kernel keeps either flag back (the allocation's record holds
its answer): write-back memory that is not coherent reads stale frames."""
from tinygrad.device import Device
from tinygrad.tensor import Tensor
frames = [np.zeros(frame_size, dtype=np.uint8) for _ in range(2)]
blobs = [Tensor.from_blob(f.ctypes.data, (frame_size,), dtype='uint8', device=Device.DEFAULT) for f in frames]
eye = [np.eye(3, dtype=np.float32) for _ in range(2)]
tfm, big_tfm = (Tensor(e, device='NPY').realize() for e in eye)
for _ in range(2):
call_warp(warp, tfm, big_tfm, blobs[0], blobs[1]).realize()
Device.default.synchronize()
dev = Device[out.device]
mem = dev._gpu_alloc(out.nbytes, flags=COHERENT_WRITEBACK)
if mem.meta[0].flags & COHERENT_WRITEBACK != COHERENT_WRITEBACK:
dev._gpu_free(mem)
raise RuntimeError(f"the GPU driver kept back IO-coherent memory (flags {mem.meta[0].flags:#x})")
out.deallocate()
out.allocate(opaque=mem)
def prepare_reset(model):
"""Capture the small model's reset before driving, keeping its JIT's buffer identities.
Its history is stale after the Jetson ran, so a fallback starts from the same
zero history as modeld startup. Nothing is allocated or compiled on the
failure frame.
The small model is whatever bundle the user picked, stock modeld's or a
modeld_v2 one, so the history is whatever GPU queues it has; the NPY tensors
are zeroed through their numpy views. Duck-typed on openpilot's ModelState:
input_queues, numpy_inputs (modeld_v2) or npy (modeld), prev_desire. The
fork's tests pin those names.
"""
def reset_queues(queues, arrays, prev_desire=None):
"""A small model's reset, captured now: `queues`, its tensors on the GPU,
zeroed by one TinyJit, then `prev_desire` and each numpy array in `arrays`
in place. Every buffer keeps its identity, which the model's own JIT is
linked to, and the reset allocates and compiles nothing, so it may run on
the frame thread. What an adapter's small_reset returns."""
from tinygrad import Tensor, TinyJit
queues = tuple(q for q in model.input_queues.values() if q.device != 'NPY')
npy = model.numpy_inputs if hasattr(model, 'numpy_inputs') else model.npy
queues, arrays = tuple(queues), tuple(arrays)
@TinyJit
def clear():
Tensor.realize(*(q.assign(0) for q in queues))
for _ in range(3):
clear()
# TinyJit captures on its second call, and one with nothing to run raises
# there. Its kernels compile here, after init_device() turned the compile
# pool off
if queues:
for _ in range(3):
clear()
def reset():
clear()
model.prev_desire.fill(0)
for array in npy.values():
if queues:
clear()
if prev_desire is not None:
prev_desire.fill(0)
for array in arrays:
array.fill(0)
return reset
# -- the build ------------------------------------------------------------------
def reset_for_stock(state, prev_desire=None):
"""The reset of stock modeld's ModelState layout, captured now, as its warmup
zeroes it: the recurrent state on the GPU (input_queues named by
state_pairs) and the one packed host buffer (packed_input), and the desire
edge. `state` is the ModelState, or a modeld_v2 native bundle's adapter,
which keeps the same layout."""
return reset_queues([state.input_queues[name] for name in state.state_pairs], (state.packed_input,), prev_desire)
def compile_warp(graph, frame_size: int, out: Path) -> Path:
"""JIT the warp graph and pickle it to `out`. Holds the GPU while it runs.
Three runs before pickling: TinyJit captures on the second call, and a
pickle taken earlier is an empty jit that silently does nothing.
def prepare_reset(model):
"""Capture the small model's reset before driving, keeping its JIT's buffer identities.
Its history is stale after the Jetson ran, so a fallback starts from the
zero history modeld's warmup leaves it at. Nothing is allocated or compiled
on the failure frame, so never through modeld_v2's reset_warmup_buffers(),
which compiles on it. jetlink's own, for an adapter without small_reset
(interface.ModelSide).
The small model is whatever the fork's modeld runs, and each keeps its
history its own way. Stock modeld's ModelState, and a modeld_v2 native
bundle's adapter: the recurrent state on the GPU (state_pairs) and one
packed host buffer (packed_input), as their warmup zeroes them. A modeld_v2
legacy bundle's adapter: history queues on the GPU, and numpy inputs. Any
other ModelState with input_queues (StarPilot's): every queue on the GPU,
and its numpy inputs (numpy_inputs, or npy); its own _reset_state() makes
new tensors, which its JIT is not linked to. The fork's tests pin those names.
"""
import numpy as np
from tinygrad.device import Device
from tinygrad.engine.jit import TinyJit
from tinygrad.tensor import Tensor
warp_jit = TinyJit(graph, prune=True)
# one set of input tensors: TinyJit captures against the buffers it is
# first handed. Random so nothing constant-folds
rng = np.random.default_rng(42)
tfm_npy, big_tfm_npy = np.eye(3, dtype=np.float32), np.eye(3, dtype=np.float32)
tfm = Tensor(tfm_npy, device='NPY')
big_tfm = Tensor(big_tfm_npy, device='NPY')
frame = Tensor.randint(frame_size, low=0, high=256, dtype='uint8', device=Device.DEFAULT).realize()
big_frame = Tensor.randint(frame_size, low=0, high=256, dtype='uint8', device=Device.DEFAULT).realize()
for _ in range(3):
tfm_npy[:] = rng.standard_normal((3, 3)).astype(np.float32)
big_tfm_npy[:] = rng.standard_normal((3, 3)).astype(np.float32)
call_warp(warp_jit, tfm, big_tfm, frame, big_frame).realize()
Device.default.synchronize()
out = Path(out)
out.parent.mkdir(parents=True, exist_ok=True)
# through a temporary: a compile killed mid-write leaves nothing under the
# name load() opens
tmp = out.with_suffix('.pkl.tmp')
with open(tmp, 'wb') as f:
pickle.dump(warp_jit, f)
tmp.replace(out)
return out
def size(text: str) -> tuple[int, int]:
"""'WxH' as (w, h)."""
w, sep, h = text.lower().partition('x')
if not sep:
raise argparse.ArgumentTypeError(f"expected WxH, not {text!r}")
try:
return int(w), int(h)
except ValueError:
raise argparse.ArgumentTypeError(f"expected WxH, not {text!r}") from None
def main(argv: list[str] | None = None) -> None:
"""Build the warp for one camera, as the fork's build runs it."""
from jetlink.openpilot.interface import load_adapter
p = argparse.ArgumentParser(prog='python -m jetlink.openpilot.warp', description=main.__doc__)
p.add_argument('--adapter', required=True, help="the fork's adapter module")
p.add_argument('--camera', type=size, required=True, help='camera resolution, WxH')
p.add_argument('--model', type=size, required=True, help='model input, WxH')
p.add_argument('--output', type=Path, required=True)
args = p.parse_args(argv)
(cam_w, cam_h), (model_w, model_h) = args.camera, args.model
op = load_adapter(args.adapter)
print(f"Compiling jetlink warp for {cam_w}x{cam_h} -> {model_w}x{model_h}...")
# before anything of tinygrad's is imported here: comma's graph module
# patches tinygrad's firmware fetch as it loads
graph, frame_size = op.make_warp(cam_w, cam_h, model_w, model_h)
out = compile_warp(graph, frame_size, args.output)
print(f" Saved to {out}")
if __name__ == "__main__":
sys.exit(main())
adapter = getattr(model, 'adapter', None)
keeper = model if adapter is None else adapter
prev_desire = getattr(model, 'prev_desire', None)
if adapter.is_native if adapter is not None else hasattr(model, 'state_pairs'):
return reset_for_stock(keeper, prev_desire)
queues = [q for q in keeper.input_queues.values() if q.device != 'NPY']
arrays = (keeper.numpy_inputs if hasattr(keeper, 'numpy_inputs') else keeper.npy).values()
return reset_queues(queues, arrays, prev_desire)
jetlink_repo/jetlink/protocol.py
+56 -1
View File
@@ -15,6 +15,7 @@ vectored write out, one read into a preallocated buffer back.
"""
from __future__ import annotations
import functools
import struct
from enum import IntEnum
@@ -42,7 +43,7 @@ GADGET_TX_ALIGN = 16 * 1024
# The comma's gadget as a USB host finds it: these IDs (pid.codes' test
# allocation; get a real PID before distributing this), which
# scripts/comma/jetlink-root.sh presents, then the link's interface by its class
# jetlink/comma/jetlink-root.sh presents, then the link's interface by its class
# triple (transport/ffs.py), because the comma set to iOS presents a composite
# gadget and a reordered config could move the number. Endpoint addresses are
# read from the descriptors, never assumed: FunctionFS renumbers them at bind.
@@ -114,6 +115,54 @@ INFER_RESP_SIZE = struct.calcsize(INFER_RESP_FMT)
_infer_resp = struct.Struct(INFER_RESP_FMT)
# A frame over the phone's cable as UDP datagrams (TcpTransport.use_datagrams):
# the INFER_REQ's stream bytes, header and pad included, cut into equal pieces
# of at most DATAGRAM_PAYLOAD, each behind a DATAGRAM_HEADER. The comma's kernel
# cuts a datagram into IP fragments after netfilter and routing, so the stack's
# per-packet work runs once a datagram, about 7 times a frame, where TCP runs it
# for each of ~272 segments: measured 7.35 vs 4.3 ms a frame.
# The server offers it in HELLO_RESP (frame_port, frame_token) on the cable only.
# A datagram with another token is from an earlier session and is dropped.
DATAGRAM_MAGIC = 0x4D52464A # b'JFRM'
# magic, token, seq, offset, total: where this piece goes in a message of `total` bytes
DATAGRAM_HEADER_FMT = '<IIIII'
DATAGRAM_HEADER_SIZE = struct.calcsize(DATAGRAM_HEADER_FMT)
# UDP carries 65507 bytes at most; equal pieces of up to this, 7 for a big model's frame
DATAGRAM_PAYLOAD = 65000
_datagram_header = struct.Struct(DATAGRAM_HEADER_FMT)
def pack_datagram_header(token: int, seq: int, offset: int, total: int) -> bytes:
return _datagram_header.pack(DATAGRAM_MAGIC, token, seq, offset, total)
def pack_datagram_header_into(buf, token: int, seq: int, offset: int, total: int) -> None:
_datagram_header.pack_into(buf, 0, DATAGRAM_MAGIC, token, seq, offset, total)
def unpack_datagram_header(buf) -> tuple[int, int, int, int]:
"""(token, seq, offset, total); ProtocolError for anything not ours."""
magic, token, seq, offset, total = _datagram_header.unpack_from(buf)
if magic != DATAGRAM_MAGIC:
raise ProtocolError(f"bad datagram magic 0x{magic:08x}")
return token, seq, offset, total
def datagram_pieces(total: int) -> tuple[tuple[int, int], ...]:
"""(offset, size) of each datagram for a message of `total` bytes: as few as
DATAGRAM_PAYLOAD allows, equal to within a byte. The same every frame of a
model, so kept."""
return _pieces(total, DATAGRAM_PAYLOAD)
@functools.lru_cache(maxsize=8)
def _pieces(total: int, payload: int) -> tuple[tuple[int, int], ...]:
n = max(1, -(-total // payload))
base, extra = divmod(total, n)
sizes = [base + (1 if i < extra else 0) for i in range(n)]
return tuple((sum(sizes[:i]), size) for i, size in enumerate(sizes))
class ProtocolError(RuntimeError):
pass
@@ -153,3 +202,9 @@ class Status(IntEnum):
BAD_SHAPE = 2
INFER_FAILED = 3
NOT_FINITE = 4 # model produced NaN/Inf; caller must fall back
class NotFinite(ValueError):
"""A frame's output with a NaN or an inf in it, by the server's word
(Status.NOT_FINITE) or the comma's own check: the host answered, with
nothing to drive on, and the link is as good as it was. Not a LinkError."""
jetlink_repo/jetlink/queues.py
+3 -4
View File
@@ -11,8 +11,8 @@ openpilot folds these into the tinygrad JIT on the GPU. Shipping the history
across the link would cost ~10 MB a frame, so the queues live on the server and
the comma sends only the newest warped frame and the packed scalars. The hidden
state the graph returns stays there too: the next frame pushes it into the
features queue, where modeld's prev_feat went. The fork's tests/test_queues.py
checks this against the tinygrad original, and the staging conformance fixture
features queue, where modeld's prev_feat went. tests/test_queues.py holds this
to openpilot's tinygrad original, and the staging conformance fixture
(JetlinkKit/Scripts/make_conformance_fixtures.py) holds the Swift server's
queues to this, bit for bit.
@@ -59,12 +59,11 @@ class PolicyQueues:
did. The queues stay until the client resets them."""
self.prev_feat[...] = 0
def after_run(self, outputs: dict[str, np.ndarray], dest=None) -> None:
def after_run(self, outputs: dict[str, np.ndarray]) -> None:
"""Keep this frame's hidden state for the next one.
Only after a frame whose outputs are all finite: modeld fed back only
what infer_end returned, and it raises on NOT_FINITE and on a failed run.
`dest` is unused; the fork's test_queues still passes it.
"""
self.prev_feat.reshape(-1)[...] = np.asarray(outputs[DRIVING_OUTPUT], np.float32).reshape(-1)[self._hidden]
jetlink_repo/jetlink/registry/catalog.py
+25 -3
View File
@@ -33,6 +33,11 @@ CATALOG_URL = CATALOG_URL_TEMPLATE.format(version=CATALOG_VERSION)
# the first that is not there, so a model published later is listed without a
# release of this package. See merge_catalogs.
PROBE_LIMIT = 10
# zoompilot's own big models: ones sunnypilot does not list (yet), such as
# previews from comma's open pull requests, in the catalog's shape. Merged
# after sunnypilot's catalogs, so their entry for the same commit wins. An
# edit on main reaches every comma within the hour, with no release.
EXTRA_CATALOG_URL = 'https://raw.githubusercontent.com/zoompilot/jetlink/refs/heads/main/catalog/extra_big_models.json'
# The selector version the fork requires (REQUIRED_JSON_VERSION on the comma).
# It is a string in the JSON; bundles at any other version describe fields we
# would misread.
@@ -137,8 +142,9 @@ def fetch_catalog(url: str = CATALOG_URL, timeout: float = CATALOG_TIMEOUT, open
def fetch_catalogs(opener=None) -> dict:
"""The pinned catalog merged with every one sunnypilot has published since,
whose commits the server runs too. The probe stops at the first version that
is not there. Any other failure raises: a short list would drop a model
whose commits the server runs too, and then zoompilot's extra models. The
probe stops at the first version that is not there, and a missing extra list
adds nothing. Any other failure raises: a short list would drop a model
found only past it from whatever the caller caches, so the caller keeps its
last complete one instead."""
found = [fetch_catalog(CATALOG_URL, opener=opener)]
@@ -147,7 +153,23 @@ def fetch_catalogs(opener=None) -> dict:
found.append(fetch_catalog(CATALOG_URL_TEMPLATE.format(version=v), opener=opener))
except NotFound:
break
return merge_catalogs(found)
try:
extra = fetch_catalog(EXTRA_CATALOG_URL, opener=opener)
except NotFound:
extra = {}
merged, pins = merge_catalogs([*found, extra]), pinned_pointers(extra)
return {**merged, 'pointers': pins} if pins else merged
def pinned_pointers(data: dict) -> dict[str, dict]:
"""The extra list's well-formed pins, {ref: {'oid', 'size'}}: the ONNX jetlink
runs for that commit whatever a lookup said. See catalog/README.md."""
pins = data.get('pointers')
if not isinstance(pins, dict):
return {}
return {ref: {'oid': p['oid'], 'size': p['size']} for ref, p in pins.items()
if is_ref(ref) and isinstance(p, dict) and is_sha256(p.get('oid'))
and type(p.get('size')) is int and p['size'] > 0}
def merge_catalogs(catalogs: list[dict], selector: int = REQUIRED_SELECTOR_VERSION) -> dict:
jetlink_repo/jetlink/registry/lfs.py
+172 -42
View File
@@ -13,13 +13,19 @@ ask a jetlink server for. The bytes themselves are on comma's LFS servers,
which is GitLab and not GitHub: each is asked in turn because which one has an
object varies with the model's age.
Since openpilot #38930 (2026-09-16) a commit can ship a precompiled tinygrad
Since openpilot #38930 a commit can ship a precompiled tinygrad
pkl instead, with no ONNX in the tree at all; Cinque Terre V3 is one. Its
subject names the export ("Use f78ed37d for the precompiled eGPU driving
model") and the ONNX is in comma's HuggingFace model repo, in the folder that
id starts. sunnypilot's model builds find it the same way. That repo speaks
the LFS batch protocol too, so it is one more endpoint to ask.
Not every such commit names its export. ResAction's pull request (#39037)
adds the ONNX in one commit and replaces it with the pkl in the
next, subject "compiled", and a squash merge's subject only names the pull
request. The pointer is still in a diff: the one the commit's own patch
deletes, or the last one its pull request's patch carries.
The zoompilot fork downloads through this module too, asking its own
.lfsconfig endpoint first. No openpilot imports.
"""
@@ -44,6 +50,14 @@ BIG_ONNX = 'big_driving_supercombo.onnx'
POINTER_URL = 'https://raw.githubusercontent.com/commaai/openpilot/{ref}/openpilot/selfdrive/modeld/models/' + BIG_ONNX
# a commit's subject without the API, whose anonymous limit a car behind CGNAT shares
COMMIT_PATCH_URL = 'https://github.com/commaai/openpilot/commit/{ref}.patch'
# every commit of a pull request, one patch after another, for a squash merge's subject
PULL_PATCH_URL = 'https://github.com/commaai/openpilot/pull/{number}.patch'
# how much of a patch is read for the pointers in its diffs; a model commit's is 3 KB
PATCH_MAX = 256 << 10
# the diff header of the big ONNX, wherever the tree keeps selfdrive
_ONNX_DIFF = re.compile(r'^diff --git a/\S*selfdrive/modeld/models/' + re.escape(BIG_ONNX) + r' ')
# a squash merge's subject ends with its pull request
_PULL = re.compile(r'\(#([0-9]+)\)$')
DRIVING_MODELS_REPO = 'commaai/openpilot_driving_models'
DRIVING_MODELS_TREE_URL = f'https://huggingface.co/api/models/{DRIVING_MODELS_REPO}/tree/main'
LFS_ENDPOINTS = (
@@ -53,6 +67,9 @@ LFS_ENDPOINTS = (
)
# an export folder is a uuid; the subject names its first eight
_EXPORT_ID = re.compile(r'\b[0-9a-f]{8}\b')
# how comma's model commits name an export: the whole uuid, or its first eight
# and the checkpoint ('8d736683/12864'). A bare eight could be any hex word
_NAMED_EXPORT = r'\b{}[-/]'
LFS_MEDIA_TYPE = 'application/vnd.git-lfs+json'
POINTER_TIMEOUT = 10.0
CONNECT_TIMEOUT = 30.0
@@ -95,7 +112,7 @@ def parse_pointer_text(text: str) -> Pointer | None:
def fetch_pointer(ref: str, timeout: float = POINTER_TIMEOUT, opener=None) -> Pointer:
"""The oid and size of the ONNX at a comma commit: in its tree, or for a
commit that ships a precompiled pkl instead, the export its subject names."""
commit that ships a precompiled pkl instead, the ONNX it was built from."""
try:
text = http_get(POINTER_URL.format(ref=ref), timeout, opener, POINTER_MAX).decode('utf-8', 'replace')
except NotFound:
@@ -106,10 +123,11 @@ def fetch_pointer(ref: str, timeout: float = POINTER_TIMEOUT, opener=None) -> Po
return pointer
def commit_subject(ref: str, timeout: float = POINTER_TIMEOUT, opener=None) -> str:
"""A comma commit's subject line, from the head of its patch."""
url = COMMIT_PATCH_URL.format(ref=ref)
lines = http_get(url, timeout, opener, POINTER_MAX).decode('utf-8', 'replace').splitlines()
def patch_subjects(text: str) -> list[str]:
"""Every commit's subject in a patch, in order: one for a commit's, one per
commit for a pull request's."""
lines = text.splitlines()
subjects = []
for i, line in enumerate(lines):
if line.startswith('Subject:'):
subject = [line.removeprefix('Subject:').strip()]
@@ -118,8 +136,42 @@ def commit_subject(ref: str, timeout: float = POINTER_TIMEOUT, opener=None) -> s
if not cont[:1].isspace() or not cont.strip():
break
subject.append(cont.strip())
return re.sub(r'^\[PATCH[^\]]*\]\s*', '', ' '.join(subject))
raise RegistryError(f"{url} has no subject line")
subjects.append(re.sub(r'^\[PATCH[^\]]*\]\s*', '', ' '.join(subject)))
return subjects
def diff_pointer(text: str) -> Pointer | None:
"""The big ONNX's pointer in a patch's diffs, the last one wins: the pointer a
diff gives the file, or for a diff that deletes it, the pointer it had. A
commit that swaps the ONNX for a pkl deletes the ONNX the pkl was built from."""
found = None
section: list[str] | None = None
def close():
nonlocal found
if section is not None:
added = parse_pointer_text('\n'.join(x[1:] for x in section if x.startswith('+') and not x.startswith('+++')))
removed = parse_pointer_text('\n'.join(x[1:] for x in section if x.startswith('-') and not x.startswith('---')))
found = added or removed or found
for line in text.splitlines():
# each diff, and each commit of a pull request's patch, ends the one before
if line.startswith(('diff --git ', 'From ')):
close()
section = [] if _ONNX_DIFF.match(line) else None
elif section is not None:
section.append(line)
close()
return found
def commit_subject(ref: str, timeout: float = POINTER_TIMEOUT, opener=None) -> str:
"""A comma commit's subject line, from the head of its patch."""
url = COMMIT_PATCH_URL.format(ref=ref)
subjects = patch_subjects(http_get(url, timeout, opener, POINTER_MAX).decode('utf-8', 'replace'))
if not subjects:
raise RegistryError(f"{url} has no subject line")
return subjects[0]
def _tree(path: str, timeout: float, opener) -> list[dict]:
@@ -128,15 +180,68 @@ def _tree(path: str, timeout: float, opener) -> list[dict]:
def fetch_export_pointer(ref: str, timeout: float = POINTER_TIMEOUT, opener=None) -> Pointer:
"""The big ONNX a precompiled-pkl commit was built from, in comma's model repo."""
subject = commit_subject(ref, timeout=timeout, opener=opener)
ids = list(dict.fromkeys(_EXPORT_ID.findall(subject)))
"""The big ONNX a precompiled-pkl commit was built from. In turn: the export
its subject names, in comma's model repo; the pointer its own diff deletes;
and for a squash merge, the last pointer its pull request carried, or the
export one of that pull request's commits names.
The newest export named wins: a pull request's commits are in order, and
Mountain Dew's (#39047) named three before the one it shipped. One that is
not in the model repo yet stops the search rather than falling back to an
older one, a different model."""
url = COMMIT_PATCH_URL.format(ref=ref)
patch = http_get(url, timeout, opener, PATCH_MAX).decode('utf-8', 'replace')
subjects = patch_subjects(patch)
if not subjects:
raise RegistryError(f"{url} has no subject line")
subject = subjects[0]
ids = _newest_first(subject)
pointer, missing = _export_pointer(ref, ids, subject, timeout, opener)
if pointer is not None:
return pointer
tried = [f"its subject {_not_uploaded(missing)}" if missing else f"no folder in {DRIVING_MODELS_REPO} for {', '.join(ids)}" if ids
else 'its subject names no export']
if (pointer := diff_pointer(patch)) is not None:
log.info("%s deletes %s %s", ref[:10], BIG_ONNX, pointer.oid[:16])
return pointer
tried.append(f"its diff has no {BIG_ONNX}")
if (pull := _PULL.search(subject)) is not None:
number = pull.group(1)
pull_patch = http_get(PULL_PATCH_URL.format(number=number), timeout, opener, PATCH_MAX).decode('utf-8', 'replace')
if (pointer := diff_pointer(pull_patch)) is not None:
log.info("%s merged #%s, whose last %s is %s", ref[:10], number, BIG_ONNX, pointer.oid[:16])
return pointer
if missing is None: # an export the subject names and the repo lacks stops the search here too
pull_subjects = ' '.join(patch_subjects(pull_patch))
pull_ids = [i for i in _newest_first(pull_subjects) if i not in ids]
pointer, pull_missing = _export_pointer(ref, pull_ids, pull_subjects, timeout, opener)
if pointer is not None:
return pointer
tried.append(f"pull request #{number} {_not_uploaded(pull_missing)}" if pull_missing else f"nor does pull request #{number}")
raise RegistryError(f"{ref[:10]} has no {BIG_ONNX}: {'; '.join(tried)} ({subject!r})")
def _newest_first(text: str) -> list[str]:
"""The eight-hex ids `text` names, the last mention first."""
return list(dict.fromkeys(reversed(_EXPORT_ID.findall(text))))
def _not_uploaded(export: str) -> str:
return f"names export {export}, which is not in {DRIVING_MODELS_REPO} yet"
def _export_pointer(ref: str, ids: list[str], subject: str, timeout: float, opener) -> tuple[Pointer | None, str | None]:
"""The ONNX in the first export folder one of `ids` starts, and None; or
None and the first id `subject` names as an export that has no folder, where
the search stops; or None and None when no id has a folder."""
if not ids:
raise RegistryError(f"{ref[:10]} has no {BIG_ONNX} and its subject names no export: {subject!r}")
return None, None
folders = [e['path'] for e in _tree('', timeout, opener) if e.get('type') == 'directory']
for export in ids:
matches = [f for f in folders if f.startswith(export)]
if not matches:
if re.search(_NAMED_EXPORT.format(export), subject):
return None, export
continue
if len(matches) > 1:
raise RegistryError(f"{ref[:10]}: {export} starts {len(matches)} folders in {DRIVING_MODELS_REPO}")
@@ -152,8 +257,8 @@ def fetch_export_pointer(ref: str, timeout: float = POINTER_TIMEOUT, opener=None
if not is_sha256(oid) or not isinstance(size, int) or size <= 0:
raise RegistryError(f"{files[0]['path']} is not an lfs object")
log.info("%s names export %s: %s", ref[:10], export, files[0]['path'])
return Pointer(oid, size)
raise RegistryError(f"{ref[:10]}: no folder in {DRIVING_MODELS_REPO} for {', '.join(ids)}")
return Pointer(oid, size), None
return None, None
def lfs_resolve(endpoint: str, pointer: Pointer, timeout: float = CONNECT_TIMEOUT, opener=None) -> str | None:
@@ -194,43 +299,50 @@ def lfs_download(href: str, pointer: Pointer, dest: Path, progress: ProgressFn |
"""Stream to a .part file, hashing as we go, and only then take the name.
A half-written model must never sit where the next start would hand it to a
backend to build from.
backend to build from. A transfer that stalls or is stopped keeps its .part,
and the next call carries on from it with a Range request: 755 MB over a
car's connection need not arrive in one go. Only bytes that prove wrong are dropped.
"""
opener = opener or urllib.request.urlopen
dest = Path(dest)
dest.parent.mkdir(parents=True, exist_ok=True)
free = shutil.disk_usage(dest.parent).free
if free < pointer.size + FREE_SLACK:
raise RegistryError(f"need {pointer.size >> 20} MB for the model, {free >> 20} MB free")
part = dest.with_name(dest.name + '.part')
digest = hashlib.sha256()
written = 0
written = _resume_from(part, pointer.size, digest)
free = shutil.disk_usage(dest.parent).free
if free < pointer.size - written + FREE_SLACK:
raise RegistryError(f"need {(pointer.size - written) >> 20} MB more for the model, {free >> 20} MB free")
# Whole percent only: a gigabyte at 4 MB a chunk would call this a few
# hundred times and the callback may write a param or a socket line.
reported = -1
try:
with opener(href, timeout=CONNECT_TIMEOUT) as response, open(part, 'wb') as out:
while True:
if should_stop is not None and should_stop():
raise RegistryError('download cancelled')
chunk = response.read(CHUNK)
if not chunk:
break
out.write(chunk)
digest.update(chunk)
written += len(chunk)
if progress is not None and pointer.size:
percent = int(100 * written / pointer.size)
if percent != reported:
reported = percent
progress(min(1.0, written / pointer.size))
except RegistryError:
part.unlink(missing_ok=True)
raise
except Exception as e:
part.unlink(missing_ok=True)
raise NetworkError(f"could not download {pointer.oid[:16]}: {e}") from e
if written < pointer.size:
request = urllib.request.Request(href, headers={'Range': f'bytes={written}-'}) if written else href
try:
with opener(request, timeout=CONNECT_TIMEOUT) as response:
if written and response.status != 206:
# the server ignored the Range and sends the whole object
log.warning("%s: no partial content, downloading from the start", pointer.oid[:16])
digest, written = hashlib.sha256(), 0
with open(part, 'ab' if written else 'wb') as out:
while True:
if should_stop is not None and should_stop():
raise RegistryError('download stopped')
chunk = response.read(CHUNK)
if not chunk:
break
out.write(chunk)
digest.update(chunk)
written += len(chunk)
if progress is not None and pointer.size:
percent = int(100 * written / pointer.size)
if percent != reported:
reported = percent
progress(min(1.0, written / pointer.size))
except RegistryError:
raise
except Exception as e:
raise NetworkError(f"could not download {pointer.oid[:16]}: {e}") from e
if written != pointer.size:
part.unlink(missing_ok=True)
@@ -243,3 +355,21 @@ def lfs_download(href: str, pointer: Pointer, dest: Path, progress: ProgressFn |
if progress is not None:
progress(1.0)
return dest
def _resume_from(part: Path, size: int, digest) -> int:
"""How much of the model an earlier attempt left in `part`, hashed into
`digest`; 0, with the file gone, for anything that cannot be a prefix."""
try:
have = part.stat().st_size
except OSError:
return 0
if have > size:
part.unlink(missing_ok=True)
return 0
buf = bytearray(CHUNK)
view = memoryview(buf)
with open(part, 'rb') as f:
while n := f.readinto(buf):
digest.update(view[:n])
return have
jetlink_repo/jetlink/spec.py
+3 -7
View File
@@ -14,14 +14,14 @@ openpilot #38916. feat_dim is prod(fb[2:]); older forks use fb[2], which gives
32 rather than 16384 for the big model's (1,32,32,512) features_buffer.
tests/test_queues.py catches the drift.
The stateful one (openpilot #38916, 2026-09-15; Cinque Terre V3 onwards)
The stateful one (openpilot #38916; Cinque Terre V3 onwards)
carries its history in the graph: the newest warped frame goes in as new_img,
each queue goes in as state_<q> and comes back advanced as next_state_<q>, and
the hidden state never leaves the graph.
Either way the wire carries the newest frame and three scalars, and the reply
the outputs less hidden_state (protocol 3): the queued graph's server feeds its
own hidden state back, which modeld did through prev_feat until then.
own hidden state back, where an older modeld sends prev_feat.
"""
from __future__ import annotations
@@ -32,7 +32,6 @@ from dataclasses import dataclass
from functools import cached_property
from pathlib import Path
from jetlink.onnx_meta import parse_file
from jetlink.protocol import INFER_REQ_SIZE, INFER_RESP_SIZE
# openpilot ModelConstants; duplicated so the server needs no openpilot import
@@ -141,10 +140,6 @@ class ModelSpec:
def packed_nelem(self) -> int:
return sum(math.prod(s) for s in self.packed_shapes.values())
def feed_back(self, packed, output) -> None:
"""Nothing: the server feeds the hidden state back itself (protocol 3).
Kept for the modeld glue that still calls it after every frame."""
@property
def packed_nbytes(self) -> int:
return self.packed_nelem * 4 # float32
@@ -243,6 +238,7 @@ def sha256_file(path: str, bufsize: int = 1 << 20) -> tuple[str, int]:
def spec_from_onnx(path: str, frame_skip: int = DEFAULT_FRAME_SKIP,
sha256: str | None = None, nbytes: int | None = None) -> ModelSpec:
from jetlink.onnx_meta import parse_file # the scripts' and fixtures': modeld and the owner never parse a graph
meta = parse_file(path)
if sha256 is None or nbytes is None:
sha256, nbytes = sha256_file(path)
jetlink_repo/jetlink/transport/aio.py
+222
View File
@@ -0,0 +1,222 @@
"""
Copyright (c) 2026-, Zeph Leggett.
This file is part of jetlink and is licensed under the MIT License.
See the LICENSE file in the root directory for more details.
Linux native AIO (io_submit), for the gadget's endpoint writes.
A synchronous FunctionFS write holds the endpoint's one usb_request, and
epfile->mutex with it, until the host has taken every byte, so a frame sent as
32 KB writes leaves the bus idle between them: 1.4 ms a frame at 13 writes,
2.8 ms at 25, against one 512 KB write. The AIO path in
ffs_epfile_io gives each iocb its own request, copies the iocb's bytes into it
and queues it before io_submit moves on, then lets go of the mutex. dwc3 puts
every queued request on its TRB ring, so the bus streams them back to back,
and the caller's buffers are free again the moment io_submit returns.
Only what FfsTransport needs: one context, PWRITEV iocbs, and reaping. Linux
only: the comma is aarch64, and CI's x86_64 runs the real thing too.
"""
from __future__ import annotations
import ctypes
import functools
import os
import platform
import time
import numpy as np
# io_setup, io_destroy, io_submit, io_getevents
_SYSCALLS = {'aarch64': (0, 1, 2, 4), 'x86_64': (206, 207, 209, 208)}
IOCB_CMD_PWRITEV = 8
# iovecs one iocb may gather. A request is cut from the message's parts and
# padding; the most it spans is the header, the inference header, the frame,
# the packed inputs and the padding.
IOVECS = 8
class _Iocb(ctypes.Structure):
# struct iocb, little-endian (aio_key before aio_rw_flags)
_fields_ = [('data', ctypes.c_uint64), ('key', ctypes.c_uint32), ('rw_flags', ctypes.c_uint32),
('opcode', ctypes.c_uint16), ('reqprio', ctypes.c_int16), ('fildes', ctypes.c_uint32),
('buf', ctypes.c_uint64), ('nbytes', ctypes.c_uint64), ('offset', ctypes.c_int64),
('reserved2', ctypes.c_uint64), ('flags', ctypes.c_uint32), ('resfd', ctypes.c_uint32)]
class _Event(ctypes.Structure):
_fields_ = [('data', ctypes.c_uint64), ('obj', ctypes.c_uint64), ('res', ctypes.c_int64), ('res2', ctypes.c_int64)]
class _Iovec(ctypes.Structure):
_fields_ = [('base', ctypes.c_void_p), ('len', ctypes.c_size_t)]
class _Timespec(ctypes.Structure):
_fields_ = [('sec', ctypes.c_long), ('nsec', ctypes.c_long)]
class _PyBuffer(ctypes.Structure):
# CPython's Py_buffer, stable since 3.3
_fields_ = [('buf', ctypes.c_void_p), ('obj', ctypes.c_void_p), ('len', ctypes.c_ssize_t),
('itemsize', ctypes.c_ssize_t), ('readonly', ctypes.c_int), ('ndim', ctypes.c_int),
('format', ctypes.c_char_p), ('shape', ctypes.c_void_p), ('strides', ctypes.c_void_p),
('suboffsets', ctypes.c_void_p), ('internal', ctypes.c_void_p)]
_get_buffer = ctypes.pythonapi.PyObject_GetBuffer
_get_buffer.argtypes = [ctypes.py_object, ctypes.POINTER(_PyBuffer), ctypes.c_int]
_get_buffer.restype = ctypes.c_int
_release_buffer = ctypes.pythonapi.PyBuffer_Release
_release_buffer.argtypes = [ctypes.POINTER(_PyBuffer)]
_release_buffer.restype = None
def address(buf) -> tuple[int, int]:
"""Where a contiguous buffer's bytes start, and how many there are;
read-only ones (bytes) included, which ctypes.c_char.from_buffer refuses.
Valid while the caller holds `buf` and nothing resizes it."""
view = _PyBuffer()
_get_buffer(buf, ctypes.byref(view), 0) # PyBUF_SIMPLE: contiguous bytes; raises otherwise
try:
return view.buf or 0, view.len
finally:
_release_buffer(ctypes.byref(view))
class Layout:
"""A message whose spans are `lengths` bytes long, cut into requests of
`size` bytes: for each piece, the request it is in, its iovec within that
request's IOVECS, the span it gathers from and where in it, and its length.
Python loops over the pieces only here; a message of the same shape (every
frame of a link) reuses it through layout()."""
def __init__(self, lengths: tuple[int, ...], size: int):
request, slot, span, offset, length, pieces = [], [], [], [], [], []
r = filled = 0
for j, n in enumerate(lengths):
at = 0
while n:
take = min(n, size - filled)
if len(pieces) == r:
pieces.append(0)
if pieces[r] == IOVECS:
raise ValueError(f'a request gathers more than {IOVECS} buffers')
request.append(r)
slot.append(pieces[r])
span.append(j)
offset.append(at)
length.append(take)
pieces[r] += 1
at, n, filled = at + take, n - take, filled + take
if filled == size:
r, filled = r + 1, 0
self.count = len(pieces)
# where each request's pieces start, and the iovec words they fill
self.first = [0]
for c in pieces:
self.first.append(self.first[-1] + c)
self.words = 2 * (IOVECS * np.array(request, dtype=np.int64) + np.array(slot, dtype=np.int64))
self.span = np.array(span, dtype=np.int64)
self.offset = np.array(offset, dtype=np.uint64)
self.length = np.array(length, dtype=np.uint64)
self.pieces = np.array(pieces, dtype=np.uint64)
@functools.lru_cache(maxsize=8)
def layout(lengths: tuple[int, ...], size: int) -> Layout:
return Layout(lengths, size)
class Aio:
"""A kernel AIO context of `depth` iocbs writing to `fd`, used from one
thread at a time.
submit() queues requests of a Layout, gathered from the spans' addresses
(`bases`), and returns how many the kernel queued; the kernel has copied
those bytes, so the caller may reuse its buffers at once. reap() returns
each finished request's result, bytes written or a negative errno, in no
promised order. The iocb and iovec arrays are reused across submits: the
kernel copies both before io_submit returns.
"""
def __init__(self, fd: int, depth: int):
self._setup, self._destroy, self._submit, self._getevents = _SYSCALLS[platform.machine()]
self._libc = ctypes.CDLL(None, use_errno=True)
self._syscall = self._libc.syscall
self._syscall.restype = ctypes.c_long
self.depth = depth
self._ctx = ctypes.c_ulong(0)
self._call(self._setup, ctypes.c_long(depth), ctypes.byref(self._ctx))
self._iocbs = (_Iocb * depth)()
self._ptrs = (ctypes.POINTER(_Iocb) * depth)(*(ctypes.pointer(c) for c in self._iocbs))
self._iovecs = (_Iovec * (depth * IOVECS))()
iovecs, size = ctypes.addressof(self._iovecs), ctypes.sizeof(_Iovec)
for i, cb in enumerate(self._iocbs):
# everything but the token, the iovec count and the iovecs is fixed
cb.opcode, cb.fildes, cb.buf = IOCB_CMD_PWRITEV, fd, iovecs + i * IOVECS * size
# The per-frame fields as 64-bit words, written with numpy: iocb word 4 is
# the iovec count, an iovec is (address, length), and an event is (data,
# iocb, result, result2).
self._iocb_words = np.frombuffer(self._iocbs, dtype=np.uint64)
self._counts = np.arange(depth) * 8 + 4
self._iovec_words = np.frombuffer(self._iovecs, dtype=np.uint64)
self._events = (_Event * depth)()
self._event_words = memoryview(self._events).cast('B').cast('q')
self._timeout = _Timespec()
def _call(self, nr: int, *args) -> int:
ret = self._syscall(ctypes.c_long(nr), *args)
if ret < 0:
err = ctypes.get_errno()
raise OSError(err, os.strerror(err))
return ret
address = staticmethod(address)
layout = staticmethod(layout)
def submit(self, plan: Layout, bases: list[int], start: int, count: int) -> int:
"""Queue `count` of the plan's requests from `start`, in order, gathered
from the spans at `bases`. Raises OSError only when the kernel took none
of them; a short count means the next one failed, and nothing of it, or of
any after it, was queued."""
if count > self.depth:
raise ValueError(f'{count} requests for a context of {self.depth}')
bases = np.array(bases, dtype=np.uint64)
lo, hi = plan.first[start], plan.first[start + count]
words = plan.words[lo:hi] - 2 * IOVECS * start
self._iovec_words[words] = bases[plan.span[lo:hi]] + plan.offset[lo:hi]
self._iovec_words[words + 1] = plan.length[lo:hi]
self._iocb_words[self._counts[:count]] = plan.pieces[start:start + count] # nbytes counts iovecs for PWRITEV
return self._call(self._submit, self._ctx, ctypes.c_long(count), self._ptrs)
def reap(self, min_nr: int, timeout: float) -> list[int]:
"""Finished requests' results, waiting up to `timeout` s (0: not at all)
for at least `min_nr`; `min_nr` must not exceed what is queued, or this
waits out the whole timeout. The GIL is released while it waits (ctypes).
A signal ends io_getevents with EINTR, and nothing retries a ctypes call
(PEP 475 is os.* only), so this does: modeld's frame thread takes a
SIGUSR2 from msgq ~160 times a second."""
end = time.monotonic() + timeout
while True:
left = max(0.0, end - time.monotonic())
self._timeout.sec = int(left)
self._timeout.nsec = int((left - int(left)) * 1e9)
try:
n = self._call(self._getevents, self._ctx, ctypes.c_long(min_nr), ctypes.c_long(self.depth),
self._events, ctypes.byref(self._timeout))
except InterruptedError:
if time.monotonic() >= end:
return []
continue
return self._event_words[2:4 * n:4].tolist()
def close(self) -> None:
"""Destroy the context. The kernel cancels what is still queued and waits,
uninterruptibly, for it to finish: only call this with nothing in flight,
or after the endpoint has been disabled, which completes everything."""
if self._ctx.value:
ctx, self._ctx = self._ctx, ctypes.c_ulong(0)
self._call(self._destroy, ctx)
jetlink_repo/jetlink/transport/base.py
+57 -59
View File
@@ -79,6 +79,33 @@ class Transport(ABC):
def send(self, msg_type: int, seq: int, parts=(), flags: int = 0, timeout: float | None = None) -> None:
"""Send one message. `parts` is an iterable of buffers, sent as one message."""
def try_send(self, msg_type: int, seq: int, parts=(), timeout: float | None = None) -> bool:
"""send(), unless the link would first make the caller wait for the peer
to take earlier messages: then False, and nothing was sent. Only a
transport that queues its writes can tell (FfsTransport); the rest send,
within `timeout` as send() does, and say True."""
self.send(msg_type, seq, parts, timeout=timeout)
return True
@property
def datagrams(self) -> bool:
"""Do frames the caller can do without go as datagrams? Only a phone's
cable, once its server offered them (TcpTransport.use_datagrams)."""
return False
def use_datagrams(self, port: int, token: int) -> bool:
"""Send those frames to the server's UDP `port` behind its `token`, where
the link can (TcpTransport); False and nothing changes elsewhere."""
return False
def stop_datagrams(self) -> None:
"""Frames on the link itself again."""
def net_drops(self) -> int | None:
"""Packets the kernel dropped on their way in since the link came up, on a
link that goes through the network stack (TcpTransport); None elsewhere."""
return None
@abstractmethod
def recv(self, timeout: float | None = None) -> Message:
"""Block for one message. Raises LinkTimeout if `timeout` elapses."""
@@ -161,34 +188,23 @@ class RxBuffer:
class StreamTransport(Transport):
"""Framing over any ordered byte stream.
TCP, USB bulk and FunctionFS are all streams, so they all need exactly this.
TCP and FunctionFS are both streams, so they both need exactly this.
Subclasses supply only the two primitives that differ.
"""
# Extra capacity past the current message, for transports whose reads need a
# minimum buffer size (a bulk OUT endpoint wants a whole packet).
read_slack = 0
# Bulk endpoints reject a read whose buffer is not a whole number of packets.
# 0 means "no constraint" (TCP).
packet_size = 0
read_chunk = 1 << 20
# Pad sent messages to a multiple of this and expect the peer's padded
# likewise; 0 uses the one-byte PADDED rule. See protocol.GADGET_TX_ALIGN.
# Pad sent messages to a multiple of this; 0 uses the one-byte PADDED rule.
# The peer's messages always use the PADDED rule: only the gadget pads to a
# burst, and only a USB host (the Swift server) reads what it sends. See
# protocol.GADGET_TX_ALIGN.
tx_align = 0
rx_align = 0
# Largest single write. FunctionFS allocates a contiguous buffer per writev,
# so a 4 MB upload chunk hits ENOMEM on a fragmented device where a few
# hundred KB of inference does not. 0 means no cap (TCP).
write_chunk = 0
def __init__(self, rx_size: int = 1 << 20):
self.rx = RxBuffer(rx_size)
self._desynced = False
self._send_deadline: float | None = None
def _write_timeout(self, default: float | None = None) -> float | None:
def _write_timeout(self) -> float | None:
if self._send_deadline is None:
return default
return None
remaining = self._send_deadline - time.monotonic()
if remaining <= 0:
raise LinkError('send timed out; link abandoned')
@@ -196,9 +212,10 @@ class StreamTransport(Transport):
# -- primitives a subclass must provide ----------------------------------
@abstractmethod
def _write(self, bufs: list[memoryview]) -> int:
"""Write from one or more buffers. Returns bytes written (may be partial)."""
"""Write from one or more buffers. Returns bytes written (may be partial).
A transport that overrides send (FfsTransport) need not provide it."""
raise NotImplementedError
@abstractmethod
def _read_into(self, dest: memoryview, timeout: float | None) -> int:
@@ -211,6 +228,21 @@ class StreamTransport(Transport):
# -- framing -------------------------------------------------------------
def send(self, msg_type: int, seq: int, parts=(), flags: int = 0, timeout: float | None = None) -> None:
bufs = self._frame(msg_type, seq, parts, flags)
self._send_deadline = None if timeout is None else time.monotonic() + timeout
try:
while bufs:
self._write_timeout()
n = self._write(bufs)
if n <= 0:
raise LinkError("peer went away during send")
bufs = advance(bufs, n)
finally:
self._send_deadline = None
def _frame(self, msg_type: int, seq: int, parts, flags: int) -> list[memoryview]:
"""One message as the buffers that go on the wire, in order: the header,
`parts`, then the padding. Nothing is copied."""
# cast('B'): slicing a float32 view in advance() would step by elements
bufs = [memoryview(p).cast('B') for p in parts]
length = sum(b.nbytes for b in bufs)
@@ -223,23 +255,8 @@ class StreamTransport(Transport):
# A bulk transfer only ends on a short packet; see protocol.PACKET_MULTIPLE.
flags |= P.Flag.PADDED
bufs.append(memoryview(_PAD)[:1])
header = P.pack_header(msg_type, seq, length, flags)
bufs.insert(0, memoryview(header))
self._send_deadline = None if timeout is None else time.monotonic() + timeout
try:
while bufs:
self._write_timeout()
n = self._write(take(bufs, self.write_chunk) if self.write_chunk else bufs)
if n <= 0:
raise LinkError("peer went away during send")
bufs = advance(bufs, n)
finally:
self._send_deadline = None
def _clamp_read(self, dest: memoryview) -> int:
"""How many bytes this transport may ask for in one read."""
n = min(dest.nbytes, self.read_chunk)
return (n // self.packet_size) * self.packet_size if self.packet_size else n
bufs.insert(0, memoryview(P.pack_header(msg_type, seq, length, flags)))
return bufs
def _fill(self, need: int, timeout: float | None) -> None:
"""Read until `need` bytes are buffered, or the deadline passes.
@@ -251,15 +268,10 @@ class StreamTransport(Transport):
whatever has already arrived and nothing else: that is how a caller
polls for a reply without blocking (JetlinkClient.drain).
"""
self.rx.reserve(need + self.read_slack)
self.rx.reserve(need)
end = None if timeout is None else time.monotonic() + timeout
while self.rx.available < need:
dest = self.rx.writable()[:self._read_limit(need - self.rx.available)]
if self._clamp_read(dest) == 0:
# No room for a whole packet: every read returns 0 and this loop spins
# while the peer blocks. read_slack is too small; say so, do not hang.
raise LinkError(f"no room to read the rest of a {need} byte message "
f"({self.rx.available} in hand); read_slack too small")
dest = self.rx.writable()[:need - self.rx.available]
remaining = None if end is None else max(0.0, end - time.monotonic())
n = self._read_into(dest, remaining)
if n:
@@ -267,17 +279,6 @@ class StreamTransport(Transport):
elif remaining == 0.0:
raise LinkTimeout(f"only {self.rx.available} of {need} bytes arrived in time")
def _read_limit(self, missing: int) -> int:
"""`missing` bytes of the current message, rounded up to a whole packet.
Keeps the USB host in sync: the gadget's messages are burst-aligned, so a
read for exactly what is left never stays outstanding past the end of a
message. Reading further desynced about once in 400 frames.
"""
if self.packet_size:
return -(-missing // self.packet_size) * self.packet_size
return missing
def recv(self, timeout: float | None = None) -> Message:
if self._desynced:
raise LinkError("stream desynced; the link must be reopened")
@@ -296,10 +297,7 @@ class StreamTransport(Transport):
raise LinkError(f"protocol error, link unusable: {e}") from e
# the remainder of the budget, so one recv cannot block for twice what it
# was given
if self.rx_align:
pad = -(P.HEADER_SIZE + length) % self.rx_align
else:
pad = 1 if flags & P.Flag.PADDED else 0
pad = 1 if flags & P.Flag.PADDED else 0
self._fill(P.HEADER_SIZE + length + pad,
None if end is None else max(0.0, end - time.monotonic()))
self.rx.take(P.HEADER_SIZE)
jetlink_repo/jetlink/transport/ffs.py
+263 -134
View File
@@ -11,7 +11,7 @@ has CONFIG_USB_F_FS built in, while a host needs no kernel driver at all, which
an L4T rootfs stripped of the gadget modules does not have. See
docs/transport.md.
Bring-up (scripts/comma/jetlink-root.sh gadget does all of this):
Bring-up (jetlink/comma/jetlink-root.sh gadget does all of this):
configfs gadget -> functions/ffs.jetlink -> mount -t functionfs
write descriptors + strings to ep0 -> ep1 (OUT) and ep2 (IN) appear
echo <udc> > UDC
@@ -28,9 +28,8 @@ import time
from collections import deque
from jetlink import protocol as P
from jetlink.transport.base import UDC_SYSFS, LinkError, LinkTimeout, StreamTransport, take, udc_speed, usb_link_info
from jetlink.transport.base import UDC_SYSFS, LinkError, LinkTimeout, StreamTransport, udc_speed, usb_link_info
from jetlink.transport.priority import background_thread, widen_affinity
from jetlink.transport.watchdog import WriteWatchdog
# --- FunctionFS ABI -------------------------------------------------------
@@ -49,17 +48,19 @@ USB_ENDPOINT_XFER_BULK = 0x02
EP_OUT = 0x01 # host -> device
EP_IN = 0x82 # device -> host
# FunctionFS kmallocs a contiguous buffer per read: order-5 and order-6 failures
# in ffs_epfile_read_iter stalled replies 100-300 ms, and order-2 does not fail.
# Do not go smaller to dodge the rare slow success: one page per read is ~19
# syscalls for a 74 KB reply and cost ~6 ms a frame.
# FunctionFS kmallocs a contiguous buffer per read and frees it on return. 16 KB
# is an order-2 page allocation, which this kernel retries without bound under
# memory pressure; vm.extra_free_kbytes (jetlink-root.sh) keeps that from
# stalling: no reclaim in 3,396 reads while recording. 8 KB would come from
# the slab like the writes, but a reply is 8,356 bytes, so it would take a
# second read: +0.3 ms a frame at p50.
READ_CHUNK = 16 * P.USB_MAX_PACKET
# How much the reader may queue before it stops. Only bounds memory if the
# consumer stalls: inference never needs more than one response.
MAX_QUEUED = 8 << 20
MAX_QUEUED = 512 << 10
# Read buffers kept for reuse. Steady state has one or two in flight, so this is
# only a ceiling for a transient backlog. 16 * 16 KB = 256 KB.
FREE_BUFS = 16
# only a ceiling for a transient backlog. 8 * 16 KB = 128 KB.
FREE_BUFS = 8
# Just below modeld's frame loop (54), so the loop wins a contended core and the
# reader still preempts everything else. See _raise_reader_priority.
READER_RT_PRIORITY = 51
@@ -74,11 +75,23 @@ EP_READY_TIMEOUT = 10.0
# EP_READY_TIMEOUT covers the claim that follows.
EP_OPEN_TIMEOUT = 10.0
# FunctionFS writes cannot time out: the request sits queued until the host
# FunctionFS writes cannot time out: a request sits queued until the host
# drains it, measured at 90 s while a Jetson booted and three minutes on a
# drive. Unbinding the UDC dequeues it, the writev returns ESHUTDOWN, and the
# caller retries. A real write is ~3.6 ms.
# drive. Only unbinding the UDC completes it (ESHUTDOWN). A send waiting for
# the host to take earlier writes waits this long by default.
WRITE_TIMEOUT = 15.0
# Bytes the IN endpoint may hold that the host has not taken. A whole frame is
# 459 KB, so a frame may go out behind one the host has not finished, and a
# third may not (try_send refuses it). The one behind covers a completion the
# kernel has not posted yet as well: FunctionFS posts each from a SCHED_OTHER
# kworker. Uploads stream through it in windows.
QUEUED_LIMIT = 1 << 20
# How long closing waits for queued writes to reach the host (a LEAVE) before
# it drops the gadget to complete them, and how long it then waits for that.
# An endpoint file whose request is still queued stays open in the kernel, and
# the next open of it answers EBUSY.
CLOSE_FLUSH = 0.5
ABORT_DRAIN = 2.0
# where the comma's gadget lives: jetlink-root.sh gadget builds it there and
# mounts its FunctionFS instance here
GADGET = '/sys/kernel/config/usb_gadget/jetlink'
@@ -91,10 +104,13 @@ READER_JOIN_TIMEOUT = 1.0
REBIND_SETTLE = 0.5
_NOT_READY = (errno.EIO, errno.ESHUTDOWN, errno.ENODEV)
# A signal inside a FunctionFS transfer is not a retry: ffs_epfile_io returns
# EINTR mid-transfer, os.writev re-issues the whole call, and the host sees the
# start of the message twice. modeld's msgq raises SIGUSR2 ~160 times a second,
# which cost one link failure per 30 s of driving. Nothing here waits on one.
# A signal inside a synchronous FunctionFS transfer is not a retry:
# ffs_epfile_io dequeues the request and returns EINTR whatever part of it the
# host already took, os.readv/os.writev re-issue the whole call, and the stream
# has lost or repeated bytes. modeld's msgq raises SIGUSR2 ~160 times a second,
# enough to fail a synchronous-write link about once per 30 s of driving.
# The reader masks every signal for its life. Writes are AIO, which never
# waits inside a transfer, so the frame thread needs no mask.
_IO_SIGNALS = signal.valid_signals()
@@ -138,20 +154,30 @@ def build_strings(name: str = 'jetlink') -> bytes:
class FfsTransport(StreamTransport):
"""The gadget transport. Reads run on their own thread.
"""The gadget transport. Reads run on their own thread; writes are AIO.
FunctionFS ignores O_NONBLOCK once the host has enabled the endpoint, so a
read on the caller's thread cannot honour a deadline: a 0.5 s timeout was
measured returning after 14 s. The reader hands whole chunks over a condition
variable the caller can wait on. Writes stay on the caller's thread; a host
that is not reading is a dead link either way.
variable the caller can wait on.
A message goes out as AIO requests of write_chunk bytes, all submitted in
one io_submit on the caller's thread (see aio.py): the kernel has copied the
bytes when it returns, the bus streams the requests back to back, and the
caller never waits inside a transfer. What it can wait on is room
(QUEUED_LIMIT): a host that is not taking data fills it, which a frame
answers by holding (try_send) and anything else by waiting, then dropping
the gadget, under its deadline.
"""
read_chunk = READ_CHUNK
# dwc3 resends a TRB about once in 400 frames. A message that fits one writev
# is replayed whole and the receiver drops it by seq; split across two writes
# the replay lands mid-stream and the link is lost. The largest inference
# request is 400 KB padded, and only uploads, which sha256 covers, are bigger.
write_chunk = 512 * P.USB_MAX_PACKET
# FunctionFS kmallocs one contiguous buffer per request, and AIO holds a
# frame's worth at once. A whole frame in one needs order-7 pages, which
# recording rollover leaves none of; 57 x 32 KB needs 15 order-3 blocks at
# once, and an aged comma can have as few as 3 free: io_submit then stalls
# ~94 ms in compaction. 8 KB is the largest size
# SLUB serves from a slab cache (kmalloc-8192), which never compacts: it
# falls back to order-1 slabs. A whole number of 1 KB packets, so no request
# ends on a short packet.
write_chunk = P.GADGET_TX_ALIGN // 2
tx_align = P.GADGET_TX_ALIGN
def __init__(self, mount: str = MOUNT, gadget: str | None = None,
@@ -185,7 +211,7 @@ class FfsTransport(StreamTransport):
read a dead link left queued keeps them busy: see _open_after_bounce); ep0
never does, which is why the owner keeps it.
The ep0 rule is unchanged and `udc` is what keeps it: the endpoints are
The ep0 rule still holds and `udc` is what keeps it: the endpoints are
still opened only once that controller reads configured. `bounce` is how a
write with no reader is freed, since the unbind that frees it belongs to
the owner. `owner_gadget` is the last resort for when the owner does not
@@ -206,9 +232,9 @@ class FfsTransport(StreamTransport):
def _prepare(self, mount: str, gadget: str | None) -> None:
# This end only receives replies: an INFER_RESP is 8 KB, or 74 KB when
# WANT_HIDDEN asks for the whole output, and the upload goes the other way.
# 256 KB leaves margin the memory-tight comma can spare, and RxBuffer grows
# 128 KB leaves margin the memory-tight comma can spare, and RxBuffer grows
# past it on demand.
super().__init__(rx_size=256 << 10)
super().__init__(rx_size=128 << 10)
self.mount = mount
self.gadget = gadget
self.bound_udc: str | None = None
@@ -217,7 +243,6 @@ class FfsTransport(StreamTransport):
self.ep0 = self.ep_out = self.ep_in = -1
self._state_fd = -1 # held-open UDC 'state' fd; see _udc_state
self._ready_deadline: float | None = None
self._read_size = READ_CHUNK
self._cv = threading.Condition()
self._chunks: deque[tuple[memoryview, float, float, float]] = deque()
self._free: deque[bytearray] = deque() # read buffers the consumer handed back; see _read_loop
@@ -227,9 +252,18 @@ class FfsTransport(StreamTransport):
self._closing = False
self._had_host = False
self._open_lock = threading.Lock()
self._write_aborted = False
self.last_send: dict = {}
self.send_totals: dict = {}
self._reader: threading.Thread | None = None
self._write_guard = WriteWatchdog(self._abort_write)
# the AIO writes: the context once ep2 is open, holding QUEUED_LIMIT in
# requests; requests submitted and reaped over its life, each write_chunk
# bytes (_requests), and when each submit's last one went in; and the
# first failure, which ends the link (see _queue)
self._tx_lock = threading.Lock()
self._aio = None
self._submitted = self._reaped = 0
self._batches: deque[tuple[int, float]] = deque()
self._tx_error: str | None = None
def _udc_state(self) -> str | None:
"""The controller's gadget state, off a held-open fd.
@@ -299,6 +333,11 @@ class FfsTransport(StreamTransport):
raise
self.ep_out = self._open_after_bounce(deadline)
self.ep_in = os.open(os.path.join(self.mount, 'ep2'), os.O_RDWR)
try:
self._aio = _open_aio(self.ep_in, QUEUED_LIMIT // self.write_chunk)
except OSError as e:
self._close_fds(('ep_in', 'ep_out'), None)
raise LinkError(f"no AIO context for the gadget's writes: {e}") from e
self._had_host = True
self._reader = threading.Thread(target=self._read_loop, name='jetlink-ffs-read', daemon=True)
self._reader.start()
@@ -314,9 +353,8 @@ class FfsTransport(StreamTransport):
def _open_after_bounce(self, deadline: float) -> int:
"""ep1 answered EBUSY: a transport this process closed after its link died
left a read queued on it, and FunctionFS keeps that read, and the endpoint
with it, until an unbind completes it. Seen when a Jetson's server
restarted mid-drive: some rejoins met EBUSY until the drive ended. The
unbind is the owner's, so a borrower asks for one, then opens once the
with it, until an unbind completes it, as when a Jetson's server
restarts mid-drive. The unbind is the owner's, so a borrower asks for one, then opens once the
host has the gadget back and the stale read has let go."""
log.warning("jetlink: ep1 is held by a read the last link left queued, asking the owner to bounce the gadget")
if not self._bounce():
@@ -336,8 +374,8 @@ class FfsTransport(StreamTransport):
An exchange leaves a read queued on the endpoint, and FunctionFS keeps it
queued until something completes it: nobody else may read that endpoint
until it does, and the only thing that dequeues it is the unbind. So the
owner gives the endpoints up like this and binds straight back - one
re-enumeration - without ever letting go of ep0, which is the whole point,
owner gives the endpoints up like this and binds straight back (one
re-enumeration) without ever letting go of ep0, which is the whole point,
since the gadget exists only while somebody holds it.
False when there is nothing to give up. Afterwards this transport is bare
@@ -346,6 +384,7 @@ class FfsTransport(StreamTransport):
if self.gadget is None or self._closing or self.ep_out < 0:
return False
udc, self._had_host = self.bound_udc, False
self._settle_writes()
self.unbind() # completes the queued read with ESHUTDOWN
reader, self._reader = self._reader, None
if reader is not None and reader is not threading.current_thread():
@@ -361,7 +400,7 @@ class FfsTransport(StreamTransport):
self._cv.notify_all()
self.rx.start = self.rx.end = 0
self._ready_deadline = None
self._write_aborted = False
self._tx_error = None
time.sleep(REBIND_SETTLE)
self.bind(udc)
return True
@@ -429,88 +468,186 @@ class FfsTransport(StreamTransport):
_close_quietly(self._state_fd)
self._state_fd = -1
def _shrink(self, attr: str) -> bool:
"""Halve a transfer size after the kernel refused to allocate for one.
ENOMEM here is contiguous memory, so it tracks fragmentation: a size that
worked at boot can fail an hour in, which is what a 1.7 GB model does to a
comma. Slow beats broken, and both directions need it.
def _abort_write(self, why: str = 'USB send exceeded its deadline') -> None:
"""Drop the gadget so the writes the host is not taking complete, with
ESHUTDOWN, and the link with them. Nothing else completes a queued
FunctionFS request (see WRITE_TIMEOUT), and one left queued holds its
endpoint file open in the kernel under the next user of the gadget.
"""
floor = (self.packet_size or P.USB_MAX_PACKET) * 16
current = getattr(self, attr)
if current <= floor:
return False
setattr(self, attr, max(floor, current // 2))
if attr == '_read_size':
self._free.clear() # pooled buffers are the old, larger size now
log.warning("jetlink: gadget could not allocate for %s, dropping to %d KB",
attr, getattr(self, attr) >> 10)
return True
def _shrink_write(self) -> bool:
return self._shrink('write_chunk')
def _abort_write(self) -> None:
"""Take the link down so a write nobody is reading returns. See WRITE_TIMEOUT.
On the watchdog thread: the one stuck in writev cannot act.
"""
self._write_aborted = True
log.warning("jetlink: no reader for %.3f s, dropping the gadget to free the write",
getattr(self, '_write_budget', WRITE_TIMEOUT))
self._set_error(f'{why}; link abandoned')
self.last_send['aborted'] = True
self.send_totals['aborts'] = self.send_totals.get('aborts', 0) + 1
log.warning("jetlink: %s, dropping the gadget to free the queued writes", why)
if self._bounce is None:
return self.unbind()
# A borrowed gadget: the unbind that dequeues this belongs to whoever owns
# ep0, so ask. If nobody answers - the owner died, or is wedged itself -
# take the gadget down from here anyway. This runs while a frame thread is
# inside a writev the kernel will never return from on its own, and a
# re-enumeration costs one rejoin where a wedged modeld costs the whole
# drive, the small model included.
# A borrowed gadget: the unbind belongs to whoever owns ep0, so ask. If
# nobody answers (the owner died, or is wedged itself), take the gadget
# down from here anyway: a re-enumeration costs one rejoin, where requests
# nothing completes cost the endpoint until the comma reboots.
if self._bounce():
return
log.error("jetlink: the gadget's owner did not answer; dropping the link from here")
self.unbind(self._owner_gadget)
def send(self, *args, **kwargs) -> None:
# Reset the quantum at each message: a latched shrink would split every
# later request across two writes and re-arm the dwc3 replay (see
# write_chunk). The pressure that forced it is usually gone by the next
# frame.
self.write_chunk = type(self).write_chunk
super().send(*args, **kwargs)
# -- writes: AIO ---------------------------------------------------------
def _write(self, bufs: list[memoryview]) -> int:
def try_send(self, msg_type: int, seq: int, parts=(), timeout: float | None = None) -> bool:
"""Queue one message unless that means waiting for the host: False, and
nothing sent, while writes it has not taken leave no room for this one
(QUEUED_LIMIT). A frame waits for nothing here, `timeout` included; the
caller holds instead (JetlinkModelState)."""
bufs = self._frame(msg_type, seq, parts, 0)
self._ensure_epfiles()
self._write_budget = self._write_timeout(WRITE_TIMEOUT)
# See _IO_SIGNALS: a signal here duplicates data on the wire.
was = signal.pthread_sigmask(signal.SIG_BLOCK, _IO_SIGNALS)
return self._queue(bufs, None, wait=False)
def send(self, msg_type: int, seq: int, parts=(), flags: int = 0, timeout: float | None = None) -> None:
bufs = self._frame(msg_type, seq, parts, flags)
self._ensure_epfiles()
self._queue(bufs, None if timeout is None else time.monotonic() + timeout, wait=True)
def _queue(self, bufs: list[memoryview], deadline: float | None, wait: bool) -> bool:
"""Put one framed message on the IN endpoint as requests of write_chunk
bytes. Returns once the kernel holds every byte, not once the host has
taken them; a failure among earlier requests is this send's LinkError.
`wait` waits for room until `deadline` (WRITE_TIMEOUT from now by
default), then drops the gadget; without it there is no waiting, and
False says the message was not sent."""
started = time.monotonic()
deadline = started + WRITE_TIMEOUT if deadline is None else deadline
total = sum(b.nbytes for b in bufs)
stats = self.last_send = {'bytes': total, 'requests': 0, 'submit_ms': 0.0, 'wait_ms': 0.0,
'backlog_kb': 0, 'backlog_ms': 0.0, 'chunk': self.write_chunk,
'refused': False, 'aborted': False, 'errno': None}
try:
if not self._write_guard.arm(self._write_budget):
raise LinkError('gadget write watchdog already expired or closed')
while True:
try:
n = os.writev(self.ep_in, bufs)
self._had_host = True
return n
except OSError as e:
if self._write_aborted:
# Say whose doing it was: ENODEV alone reads like a cable falling
# out of a socket.
raise LinkError(f"gadget write had no reader for {self._write_budget:.3f}s") from e
# FunctionFS submits a write as one request, so a failed writev put
# nothing on the wire and is safe to retry.
if e.errno in _NOT_READY and self._wait_for_host_ready():
continue
if e.errno == errno.ENOMEM and self._shrink_write():
# A short write is fine: send() loops until the message is out.
bufs = take(bufs, self.write_chunk)
continue
raise LinkError(f"gadget write failed: {e}{self._udc_note()}") from e
with self._tx_lock:
if self._closing or self._aio is None:
raise LinkError('gadget closing')
self._collect()
if self._tx_error is not None:
raise LinkError(self._tx_error)
if queued := self._submitted - self._reaped:
# what the host had not taken when this message came: the measure
# of a host falling behind, which nothing else on this end can see
stats['backlog_kb'] = queued * self.write_chunk >> 10
stats['backlog_ms'] = (started - self._batches[0][1]) * 1e3
if not wait and (queued + total // self.write_chunk) > self._aio.depth:
stats['refused'] = True
return False
# the views in `bufs` keep these addresses valid until io_submit returns
spans = [self._aio.address(b) for b in bufs if b.nbytes]
plan = self._aio.layout(tuple(n for _, n in spans), self.write_chunk)
self._submit_message(plan, [a for a, _ in spans], deadline, stats)
return True
finally:
completed = self._write_guard.disarm()
signal.pthread_sigmask(signal.SIG_SETMASK, was)
if not completed:
raise LinkError('gadget write had no reader before deadline; link abandoned')
self._count(stats)
def _submit_message(self, plan, bases: list[int], deadline: float, stats: dict) -> None:
"""io_submit the message's requests (an aio.Layout over the spans at
`bases`) as room allows: all at once for a frame, in windows for an upload
chunk larger than the context."""
done = 0
while done < plan.count:
room = self._aio.depth - (self._submitted - self._reaped)
if room <= 0:
self._wait_for_room(deadline, stats)
continue
done += self._submit(plan, bases, done, min(room, plan.count - done), stats)
def _submit(self, plan, bases: list[int], start: int, count: int, stats: dict) -> int:
"""io_submit `count` of the message's requests from `start`; how many the
kernel queued. FunctionFS reports a failed write as its completion
(_collect): io_submit itself fails only on a bad context or iocb."""
t0 = time.monotonic()
try:
n = self._aio.submit(plan, bases, start, count)
except OSError as e:
stats['errno'] = e.errno
why = f"gadget write failed: {e}{self._udc_note()}"
self._set_error(why)
raise LinkError(why) from e
now = time.monotonic()
stats['submit_ms'] += (now - t0) * 1e3
stats['requests'] += n
self._submitted += n
self._batches.append((self._submitted, now))
return n
def _wait_for_room(self, deadline: float, stats: dict) -> None:
"""Wait for the host to take queued writes, until `deadline`."""
now = time.monotonic()
if now >= deadline:
self._abort_write(f'the host took no USB data for {now - self._batches[0][1]:.2f} s')
raise LinkError(self._tx_error)
self._collect(1, deadline - now)
stats['wait_ms'] += (time.monotonic() - now) * 1e3
if self._tx_error is not None:
raise LinkError(self._tx_error)
def _set_error(self, why: str) -> None:
"""The link's first write failure, which every send after it raises."""
if self._tx_error is None:
self._tx_error = why
def _collect(self, min_nr: int = 0, timeout: float = 0.0) -> None:
"""Reap finished writes, waiting up to `timeout` for `min_nr` of them. The
first that did not write all of its bytes is the link's failure."""
queued = self._submitted - self._reaped
if not queued:
return
results = self._aio.reap(min(min_nr, queued), timeout)
self._reaped += len(results)
while self._batches and self._batches[0][0] <= self._reaped:
self._batches.popleft() # what is left starts with the oldest still queued
chunk = self.write_chunk
if results.count(chunk) != len(results):
bad = next(r for r in results if r != chunk)
self._set_error(f"gadget write failed: {os.strerror(-bad) if bad < 0 else f'{bad} of {chunk} bytes'}"
f"{self._udc_note()}")
def _count(self, stats: dict) -> None:
"""Session totals, which keep a slow send between the 1 Hz log samples."""
totals = self.send_totals
totals['messages'] = totals.get('messages', 0) + 1
for key in ('bytes', 'requests', 'refused'):
totals[key] = totals.get(key, 0) + stats[key]
for key, top in (('submit_ms', 'max_submit_ms'), ('wait_ms', 'max_wait_ms'), ('backlog_ms', 'max_backlog_ms')):
totals[top] = max(totals.get(top, 0.0), stats[key])
def _settle_writes(self) -> None:
"""Before the endpoint files go: give queued writes CLOSE_FLUSH to reach
the host (a LEAVE), drop the gadget to complete the rest, and destroy the
AIO context once nothing is left in it. io_destroy waits, uninterruptibly,
for what it cancels, and FunctionFS cannot cancel a request while the bus
is suspended (dwc3 refuses in LPM), so it is only called on an empty
context; otherwise the context is left to the process."""
if self._aio is None:
return
if not self._tx_lock.acquire(timeout=CLOSE_FLUSH):
# a send is waiting for room the host is not making; this frees it
self._abort_write('closing a link whose host stopped taking USB data')
if not self._tx_lock.acquire(timeout=ABORT_DRAIN):
log.error("jetlink: a send still holds the gadget's writes; leaving their AIO context to the process")
return
try:
self._drain(CLOSE_FLUSH)
if self._submitted > self._reaped:
self._abort_write(f'{self._submitted - self._reaped} USB writes still queued at close')
self._drain(ABORT_DRAIN)
if self._submitted > self._reaped:
log.error("jetlink: %d USB writes never completed; leaving their AIO context to the process",
self._submitted - self._reaped)
else:
self._aio.close()
self._aio = None
self._submitted = self._reaped = 0
self._batches.clear()
finally:
self._tx_lock.release()
def _drain(self, seconds: float) -> None:
end = time.monotonic() + seconds
while self._submitted > self._reaped and (left := end - time.monotonic()) > 0:
self._collect(1, left)
# -- the reader thread ---------------------------------------------------
@@ -528,8 +665,8 @@ class FfsTransport(StreamTransport):
"""Realtime priority, so a completed read leaves the endpoint at once.
The creator has dropped to SCHED_OTHER, and under recording plus onroad load
the wait to be scheduled after a transfer was 17 ms mean, 23 ms max - over
budget on read_wait alone. The reader copies a chunk and blocks again, so it
the wait to be scheduled after a transfer measures 17 ms mean, 23 ms max,
over budget on read_wait alone. The reader copies a chunk and blocks again, so it
never holds a core. Best effort, and open-coded because jetlink must not
import openpilot.
"""
@@ -539,15 +676,14 @@ class FfsTransport(StreamTransport):
pass
def _read_loop(self) -> None:
# As in _write: an interrupted read drops the packets it already took. This
# thread handles no signals, so mask them for its whole life.
# See _IO_SIGNALS.
signal.pthread_sigmask(signal.SIG_BLOCK, _IO_SIGNALS)
self._widen_affinity()
self._raise_reader_priority()
# Fill the pool up front: recycling alone leaves a gap while a reply's
# chunks arrive and the consumer is a scheduling beat behind, and an
# allocation there reclaims under memory pressure (24 ms, over budget).
self._free.extend(bytearray(self._read_size) for _ in range(FREE_BUFS))
self._free.extend(bytearray(READ_CHUNK) for _ in range(FREE_BUFS))
while not self._closing:
with self._cv:
while self._queued >= MAX_QUEUED and not self._closing:
@@ -566,19 +702,16 @@ class FfsTransport(StreamTransport):
# Reuse a buffer the consumer handed back: a fresh bytearray here was
# measured stalling 20+ ms mid-frame under memory pressure, and the read
# cannot start until it returns. Only this thread pops the free list.
buf = self._free.popleft() if self._free else bytearray(self._read_size)
buf = self._free.popleft() if self._free else bytearray(READ_CHUNK)
read_started = time.monotonic()
try:
# Multiples of the packet size only: the OUT endpoint rejects anything
# else, and _read_size is only ever halved from one.
# A multiple of the packet size: the OUT endpoint rejects anything else.
got = os.readv(self.ep_out, [buf])
except OSError as e:
if self._closing:
return
if e.errno in _NOT_READY and self._wait_for_host_ready():
continue
if e.errno == errno.ENOMEM and self._shrink('_read_size'):
continue
self._fail(f"gadget read failed: {e}{self._udc_note()}")
return
if got == 0:
@@ -621,10 +754,9 @@ class FfsTransport(StreamTransport):
else:
self._chunks.popleft()
# Copied out, so return the buffer to the pool. chunk.obj survives a
# reslice; skip anything not from this pool at the current size, such
# as a shrunk buffer or a test's bytes.
# reslice; skip anything not from this pool, such as a test's bytes.
buf = chunk.obj
if type(buf) is bytearray and len(buf) == self._read_size and len(self._free) < FREE_BUFS:
if type(buf) is bytearray and len(buf) == READ_CHUNK and len(self._free) < FREE_BUFS:
self._free.append(buf)
self._queued -= n
self._cv.notify_all()
@@ -654,7 +786,7 @@ class FfsTransport(StreamTransport):
The endpoints answer ENODEV both before a host configures the gadget and
after it disconnects, and only the first deserves the grace period; giving
it to the second cost 10 s and 201 frames on a mid-drive disconnect. The UDC
it to the second costs a mid-drive disconnect 10 s, about 200 frames. The UDC
state separates them, and it is set in the interrupt that disabled the
endpoint, so it is current when the failed call returns.
"""
@@ -665,18 +797,7 @@ class FfsTransport(StreamTransport):
def close(self) -> None:
self._closing = True
self._write_guard.close()
self._write_guard.thread.join(READER_JOIN_TIMEOUT)
if self._write_guard.thread.is_alive():
# It is stuck inside its own abort, which is already unbinding the
# gadget. This used to raise, to keep a late abort from unbinding a
# gadget the next owner had bound - but the raise came before a single
# fd was closed, so ep0 stayed open for the life of the process and
# every descriptor write after it answered ESRCH. unbind() is
# idempotent, so a late abort finds nothing left to let go of; the leak
# was the worse half by far.
log.warning("jetlink: gadget watchdog teardown did not finish in %.0f s, "
"closing the endpoints anyway", READER_JOIN_TIMEOUT)
self._settle_writes()
# Unbinding disables the endpoints, which completes the reader's pending
# request with ESHUTDOWN and lets the thread exit before its fd goes away.
self.unbind()
@@ -707,6 +828,14 @@ class FfsTransport(StreamTransport):
_close_quietly(fd)
def _open_aio(fd: int, depth: int):
"""The AIO context for ep2's writes (aio.Aio). Imported on first use: the
resident gadget owner imports this module and never writes, and ctypes is
not free."""
from jetlink.transport.aio import Aio
return Aio(fd, depth)
def _close_quietly(fd: int) -> None:
# Runs on its own thread when a read never came back; that thread inherits
# the creator's scheduling, which in modeld is the frame loop's.
jetlink_repo/jetlink/transport/tcp.py
+81 -8
View File
@@ -12,13 +12,19 @@ the vendor interface instead. See docs/transport.md.
One sendmsg per message (header and body in one segment train, NODELAY set)
and reads straight into the reusable receive buffer, so the steady state does
not allocate on the wire.
not allocate on the wire. Over the cable, frames can go as UDP datagrams
instead (use_datagrams, protocol.DATAGRAM_MAGIC); everything else stays here.
"""
from __future__ import annotations
import logging
import socket
from collections.abc import Iterator
from jetlink.transport.base import LinkError, StreamTransport, udc_speed, usb_link_info
from jetlink import protocol as P
from jetlink.transport.base import LinkError, StreamTransport, advance, take, udc_speed, usb_link_info
log = logging.getLogger('jetlink.tcp')
DEFAULT_PORT = 5599
# The comma's end of the USB network link a phone dials (jetlink-root.sh gadget --ios).
@@ -32,6 +38,9 @@ class TcpTransport(StreamTransport):
self.sock = sock
self._timeout: float | None = -1.0 # force the first settimeout
_tune(sock)
# (socket connected to the server's frame port, its token); see use_datagrams
self._udp: tuple[socket.socket, int] | None = None
self._drops_at = _softnet_dropped()
def on_the_cable(self) -> bool:
"""Is either end the comma's cable address? Then this is a phone's USB
@@ -45,6 +54,54 @@ class TcpTransport(StreamTransport):
def link_info(self) -> dict:
return usb_link_info('cable', udc_speed()) if self.on_the_cable() else {'kind': 'tcp'}
def net_drops(self) -> int:
"""Every CPU's receive backlog drops since this link came up
(/proc/net/softnet_stat): a receive core starved of time shows here
before anywhere else. -1 where it cannot be read."""
now = _softnet_dropped()
return -1 if now < 0 or self._drops_at < 0 else now - self._drops_at
# -- frames as datagrams ---------------------------------------------------
def use_datagrams(self, port: int, token: int) -> bool:
"""Send frames the caller can do without (send_datagrams) to the peer's
UDP `port` from now on, behind the server's `token`: what its HELLO_RESP
offers over the cable. False, and nothing changes, anywhere else: off the
cable a lost datagram has more than a USB link between it and the phone.
Blocking: the 4 MB send buffer takes a frame whole."""
self.stop_datagrams()
if not self.on_the_cable():
return False
try:
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 4 << 20)
udp.connect((self.sock.getpeername()[0], port))
except OSError as e:
log.warning("jetlink: no frame datagrams to port %d (%s); frames stay on TCP", port, e)
return False
self._udp = (udp, token)
return True
def stop_datagrams(self) -> None:
udp, self._udp = self._udp, None
if udp is not None:
udp[0].close()
@property
def datagrams(self) -> bool:
return self._udp is not None
def send_datagrams(self, msg_type: int, seq: int, parts=(), flags: int = 0) -> None:
"""The message as `datagrams`, one sendmsg each, once use_datagrams took
an offer. A send the kernel refuses (the phone's server gone) fails the
link, as a refused send on the stream does."""
udp, token = self._udp
try:
for piece in datagrams(self._frame(msg_type, seq, parts, flags), token, seq):
udp.sendmsg(piece)
except OSError as e:
raise LinkError(f"frame datagrams failed: {e}") from e
@classmethod
def connect(cls, host: str, port: int = DEFAULT_PORT, timeout: float = 5.0) -> TcpTransport:
return cls(socket.create_connection((host, port), timeout=timeout))
@@ -57,16 +114,11 @@ class TcpTransport(StreamTransport):
srv.listen(backlog)
return srv
@classmethod
def accept(cls, srv: socket.socket) -> tuple[TcpTransport, tuple]:
conn, addr = srv.accept()
return cls(conn), addr
@classmethod
def listen_once(cls, host: str = '0.0.0.0', port: int = DEFAULT_PORT,
timeout: float | None = None) -> tuple[TcpTransport, tuple]:
"""Take exactly one incoming connection and stop listening: a phone that
dials us, for the bench and parity scripts run from a Mac."""
dials us, for the benchmark and parity scripts run from a Mac."""
srv = cls.listen(host, port)
try:
srv.settimeout(timeout)
@@ -109,6 +161,7 @@ class TcpTransport(StreamTransport):
return n
def close(self) -> None:
self.stop_datagrams()
# shut down first: a close alone keeps the connection up while another
# process holds a copy of the socket (the comma's gadget owner holds the
# phone's dial), and the peer hears nothing until that copy goes too
@@ -119,6 +172,26 @@ class TcpTransport(StreamTransport):
pass
def _softnet_dropped(path: str = '/proc/net/softnet_stat') -> int:
"""The kernel's receive backlog drops, all CPUs: the second column, in hex."""
try:
with open(path) as f:
return sum(int(line.split()[1], 16) for line in f if line.strip())
except (OSError, ValueError, IndexError):
return -1
def datagrams(bufs: list[memoryview], token: int, seq: int) -> Iterator[list]:
"""A message's stream bytes `bufs` as protocol.datagram_pieces, each its
header and views into `bufs`. The header's buffer is reused: send or copy
each datagram before taking the next."""
total = sum(b.nbytes for b in bufs)
header = bytearray(P.DATAGRAM_HEADER_SIZE)
for offset, size in P.datagram_pieces(total):
P.pack_datagram_header_into(header, token, seq, offset, total)
yield [header, *take(advance(bufs, offset), size)]
def _tune(sock: socket.socket) -> None:
# NODELAY is the one that matters: without it the header and the body can be
# split across an RTT. Guarded anyway; a transport must not die in setsockopt.
jetlink_repo/jetlink/transport/watchdog.py
-71
View File
@@ -1,71 +0,0 @@
"""
Copyright (c) 2026-, Zeph Leggett.
This file is part of jetlink and is licensed under the MIT License.
See the LICENSE file in the root directory for more details.
"""
from __future__ import annotations
import threading
import time
from jetlink.transport.priority import background_thread
class WriteWatchdog:
"""One worker per link, with cancellation serialized against expiry.
Once expiry wins, the link cannot be armed again. The callback runs outside
the lock: unbinding a broken USB controller must not block disarm().
"""
def __init__(self, abort):
self._abort = abort
self._cv = threading.Condition()
self._deadline = None
self._expired = False
self._closed = False
self.thread = threading.Thread(target=self._run, name='jetlink-write-guard', daemon=True)
self.thread.start()
def arm(self, budget: float) -> bool:
with self._cv:
if self._expired or self._closed:
return False
if self._deadline is not None:
raise RuntimeError('concurrent writes on one link')
self._deadline = time.monotonic() + budget
self._cv.notify()
return True
def disarm(self) -> bool:
with self._cv:
self._deadline = None
self._cv.notify()
return not self._expired and not self._closed
def close(self) -> None:
with self._cv:
self._closed = True
self._deadline = None
self._cv.notify()
def _run(self) -> None:
# This inherits the creator's policy and pin, which in modeld is the frame
# loop's SCHED_FIFO 54 on one core.
background_thread()
with self._cv:
while not self._closed:
if self._deadline is None:
self._cv.wait()
continue
remaining = self._deadline - time.monotonic()
if remaining > 0:
self._cv.wait(remaining)
continue
self._expired = True
self._deadline = None
break
else:
return
self._abort()
jetlink_repo/pyproject.toml
+5
View File
@@ -24,11 +24,16 @@ dev = ["pytest>=7", "ruff", "onnx>=1.16"]
[tool.setuptools.packages.find]
include = ["jetlink*"]
# The root script the comma runs (jetlink/comma/root.py), so a wheel has it too
[tool.setuptools.package-data]
"jetlink.comma" = ["jetlink-root.sh"]
[tool.setuptools.dynamic]
version = { attr = "jetlink.__version__" }
[tool.pytest.ini_options]
testpaths = ["tests"]
markers = ["real_aio: uses Linux AIO itself, not tests/aio_fakes.FakeAio"]
# The fork's rule set (sunnypilot's pyproject.toml), so code moves between the
# two without reformatting, with two differences the fork does not allow:
openpilot/starpilot/models/JETLINK.md
+8 -8
View File
@@ -23,19 +23,19 @@ that StarPilot has switched to it. Set up parked, with both devices online.
The host program is the upstream **Jetlink** application, developed by Zeph
Leggett / ZoomPilot. Its download identifiers and commands keep their upstream
names; installing the host does not require installing ZoomPilot on the comma.
The bundled client uses Jetlink protocol 3 (package 0.8.3). A different host
The bundled client uses Jetlink protocol 3 (package 0.8.7). A different host
release still needs a successful compatible handshake and device validation.
### Jetson and Linux PC
On a supported NVIDIA host, review and run the [upstream installer](https://github.com/zoompilot/jetlink/blob/c17cd5c/install.sh).
On a supported NVIDIA host, review and run the [upstream installer](https://github.com/zoompilot/jetlink/blob/d24328d/install.sh).
The upstream Jetson guide specifies an Orin Nano Super 8 GB and JetPack 7.2.1;
the Linux guide specifies an RTX 20-series or newer GPU with driver 580 or newer.
Keep the computer powered, cool, and awake. Connect its USB-A port to the comma's USB-C port.
Use `jetlink status` to check the host and `jetlink logs` to inspect problems.
Host power, sleep, and driver setup details remain in the credited
[Jetson](https://github.com/zoompilot/jetlink/blob/c17cd5c/docs/jetson.md) and
[Linux](https://github.com/zoompilot/jetlink/blob/c17cd5c/docs/linux-pc.md) references;
[Jetson](https://github.com/zoompilot/jetlink/blob/d24328d/docs/jetson.md) and
[Linux](https://github.com/zoompilot/jetlink/blob/d24328d/docs/linux-pc.md) references;
their comma-side ZoomPilot instructions are replaced by **Set up StarPilot** above.
### Mac
@@ -45,7 +45,7 @@ Use an Apple silicon Mac with macOS 15 or newer. Get the Jetlink DMG from
in Applications, and start it. Keep it awake and connect by USB 3.
Use **USB computer** in StarPilot's Galaxy, rather than following the comma-side
ZoomPilot installation instructions in the credited
[Mac host reference](https://github.com/zoompilot/jetlink/blob/c17cd5c/docs/macos-app.md).
[Mac host reference](https://github.com/zoompilot/jetlink/blob/d24328d/docs/macos-app.md).
### iPhone and iPad
@@ -54,7 +54,7 @@ It requires a USB-C device supporting USB 3 and iOS/iPadOS 26.1 or newer.
Choose **iPhone / iPad** in StarPilot, allow the host app's Local Network access,
connect by USB 3, and keep the app open and the device unlocked. Check sustained
thermal performance with the app's benchmark; that benchmark excludes cable latency.
The [credited Apple host reference](https://github.com/zoompilot/jetlink/blob/c17cd5c/docs/iphone-app.md)
The [credited Apple host reference](https://github.com/zoompilot/jetlink/blob/d24328d/docs/iphone-app.md)
contains hardware details. StarPilot supplies the comma-side instructions above.
### Android
@@ -64,7 +64,7 @@ upstream Jetlink Android APK from [its releases](https://github.com/zoompilot/je
Use a recent USB 3-capable Android 12+ phone, a powered USB 3 hub, and separate
comma power. Choose **USB computer** in StarPilot. Follow the host's USB permission
prompt and benchmark sustained performance before use.
[Credited Android host reference](https://github.com/zoompilot/jetlink/blob/c17cd5c/docs/android-app.md).
[Credited Android host reference](https://github.com/zoompilot/jetlink/blob/d24328d/docs/android-app.md).
## Use and recovery
@@ -83,7 +83,7 @@ and defaults off.
## Attribution and license
This StarPilot guide adapts the host setup structure from
[ZoomPilot / Jetlink at c17cd5c](https://github.com/zoompilot/jetlink/tree/c17cd5c/docs).
[ZoomPilot / Jetlink at d24328d](https://github.com/zoompilot/jetlink/tree/d24328d/docs).
Jetlink is copyright (c) 2026 Zeph Leggett and licensed under the
[MIT license](../../../jetlink_repo/LICENSE), whose full text is preserved.
ZoomPilot source links, project names, authorship, and history are attribution;
upstream-sync.json
+9 -27
View File
@@ -65,33 +65,15 @@
{
"path": "jetlink_repo",
"url": "https://github.com/zoompilot/jetlink.git",
"commit": "c17cd5c52be0ee83b9a6265b7da69d24ad6c5d02",
"tree": "5199f58cb390fcaa4c798be82566cc87d9271da7",
"exclude": [
".github",
".gitignore",
"CHANGELOG.md",
"JetlinkKit",
"android",
"docs",
"install.sh",
"ios",
"macos",
"tests",
"scripts/99-jetlink-host.rules",
"scripts/99-jetlink-usb-wakeup.rules",
"scripts/bench_link.py",
"scripts/build-linux.sh",
"scripts/changelog.py",
"scripts/deploy_to_comma.sh",
"scripts/jetlink",
"scripts/jetlink-server.service",
"scripts/verify_parity.py",
"scripts/comma/jetlink_bench.py",
"scripts/comma/jetlink_hold.py",
"scripts/comma/jetlink_live_bench.sh",
"scripts/comma/jetlink_replay.py"
]
"commit": "d24328d849b0e5154b55f149dc18c5dfe8bd79cc",
"tree": "21f19948ccaa8b0055fc83ed916172997fb80e00",
"include": [
"jetlink",
"LICENSE",
"pyproject.toml",
"README.md"
],
"exclude": []
}
]
}