Files
StarPilot/selfdrive/debug/format_fingerprints.py
T
Shane Smiskol 95224db413 selfdrive/car: ban cereal and capnp (#33208)
* ban cereal and msgq

* common too

* do toyota/values.py

* do all fingerprints

* example without builder

* this still works, but no type checking anymore

* stash

* wtf, how does this work

* okay actually not bad

* safe

* epic!

* stash data_structures.py

* some clean up

* hell yeah

* clean up old file

* add to delete

* delete

This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf.

* switch more CarParams stuff over

remove unused

* fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :(

* fix this too

* fix this too

* remove more cereal and add some good hyundai tests

* bunch more typing

* override default with 20hz radar

* temp capnp converter helper

* more lateralTuning

* small union replicator is better than what i was trying, and fixes mypy dynamic typing issues

* can keep all this the same now!

* type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!)

* revert more

* get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(

would have been `CarParams.NetworkLocation()`

* Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :("

This reverts commit bb28b228becba932052d2fc5a4389784027435b1.

* remove cereal from car_helpers (TODO: caching)

* remove a bunch of temp lines

* use dataclass_transform!

* remove some car.CarParams from the interfaces

* remove rest of car.CarParams from the interfaces

* same which() API

* sort

* from cereal/cache from fingerprinting!

* more typing

* dataclass to capnp helper for CarParams, cached it since it's kinda slow

* (partial) fix process replay fingerprintig for new API

* latcontrollers take capnp

* forgot this

* fix test_models

* fix unit tests

* not here

* VehicleModel and controller still takes capnp CP since they get it from Params()

* fix modeld test

* more fix

* need to namespace to structs, since CarState is both class and struct

* this was never in the base class?!

* clean that up again

* fix import error

fix import error

* cmts and more structs

* remove some more cereal from toyota + convert CarState to capnp

* bruh this was wrong

* replace more cereal

* EventName is one of the last things...

* replace a bunch more cereal.car

* missing imports

* more

* can fix this typing now

* proper toyota+others CS typing!

* mypy can detect return type of CS.update() now

* fix redeclaration of cruise_buttons type

* mypy is only complaining about events now

* temp fix

* add carControl struct

* replace CarControl

i hope there's no circular imports in hyundai's CC

* fine now

* lol this was wrong too

* fix crash

* include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( )

but attrs does, maybe will switch in the future

* clean up

* try out attr.s for its converter (doesn't work recursively yet, but interesting!)

* Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)"

This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9.

* test processes doesn't fail anymore (on toyota)!

* fix honda crash

* stash

* Revert "stash"

This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0.

* remove a bunch more cereal!

* LET'S GOOO

* fix these tests

* and these

* and that

* stash, something is wrong with hyundai enable

* Revert "stash, something is wrong with hyundai enable"

This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03.

* forgot these

* remove cereal from fw_versions

* Revert "remove cereal from fw_versions"

This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56.

* remove rest of the cereal exceptions!

* fix that

* add typing to radard since I didn't realize RI.update() switched from cereal to structs

* and here too!

* add TODO for slots

* needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( )

* remove the struct converter

* save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy

btw pickle.loads(pickle.dumps()) is faster by ~1% CPU

* deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone

saves ~1% CPU

* add non-copying asdict function

* should save ~3% CPU (still 4% above baseline)

* fix that, no dict support

* ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s)

* give a better name

* fix

* dont support none, capitalize

* sheesh, this called type() on every field

* remove CS.events, clean up

* bump card %

* this was a bug on master!

* add a which enum

* default to pid

* revert

* update refs

* not needed, but consistent

* just Ecu

* don't need to do this in this pr

* clean up

* no cast

* consistent typing

* rm

* fix

* can do this if we're desperate for the last few %

* Revert "can do this if we're desperate for the last few %"

This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299.

* type this

* don't need to convert carControl

* i guess don't support set either

* fix CP type hint

* simplify that
old-commit-hash: 6a15c42143a4764e79df851d797f950a6212f464
2024-08-16 15:13:00 -07:00

77 lines
2.3 KiB
Python
Executable File

#!/usr/bin/env python3
import jinja2
import os
from openpilot.common.basedir import BASEDIR
from openpilot.selfdrive.car.interfaces import get_interface_attr
CARS = get_interface_attr('CAR')
FW_VERSIONS = get_interface_attr('FW_VERSIONS')
FINGERPRINTS = get_interface_attr('FINGERPRINTS')
FINGERPRINTS_PY_TEMPLATE = jinja2.Template("""
{%- if FINGERPRINTS[brand] %}
# ruff: noqa: E501
{% endif %}
{% if FW_VERSIONS[brand] %}
from openpilot.selfdrive.car.structs import CarParams
{% endif %}
from openpilot.selfdrive.car.{{brand}}.values import CAR
{% if FW_VERSIONS[brand] %}
Ecu = CarParams.Ecu
{% endif %}
{% if comments +%}
{{ comments | join() }}
{% endif %}
{% if FINGERPRINTS[brand] %}
FINGERPRINTS = {
{% for car, fingerprints in FINGERPRINTS[brand].items() %}
CAR.{{car.name}}: [{
{% for fingerprint in fingerprints %}
{% if not loop.first %}
{{ "{" }}
{% endif %}
{% for key, value in fingerprint.items() %}{{key}}: {{value}}{% if not loop.last %}, {% endif %}{% endfor %}
}{% if loop.last %}]{% endif %},
{% endfor %}
{% endfor %}
}
{% endif %}
FW_VERSIONS{% if not FW_VERSIONS[brand] %}: dict[str, dict[tuple, list[bytes]]]{% endif %} = {
{% for car, _ in FW_VERSIONS[brand].items() %}
CAR.{{car.name}}: {
{% for key, fw_versions in FW_VERSIONS[brand][car].items() %}
(Ecu.{{key[0]}}, 0x{{"%0x" | format(key[1] | int)}}, \
{% if key[2] %}0x{{"%0x" | format(key[2] | int)}}{% else %}{{key[2]}}{% endif %}): [
{% for fw_version in (fw_versions + extra_fw_versions.get(car, {}).get(key, [])) | unique | sort %}
{{fw_version}},
{% endfor %}
],
{% endfor %}
},
{% endfor %}
}
""", trim_blocks=True)
def format_brand_fw_versions(brand, extra_fw_versions: None | dict[str, dict[tuple, list[bytes]]] = None):
extra_fw_versions = extra_fw_versions or {}
fingerprints_file = os.path.join(BASEDIR, f"selfdrive/car/{brand}/fingerprints.py")
with open(fingerprints_file) as f:
comments = [line for line in f.readlines() if line.startswith("#") and "noqa" not in line]
with open(fingerprints_file, "w") as f:
f.write(FINGERPRINTS_PY_TEMPLATE.render(brand=brand, comments=comments, FINGERPRINTS=FINGERPRINTS,
FW_VERSIONS=FW_VERSIONS, extra_fw_versions=extra_fw_versions))
if __name__ == "__main__":
for brand in FW_VERSIONS.keys():
format_brand_fw_versions(brand)