Commit Graph

606 Commits

Author SHA1 Message Date
Dean Lee 2ad501cc94 common/params.cc: Handle EINTR for fsync in Params::put to Improve Robustness (#35142)
Handle EINTR for fsync in Params::put to Improve Robustness
2025-05-14 11:21:36 -07:00
Maxime Desroches 47ed90c6cf Reapply "Mypy: Got passing on macos (#34591)" (#35126) (#35153)
* Mypy: Got passing on macos (#34591)

* Mypy: Got mypy passing on macos

* common/realtime.py refactor

* Mypy: mypy passing on darwin

* Refactor: Removed else: pass statement

* Refactor: Removed unnecessary check

* added xattr to pyproject

* loggerd: switched to xatter module

* loggerd: removed unused module in xattr_cache.py

* UV: update uv.lock

* Update system/athena/athenad.py



* athenad: fixed blank lines

* loggerd: refactor of xattr_cache

* cleanup

---------



* fix getxattr no attribute on macOS

* try fixing missing ENOATTR on Linux

---------

Co-authored-by: Andrei Radulescu <andi.radulescu@gmail.com>
Co-authored-by: BrainLess <116778989+BrainLessPea@users.noreply.github.com>
2025-05-07 19:11:37 -07:00
Maxime Desroches aa1b790708 revert agnos 12 2025-05-06 21:38:50 -07:00
Andrei Radulescu bbf37ae5c7 Reapply "Mypy: Got passing on macos (#34591)" (#35126)
* Mypy: Got passing on macos (#34591)

* Mypy: Got mypy passing on macos

* common/realtime.py refactor

* Mypy: mypy passing on darwin

* Refactor: Removed else: pass statement

* Refactor: Removed unnecessary check

* added xattr to pyproject

* loggerd: switched to xatter module

* loggerd: removed unused module in xattr_cache.py

* UV: update uv.lock

* Update system/athena/athenad.py

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>

* athenad: fixed blank lines

* loggerd: refactor of xattr_cache

* cleanup

---------

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>

* fix getxattr no attribute on macOS

* try fixing missing ENOATTR on Linux

---------

Co-authored-by: BrainLess <116778989+BrainLessPea@users.noreply.github.com>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2025-05-06 21:09:59 -07:00
William Bonilla c5b75168e9 Fix test_update_returns_state to flatten Kalman output (#35083)
fix(tests): flatten Kalman filter output in test_update_returns_state
2025-04-28 13:30:16 -07:00
Cameron Clough 362ddfc0c7 ui: replace qt text window with raylib (#35064)
* remove qt text window

* use wrapper, render text window in thread

* add wait_for_exit method

* update imports
2025-04-25 14:17:47 +01:00
Cameron Clough 3ee2882093 ui: replace qt spinner with raylib (#35059)
* use raylib spinner

* remove qt spinner

* use wrapper, render spinner in thread

* english

* fix?

* match Qt font size more closely
2025-04-24 21:16:38 +01:00
Shane Smiskol abad7649d6 Rename experimental long (#35011)
* now alpha long

* fix

* rename param

* thought i did this

* debug

* debug

* more

* more

* moar

* Revert "moar"

This reverts commit e347106246583982e1be206485ad4d8eb944f817.

* Revert "more"

This reverts commit 0746e6d2e400ce1715614c62af84a9680c4d65c1.

* Revert "more"

This reverts commit 0db5a8792c01bb2d687f9152e6ba37fb76d4a453.

* Revert "debug"

This reverts commit 72f235b13727980e4331ff487998ddc7c7f7fb63.

* Revert "debug"

This reverts commit 7871b69f8c1bdffed386c2baac99542fcc50f7eb.
2025-04-14 17:17:59 -07:00
Kacper Rączy f237649a7a Reapply "Online lateral lag learning" (#34975)
* Online lateral lag learning (#34974)

This reverts commit b4cc9e68d1.

* pad to the best size for fft

* Fix static analysis

* Add typing

* Fix typing

* MAX_LAG

* Calculate cross correlation regardless if the points are valid

* Back to lagd

* Add lagd to process_config

* Lagd in test onroad

* Move lag estimator for lagd

* Remove duplicate entry from test_onroad

* Update process replay

* pre-fill the data

* Update cpu usage

* 25sec window

* Change the meaning of lateralDelayEstimate

* No newline

* Fix typing

* Prefill

* Update ref commit

* Add a unit test

* Fix static issues

* Time limit

* Or timeout

* Use mocker

* Update estimate every time

* empty test

* DT const

* enable RIVIAN again

* Update ref commit

* Update that again

* Improve the tests

* Fix static

* Add masking test

* Increase timeout

* Add liveDelay to selfdrived

* Add liveDelay to selfdrived in process_replay

* Fix block_avg restore after num_blocks

* regen most

* Update bolt

* Update ref commit

* Change the key name

* Add assert

* True weighted average
2025-04-09 22:31:07 +00:00
Kacper Rączy fc65a8ea8f paramsd: cache backwards compatibility (#34991)
* LiveParametersV2 key

* Param key

* Fix tests

* Remove old params if error

* Update exception message

* Update the test

* Add test for corrupted byte format in v1

* Rename params_reader => params
2025-04-08 02:23:11 +00:00
Kacper Rączy b4cc9e68d1 Revert "Online lateral lag learning" (#34974)
Revert "Online lateral lag learning (#34531)"

This reverts commit 1034dbd37c.
2025-04-01 21:44:30 +00:00
Kacper Rączy 1034dbd37c Online lateral lag learning (#34531)
* Add struct and services

* Proof of concept

* Use vego

* Process replay support

* Fix issues

* Max not min

* Remove print

* Make lag positive

* Points and updates

* Sliding cross corr

* Stuff

* Remove redundant param

* Move to lagd

* Update alert

* Something that works

* Report isEstimated

* Corr field

* Use skimage masked corr

* Fixes

* Move to lat_accel_loc

* Use the delay it in the model

* Mark as executable

* Add 0.2 to initial

* add install line

* Back to 5m window

* Move the import up

* corr window 120

* Show is estimated

* Sort messages

* New impl

* Fix

* Params

* Set initial_lag elsewhere

* Add param

* Rename Param

* Fix type isues

* More blocks

* Masked fft NCC implementation

* Remove package installation

* Use 20hz data and interp the correlation peak

* Move ncc code to separate function

* steering_saturated support

* Constants

* Add recovery buffer

* Fix static

* min_valid_block_count nad liveDelay

* lateralDelayEstimate

* Fix comm issues. Do estimations at 4hz

* Fix

* Display the estimate

* Increase the block size

* Cache every minute

* DEBUG flag

* Add progress to the alert

* Fix crash

* points only in debug

* Fix BlockAverage restoration

* Comment

* Move estimators into estimator directory

* Remove lagd

* Feed only relevant messages

* Rewrite as a set literal

* np.diag with P_initial

* Fix static

* Struct renames

* Update ref commit

* Make it nicer

* Some renames
2025-04-01 04:02:02 +00:00
Dean Lee e3ce984701 clutil: remove deprecated and unused code (#34793)
remove deprecated and unused code
2025-03-06 10:17:11 -08:00
Dean Lee 47f37d5fec remove numpy_fast.py (#34796) 2025-03-06 10:14:53 -08:00
Adeeb Shihadeh 701868d5cd firehose auto start + stats (#34747)
* don't need that

* stats

* lil more

* lil more

* cleanup

* tweaks

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-03-03 18:21:02 -08:00
Adeeb Shihadeh 54f1b00447 Log git source commit (#34749)
* Log git source commit

* strip

* fix trim

* no trim
2025-03-02 13:11:38 -08:00
Adeeb Shihadeh f90faf30bc bump to 0.9.9 2025-02-28 19:35:34 -08:00
Dean Lee 61bec65f32 common: add MovingAverage class for real-time windowed average calculation (#34569)
* add MovingAverage class for real-time windowed average calculation

* move to util.py
2025-02-21 14:47:21 -08:00
Adeeb Shihadeh 9881cf9c20 slightly more robust time validity check 2025-02-21 10:22:19 -08:00
Dean Lee 51ba8f796d params: separate parameter keys from params.cc (#34526)
seperate keys
2025-02-18 21:42:31 -08:00
Adeeb Shihadeh 27b5a727e3 Revert "Mypy: Got passing on macos (#34591)"
This reverts commit b09b48130e.
2025-02-16 08:50:46 -08:00
BrainLess b09b48130e Mypy: Got passing on macos (#34591)
* Mypy: Got mypy passing on macos

* common/realtime.py refactor

* Mypy: mypy passing on darwin

* Refactor: Removed else: pass statement

* Refactor: Removed unnecessary check

* added xattr to pyproject

* loggerd: switched to xatter module

* loggerd: removed unused module in xattr_cache.py

* UV: update uv.lock

* Update system/athena/athenad.py

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>

* athenad: fixed blank lines

* loggerd: refactor of xattr_cache

* cleanup

---------

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2025-02-16 00:05:52 -08:00
Adeeb Shihadeh d98a554b17 tici: set core affinity for all realtime processes (#34574) 2025-02-11 21:20:31 -08:00
Dean Lee 27c11eb597 athena upload: streaming file upload (#34559)
streaming file upload
2025-02-10 13:57:31 -08:00
Adeeb Shihadeh 9bc35c0919 Revert "athena upload: reduce memory usage and improve efficiency with streaming (#34528)"
This reverts commit 4c65f51a55.
2025-02-06 11:50:04 -08:00
Dean Lee 4c65f51a55 athena upload: reduce memory usage and improve efficiency with streaming (#34528)
* reduce memory usage and improve upload efficiency with streaming

* space
2025-02-04 13:10:13 -08:00
Adeeb Shihadeh 79e52ea83d FIREHOSE mode v0.1 (#34496)
* add toggle

* athena

* fix mypy
2025-01-29 15:31:15 -08:00
Adeeb Shihadeh cf83167ce6 add AdbEnabled param 2025-01-23 16:14:46 -08:00
Shane Smiskol 9ceb8cc324 swaglog: support np.bool_ (#34439)
fix np.bool_
2025-01-21 12:52:51 -08:00
Dean Lee 7a1bf26aa5 ui: prevent device pairing if no internet or system time is invalid (#34403)
* gate pairing device if system time is invalid

* update translations

* Check for internet connectivity

* Update selfdrive/ui/qt/widgets/prime.cc

* Update selfdrive/ui/qt/widgets/prime.cc

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2025-01-18 11:18:45 -08:00
Sammohana 8eebce75ac Getting rid of openpilot.common.numpy_fast (#34368)
* Got rid openpilot.common.numpy_fast

* fixed some data type erros

* importing numpy instead of importing specific functions

* fixing some numpy importing mistakes

* Update selfdrive/car/cruise.py

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2025-01-14 14:52:56 -08:00
commaci-public 4d714113d1 [bot] Update Python packages (#34376)
* Update Python packages

* time -> time_helpers

---------

Co-authored-by: Vehicle Researcher <user@comma.ai>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2025-01-13 09:38:17 -08:00
Adeeb Shihadeh d35ef3b3d5 swaglog: fix locale dependence (#34367) 2025-01-12 11:57:26 -08:00
Dean Lee 1068779294 modeld: properly release OpenCL context in __dealloc__ method (#34353)
release OpenCL context in __dealloc__
2025-01-12 10:15:26 -08:00
Adeeb Shihadeh 420d6f0d5e Quick GC pass heading into 2025 (#34330)
* first pass

* bye bye snpe
2025-01-05 15:10:24 -08:00
Adeeb Shihadeh b36db7810c tici: fix cpp device type (#34315)
fix cpp
2024-12-26 20:48:45 -08:00
Adeeb Shihadeh 9cf02ca8db Skip registration on newer devices (#34316) 2024-12-26 20:22:44 -08:00
ZwX1616 7352e612a2 dmonitoringmodeld: use cl transform without tinygrad (#34266)
* merge

* why

* self.buf_size

* 0.05 more than with tg due to copy

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-12-16 16:29:06 -08:00
Harald Schäfer f6885dcbec Revert Tinygrad (#34243)
* Revert "dmonitoringmodeld: use cl transform (#34235)"

This reverts commit 684b0b9d4d.

* Revert "load model before calling convert_fp16_to_fp32"

This reverts commit 31606a7d15.

* Revert "bump tinygrad"

This reverts commit 44f58ff758.

* Revert "Tinygrad runner (#34171)"

This reverts commit 7b5a4fbb03.

* Allow init buffer

* typo
2024-12-14 12:15:36 -08:00
ZwX1616 684b0b9d4d dmonitoringmodeld: use cl transform (#34235)
* needs cleanup

* only if tici

* bump tinygrad

* check width

* base modelframe

* .

* need to be args

* more cleanup

* no _frame in base

* tici only

* its DrivingModelFrame

* .6 is fair

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-12-14 00:44:15 -08:00
Adeeb Shihadeh 8f73bcffe4 micd: fix fake lagging warnings (#34158)
* micd: fix fake lagging warnings

* fix it properly

* simple
2024-12-05 13:35:44 -08:00
DevTekVE 8f7bbe4ee3 macos: use /tmp instead of /dev/shm (#34097)
* No /dev/shm on MacOS so we go to tmp instead given that the /dev/shm makes tests not really work fine on mac

* Cleanups

Fix cereal messaging tests

* keep msgq stock

More shm updates for macos

* typo

* fix return

* Semicolon...
2024-12-04 10:51:09 -08:00
Adeeb Shihadeh 1ec2c56b4e gc old DM init param 2024-12-03 14:59:28 -08:00
JaSpa99 dc73e6e2aa Make SimulatorBridge picklable (MacOS support) (#34023)
* remove unused

* make Params (un)serializable

* move initialization after spawn

_thread.lock objects can't be pickled
2024-11-14 10:25:50 -08:00
Shane Smiskol 44c889fa41 longcontrol: prevent integral windup when feedforward rises gradually (#33970)
* try

* messy

* simplify that

* clip instead

* clean up

* clean up

* update refs

* this is better for latcontrol

* Revert "update refs"

This reverts commit 1d2508237f76121248060614b0c5e6ecdfdc4daf.

* ref commit
2024-11-08 18:15:36 -08:00
Jason Young 17edc5f660 support for SecOC cars (#33689)
* bump opendbc

* support for SecOC cars

* missed this

* bump opendbc

* un-nest SecOC params

* gate saved key retrieval on IsReleaseBranch false

* bump opendbc

* bump opendbc

* bump opendbc to point of SecOC merge

* bump opendbc
2024-10-01 20:22:46 -04:00
Adeeb Shihadeh df9b8f0222 camerad: final debayer prep (#33598)
* prep part 4?

* cleanup

* little less magic

* reorg

* final touches

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-09-19 16:14:29 -07:00
Shane Smiskol 82f8db87f4 add longitudinal maneuvers (#33527)
* add longitudinal profiles

* stash

* unfortunately even longitudinalPlan causes circle

* add to process config

* reach target speed smoothly

* stash

* works

* clean up

* debug alert

* rename

* fix

* better text

* toggle via exp button

* try coming to a stop better, smoother target reaching

* closer to target

* revert controlsd migration

* add description to alert

* generate report from local logs

* hide bad maneuvers

* pdflike

* Revert "pdflike"

This reverts commit 6d4af1bf9be2e9e0798eaecf026a53d860da7613.

* try this

* use alert manager

* fix that check

* wat

* Revert "wat"

This reverts commit 93d0d27ab838d3f580d06ff212f380e0b912d599.

* some clean up

* rm

* cleanup

* move

* fix test

* more fix

* clean up

* fix that
2024-09-12 14:10:18 -07:00
Dean Lee b286a2f0e5 transformations: standardize parameter passing by reference (#33469)
pass parameters by reference
2024-09-07 17:06:57 -07:00
Kacper Rączy 236dffe400 locationd no GPS (#33029)
* Pose kf draft

old-commit-hash: 17dd4d576e597792f0e18826498c00076739f92b

* Fix it

old-commit-hash: 13ac120affe58fd22e871586ea5f4d335b3e9d2b

* Add translation noise

old-commit-hash: 166529cb612858c4ce80649367ac35b2b6007e1d

* Add gravity to acc

old-commit-hash: 8fcfed544b8e090ccc86b189c13bc03c6c190613

* Use pyx

old-commit-hash: 8e69e0baa0a4c43b4d0c22535711f296f05420aa

* Indent

old-commit-hash: 25b19a73644cdcb571ccf1a1d8acb88a0d066c67

* Reset function

old-commit-hash: ca5d2736da15e4fd6539f7268133320735b7c9cc

* Add device_from_ned and ned_from_device transformations

old-commit-hash: a60d25da0edc311e583549dc015fa595749fd4ae

* Fix rotations

old-commit-hash: d6d582f7f6d19a2bc2308dbcb0c9f81363e325b6

* kms

old-commit-hash: 681bc4e50374795ccc61422c3ce4ffb51389fce2

* Centripetal acceleration

old-commit-hash: 6e574506d27e5b76a04b2097d94efa4ca91ead71

* Rewrite draft

old-commit-hash: 4a2aad0146267460e5d30036c8cdb2bef94d1d7c

* Remove old locationd stuff

old-commit-hash: c2be9f7dbf22fb5cd29e437cd7891a7d52266fba

* Python process now

old-commit-hash: 83fac85f28c0b546b6965aafe1dd8a089e67f5b3

* Process replay fix

old-commit-hash: c44f9de98583c49dad0b22497869b3bb0266fcd9

* Add checks for timing and validity

old-commit-hash: aed4fbe2d00ca620e01a0e0ee99a4871c939de36

* Fixes

old-commit-hash: 3f052c658c16984a34915f38afdfbfd0fb19a267

* Process replay config fixes

old-commit-hash: 1c56690ee7ceb3c23c9ec2b2713352191212716e

* static analysis fixes

old-commit-hash: 6145e2c140ea9aa97e75069c3ddd82172cadc866

* lp in latcontrol

old-commit-hash: 9abf7359d68e794c69052724f3aca14b04dd3cca

* Fix SensorEvent name for acceleration

old-commit-hash: 91a1ad6c604727c9c898ba4aefe9478022b167fd

* Ignore sensor readings from segments with multiple imus

old-commit-hash: 1f05af63d6cc605ea98d7da0d727a5bd8d7819b0

* Update shebang

old-commit-hash: e3f2f5c10df3a4ba698421335bfeffc63d1a8797

* Replace llk with lp

old-commit-hash: 99b6c7ba08de6b703708fef0b8fd2d8cb24b92c0

* Refactor locationd scenario test

old-commit-hash: 7f5f788f071b7647e36f854df927cc5b6f819a84

* Add more debugging tools

old-commit-hash: 8d4e364867e143ea35f4bfd00d8212aaf170a1d1

* Param name update

old-commit-hash: 5151e8f5520f067e7808e3f0baa628fbf8fb7337

* Fix expected observations

old-commit-hash: d6a0d4c1a96c438fb6893e8b6ff43becf6061b75

* Handle invalid measurements

old-commit-hash: 549362571e74ad1e7ec9368f6026378c54a29adf

* Fix spelling

old-commit-hash: eefd7c4c92fb486452e9b83c7121d2599811852b

* Include observations in debug info too

old-commit-hash: 625806d1110b3bffe249cd1d03416f2a3f2c1868

* Store error instead of expected observation

old-commit-hash: 1cb7a799b67e56af4eddc6608d5b0e295f2d888c

* Renames

old-commit-hash: a7eec74640fc5cc7a5e86172d2087b66cb93d17d

* Zero the yaw

old-commit-hash: 96150779590fcb8ac50e8ffe8f8df03105983179

* New state_dot for orientation

old-commit-hash: c1456bf3a0c5af2f888aa6ff0b5ffc2e5516ddf7

* Fix the state transformations

old-commit-hash: 7cb9b91e2f99caa4ac3fb748c7f23bb8bf9f65db

* Update process in test_onroad

old-commit-hash: 854afab7c39ca7dec2b42974cecbb5310b82b617

* Test polling on cameraOdometry

old-commit-hash: a78e8b7d61618177f15c9892e2fa1e51620deca8

* Keep the copy of x and P returned from predict

old-commit-hash: 3c4159a6a7d7383265a99f3f78f8805d2fcfc8cd

* Remove polling again

old-commit-hash: f7228675c5fd2de5f879c4786859f1abcec27d68

* Remove locationd.cc

old-commit-hash: d7005599b2b178e688c3bd1959d6b69357d3a663

* Optim in _finite_check

old-commit-hash: 58ad6a06b9380960e9f69eb98663ddb97461e8d5

* Access .t once

old-commit-hash: d03252e75ed4cbdb49291a60c904568e6a3d3399

* Move the timing check to cam odo code path

old-commit-hash: 6a1c26f8c201e1feb601753f0cb7299cf981b47e

* Call all_checks only once

old-commit-hash: 373809cebf8d9db89d1ab00f4c8c933f33038e78

* Do not sort

old-commit-hash: 2984cd02c0ab76827b8c7e32f7e637b261425025

* Check sm.updated

old-commit-hash: 11c48de3f0802eb4783899f6a37737078dbf2da4

* Remove test_params_gps

old-commit-hash: 82db4fd1a876cc2402702edc74eba0a8ac5da858

* Increase tolerance

old-commit-hash: 927d6f05249d2c8ec40b32e2a0dcde8e1a469fb3

* Fix static

old-commit-hash: 2d86d62c74d5ac0ad56ec3855a126e00a26cd490

* Try separate sockets for sensors

old-commit-hash: 5dade63947ab237f0b4555f45d941a8851449ab1

* sensor_all_checks

old-commit-hash: e25f40dd6b37ee76cd9cc2b19be552baf1355ec3

* Fix static

old-commit-hash: 328cf1ad86079746b4f3fde55539e4acb92d285e

* Set the cpu limit to 25

old-commit-hash: 7ba696ff54c5d3bfa42e42624d124f2a1914a96d

* Make it prettier

old-commit-hash: cd6270dec80d8b9dac784ddd4767a1a46bcff4b7

* Prettier

old-commit-hash: 1b17931d23d37f299dad54139eaf283a89592bf5

* Increase the cpu budget to 260

old-commit-hash: 20173afb937a2609c8a9905aee0b2b093cb8bba4

* Change trans std mult. 2 works better

* Update ref commit

* Update ref commit
2024-09-04 11:54:57 +02:00