Commit Graph

162 Commits

Author SHA1 Message Date
Adeeb Shihadeh 3512ac86eb pre-commit: always check all file sizes (#30050)
* pre-commit: always check all file sizes

* fix release
old-commit-hash: 425fa316d15e45595ecdd7d394ee28d7fb84e214
2023-09-25 22:22:48 -07:00
Adeeb Shihadeh e69eeb7344 move remaining big files into LFS (#30053)
old-commit-hash: e185e5ffcd165ee35e1bbd8c80631a89247a1dfc
2023-09-25 21:53:03 -07:00
Justin Newberry 52b437f6b6 CI: remove warning messages (#30013)
remove ci warnings
old-commit-hash: f223b66b7ba3f0b5f9cfb86b6350e8bb1d53d0f3
2023-09-22 19:59:38 -07:00
Justin Newberry 70a5d52887 CI: disable buildjet on PR's from forks (#29979)
* fix fork CI

* linter

* needed for docker build too

* slightly more clear

* you should have gone for the head
old-commit-hash: 6f0affbcd29d3ac84ca5db73d3eb12dbfcd6b79f
2023-09-20 13:15:23 -07:00
Adeeb Shihadeh e14996e079 CI: login to docker hub on BuildJet (#29961)
* CI: login to docker hub on BuildJet

* shell

* try quotes

* forward

* try this

* can't believe this is broken

* try

* fix process replay
old-commit-hash: 4b7b234b9e91c0a291e2ff3fae6e6b5f0cf02113
2023-09-18 15:53:16 -07:00
Adeeb Shihadeh ee0ddd7e26 Revert "CI: login to dockerhub on aarch64 runners (#29915)"
This reverts commit 28211fe8ada8997119c57c5619349332a406aa76.

old-commit-hash: 079a51cfdff8e79aa4e85d8e3c70285c0a7ab2e0
2023-09-17 15:16:07 -07:00
Kacper Rączy 4fd9b553be macOS: add prompt about support in setup script (#29939)
* Add prompt about the state of macos support in mac_setup

* Change words

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: f0b8ecd14f7aed3d3ad46c770881e3028ed9781f
2023-09-15 19:05:32 -07:00
Justin Newberry 79feec2438 CI: reduce unittest timeout (#29937)
reduce timeout
old-commit-hash: c6f1f36789427e115c533a5d5c319d4e58ff644a
2023-09-15 14:38:51 -07:00
Kacper Rączy 9525da02b6 CI: login to dockerhub on aarch64 runners (#29915)
* DOCKER_HUB_LOGIN command

* Add docker creds

* checkout runner name

* test again

* Use runner.name to distinguish between buildjet and GH runners
old-commit-hash: c9ec7bc2a3de1bd889d9034bd9336a4f65d36e23
2023-09-14 17:08:49 -07:00
Adeeb Shihadeh cc4cb7e078 don't run test twice
old-commit-hash: 7ef8898624cddb0f2afb3322e2691dd5212239f2
2023-09-14 14:08:24 -07:00
Vivek Aithal 244a2fc793 locationd: Unit tests for different scenarios (#29908)
* add different scenarios for locationd

* add locationd scenario tests to unit tests
old-commit-hash: e9edf1300ac486e1e1f79035b14a648d41f0d16d
2023-09-14 13:59:21 -07:00
Vivek Aithal 370d912053 liblocationd: No longer used (#29909)
remove liblocationd
old-commit-hash: 01011c6c1d8a211092d8f90658497f50ebd06bf1
2023-09-13 18:59:32 -07:00
Adeeb Shihadeh 1f8459d285 use pytest for cpp tests (#29833)
* add pytest-cpp package (#29828)

* add pytest-cpp package

* disable for now

* use in ci

* use pytest cpp

* use my branch

* add missing tests

* that doesn't make sense

* it was merged upstream

* replace TEST_RLOG_URL with a publicly accessible url

* PR Cleanup

* assert instead

* update poetry lock

* add comment about  pytest-cpp

---------

Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
Co-authored-by: Justin Newberry <justin@comma.ai>
Co-authored-by: deanlee <deanlee3@gmail.com>
old-commit-hash: 2c1dfc2f341b09381af2cc7872c4092df28dc9d5
2023-09-11 14:01:50 -07:00
Kacper Rączy fc4972539f docker: fix cl tag (#29858)
unset TARGET_ARCHITECTURE before running build
old-commit-hash: c417669cbdadbc3a4fc31073116987264aa0e2d4
2023-09-08 17:20:03 -07:00
Justin Newberry 5ac2ef0e3f CI: fix aarch64 cache (#29857)
was not passed in
old-commit-hash: d3e5f66fc19c15accd684d397b8e25823e431b47
2023-09-08 17:13:18 -07:00
Kacper Rączy 73c25cc743 docker: fix multiarch tag push (#29850)
Fix multiarch merge script
old-commit-hash: b09ec6bb5bd73b6623e686ebc94f8a316102ae11
2023-09-08 15:03:40 -07:00
Kacper Rączy f8c6ebd3ad docker: multiarch images (amd64/aarch64) (#29691)
* arch specific tags

* merge multiarch script

* arm64 build and merge jobs

* update tag script

* casual ci test

* change image to 2204

* docker -v test

* arch specific naming

* Remove test_buildjet

* build -arch images only when CURRENT_ARCH_BUILD is set

* support new tags in tag_multiarch

* Toggle CURRENT_ARCH_BUILD on in ci

* Docker common

* Fix condition

* Fix path to docker_common

* Add more stuff to common

* Add build_arm job

* composite job for building

* Run checkout before build composite

* add shell arg

* move timeout to selfdrive_tests

* TARGET ARCHITECTURE var

* Support TARGET_ARCHITECTURE in workflow

* Rewrite to Single build job with matrix

* Remove shebang from docker_common

* Attempt for fix build matrix

* Remove setup arch

* build matrix for docker push

* Use 2vcpu for arm build

* 2vcpu for docker_push too

* temporarly unlock docker_push

* Remove requirement for target arch when pushing

* Unset target architecture in docker_push cl

* fix sha tags

* Rename action to compile-openpilot

* move push_image line to tag_multiarch step

* arch suffix for scons cache

* cache_key_prefix for setup-with-retry

* Re-disable docker_push on non-master

* Add newlines
old-commit-hash: 1344a93a356e4734fedb5f717778a8b1a273c74b
2023-09-08 14:46:39 -07:00
Adeeb Shihadeh 13c3bf2cbd CI: increase model replay timeout
old-commit-hash: c96fe132e8e708d8a9cb2d4d6a3e8b9f1028018b
2023-09-08 10:41:54 -07:00
Justin Newberry 5b1e60f5da Pytest: enforce a default timeout (#29793)
* pytest: enforce default timeout

* disable that for now

* 30 second timeout on tests
old-commit-hash: b0a71d45536d6fe0c0e8365913c6ffbc64a58b1b
2023-09-07 17:53:51 -07:00
Justin Newberry 37e4b2cba9 CI: run unittests in parallel with xdist (#29756)
* parallel tests

* review suggesions

* add to pyproject

* add a bit more buffer on that

* fix rare athena issue

* remove from pypoetry
old-commit-hash: dde225221e8fa0063bb52786232be33b8374dfcd
2023-09-07 16:29:36 -07:00
Justin Newberry e468072f1f CI: retry setup on failure (#29785)
* try a setup action

* should be uses

* fix that formatting

* try conclusion

* continue on error

* try without hyphens

* only when failure

* make it optional

* continue on error

* those don't fail anymore

* what about 3 failures

* remove stuff for debugging

* cleanup

* review suggestions

* change that too

* fix pj
old-commit-hash: 917f71d4460bf5503190ade72a3fa760b64dca66
2023-09-06 11:11:15 -07:00
Adeeb Shihadeh 53e0f5eb49 pytest: use a clean environment for all tests (#29788)
* pytest: use a clean environment for all tests

* rm that

* fix pj

* put build back

* fix params

* fix that

* handle no key

* that was removed

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: 7b6afbc1623b8c159a92f6a5a951fb4d688045ee
2023-09-05 18:52:40 -07:00
Justin Newberry 839e72de70 CI: use pytest for unittesting (#29709)
* switch to pytest

* static analysis

* fix the helpers

* static analysis

* remove that

* more parallel test

* cleanup

* static analysis

* durations min and tolerance

* reduce those iterations

* sleep time could be decreased too

* in case it's negative

* try using caching

* remove parallelization for first PR

* PR Cleanup

* reduce diff

* fix that

* bump panda
old-commit-hash: ca435d344714f08276f9f9da5039f730987f31fa
2023-09-04 11:54:35 -07:00
Adeeb Shihadeh 0107dfb391 don't rerun those here
old-commit-hash: 75081b2e54134527735b1b5a7b000f452d180093
2023-08-31 18:34:20 -07:00
Justin Newberry 47eb055283 CI: fix proc replay download cache permissions (#29636)
chmod that
old-commit-hash: ab05bff07ded3d3cd932c59beaaa75c698195692
2023-08-27 09:45:16 -07:00
Adeeb Shihadeh 68dd600872 CI: use buildjet for process replay (#29277)
* squash for pr

* backoff to 4vcpu

* review suggestions

* simplify

---------

Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: d35beffbcec1524ff4c4096079208b109fede3f0
2023-08-25 18:50:21 -07:00
Dean Lee 00f900e8c4 common/tests: combine multiple test binaries into one (#29623)
old-commit-hash: 2709cd6538363e439b3de39a9f4df05619dfbaa0
2023-08-25 10:35:39 -07:00
Justin Newberry e128aedc63 CI: move cache dir to workspace directory (#29622)
* comma dir

* mkdir

* fix issues

* test

* fix that

* fix

* do the other ones too

* missed that

* fix

* fix pr comments by moving into two different directories

* fix location

* fix that

* only change current

* fix
old-commit-hash: c0b9458a45f208c9773f859169a7de56d19b1bec
2023-08-25 01:41:04 -07:00
Adeeb Shihadeh 8f9dcca26a CI: workaround syntax error in azure lib (#29618)
old-commit-hash: e8638280518938e7029c576be421e7fc35ddc695
2023-08-24 17:18:11 -07:00
Justin Newberry 4063a92683 CI: use GITHUB_ENV for env variables (#29616)
export doesn't work across steps
old-commit-hash: 90cb417caa049261797449d7e75eea01093e073d
2023-08-24 17:03:25 -07:00
Justin Newberry 630b440536 CI: fix docker push (#29614)
* fix those

* fix docker push too

* dont need that twice
old-commit-hash: ede9f2cda5500d9e094e89a32518c88b56c3335b
2023-08-24 16:30:33 -07:00
Justin Newberry 6847f819d1 CI: use scripts for docker build (#29599)
* use scripts

* remove common

* do login in the workflow

* separate step

* cleanup

* one script is cleaner

* keep it consistent

* review suggestions

* move in ci too

* PR cleanup
old-commit-hash: 3d9f238bf9a1f27c5a9a44df1b0f116182829be0
2023-08-24 16:09:08 -07:00
Adeeb Shihadeh 61cf541322 Minimal build (#29600)
* add build flag for minimum build

* add minimal build

* update

* fix pj

* bump opendbc

* update subaru
old-commit-hash: 4917e22615adbfcc9b3c3ec89b00d240e0b48883
2023-08-24 14:58:30 -07:00
Justin Newberry de7257c22c CI: Normalize file permissions (#29584)
* normalize file permissions

* comment

* add comment

* remove that

* shell

* should be on the top
old-commit-hash: da7504e1e72ac09063901361377a64ec074efe8f
2023-08-23 23:50:51 -07:00
Justin Newberry 7a532e02f3 CI: switch to --cache-to type=inline for buildkit cache (#29575)
this is cleaner
old-commit-hash: 2fdde9b77d5d830277500d6841f32a356544718a
2023-08-23 17:41:33 -07:00
Adeeb Shihadeh b38403f2ca Revert Qt MOC hacks (#29528)
* Revert "CI: prevent moc from running in cache population step (#29401)"

This reverts commit 34634b95aa15dc608315016c40857f3e16ca4ff9.

* Revert "SCons: respect cache read only (#29437)"

This reverts commit 77cb4da6381fdf94ea98c55d6fcc30db4d24dcf6.

* don't change moc path
old-commit-hash: 91dc8457ae8e9b188c8fe613be5b3aa2e86be276
2023-08-22 11:16:58 -07:00
Kacper Rączy 0c17a4f4d0 CI: prevent moc from running in cache population step (#29401)
* Do not remove moc_files from cache

* Add cache cleanup script

* Use cache cleanup script

* Dry-run in cache population command

* Add newline

* Temporarly disable master only constraint for cache cleanup step

* Add test lines

* one more

* Fix shebang

* Remove debug lines

* Bring back master branch constraints
old-commit-hash: d1aedccfbe3cb880ad54adea578f91c04857d81b
2023-08-22 10:47:56 -07:00
Justin Newberry 3e6ce7d6cd CI: use xdist for car model tests (#29514)
* bump panda

* try multithreaded for car models

* need pytest-cov

* upgraded to poetry 1.5.1 and regenerated
old-commit-hash: 3492e80d19a4395d9481076c916f9ab96a8fb004
2023-08-21 20:51:27 -07:00
Kacper Rączy 1d47429f8e Setup cleanup vol. 2 (#29456)
* code from setup cleanup #29419

update ci

update ci

export variables for mac os

check env

what is in the profile script?

install pyenv the same everywhere

temporarily disable brew cache

temporarily disable brew cache

try fixing pyenv virtualenv-init

try this

inject shims path earlier

try alternate install method

switch back to brew install

try         eval "$(pyenv init --path)"

add eval "\$(pyenv init --path)"

* Add local .env file to the cache list

* Change .env paths

* debug line

* order?

* remove .env

* Add .venv to caches

---------

Co-authored-by: Greg Hogan <gregjhogan@gmail.com>
old-commit-hash: 0f7be5b1e6efa04c59ce2a45afacd650b2e4d417
2023-08-21 13:30:48 -07:00
Dean Lee 97631ec362 common: C++ RateKeeper (#29374)
* c++ RateKeeper

* add to files_common

* use util::random_int

* improve monotor_time

* remove ~ratekeeper
old-commit-hash: 3eef63af9b4f4a25bc4c3a0d6ad450bba9f62205
2023-08-20 21:27:02 -07:00
Adeeb Shihadeh bd27be2b78 move mypy config into pyproject.toml (#29501)
* move mypy config into pyproject.toml

* rm reference
old-commit-hash: ff068dd722b31078b9356d3f55e4898cf00a89cb
2023-08-20 21:08:04 -07:00
Harald Schäfer 3022f06e1e update process replay timeout (#29478)
* Always dict differ

* Update selfdrive_tests.yaml

* bump up more

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: bc4b75822a609e6897058bc83688c84004f29093
2023-08-20 17:24:33 -07:00
Justin Newberry 1988b4805e Revert "setup cleanup (#29419)" (#29451)
This reverts commit bc9c60a3f955bb5b96dc41e6072cab47191f8896.
old-commit-hash: 1944945d9dfee6f42a875b119a5cf60daf8ade09
2023-08-17 13:23:06 -07:00
Greg Hogan cce2c732c6 setup cleanup (#29419)
* setup cleanup

* update ci

* update ci

* export variables for mac os

* check env

* what is in the profile script?

* install pyenv the same everywhere

* temporarily disable brew cache

* temporarily disable brew cache

* try fixing pyenv virtualenv-init

* try this

* inject shims path earlier

* try alternate install method

* switch back to brew install

* try         eval "$(pyenv init --path)"

* add eval "\$(pyenv init --path)"
old-commit-hash: ad32816fd6bdd744e7f596209822707494d3ba5e
2023-08-17 08:51:16 -07:00
Kacper Rączy 4096be66e4 macOS: fix GH actions cache (#29399)
Add lz4 and xz to dep cache exceptions
old-commit-hash: ae649ec68d670cb2d90773906665695807e4dc2e
2023-08-14 16:13:35 -07:00
Adeeb Shihadeh e76af64065 CI: enable PYTHONWARNINGS=error (#29364)
* CI: enable PYTHONWARNINGS=error

* fixes

* more ignore

* more modeld ignore
old-commit-hash: b11b14aff86a4c993bf5342ee37af169788c9026
2023-08-12 17:15:16 -07:00
Adeeb Shihadeh b3f734283d remove xx dependencies (#29263)
* rm xx deps

* mv that

* fix

* fix a path

* relock

* relock

* update

* update sentry

* update

* revert sentry

* Update lock

* Dummy pip package

* Not needed with poetry

* dont need dummy

---------

Co-authored-by: Bruce Wayne <harald.the.engineer@gmail.com>
old-commit-hash: 0ced56b2ea7e0800f309c4e77d9a78bb90d89fb6
2023-08-11 20:05:44 -07:00
Justin Newberry 20f6a25a1c CI: move other caches out of /tmp (#29348)
* also move other caches

* macos can stay
old-commit-hash: 2482b58968df1faa509a9043b71d164353284146
2023-08-11 17:41:23 -07:00
Justin Newberry afa33bee18 CI: move cache to home directory (#29347)
move cache to home dir
old-commit-hash: ace4d78e315f48655808e038c96c35b89096b654
2023-08-11 15:45:38 -07:00
Justin Newberry f8cd844e7c CI: use default caching action (#29339)
old-commit-hash: a0f399677e365a2b7c9913c76ca72b1a22814825
2023-08-11 14:03:10 -07:00