Commit Graph

108 Commits

Author SHA1 Message Date
Willem Melching 5c97a090b6 athena: skip duplicate upload requests (#25062)
* athena: skip duplicate upload requests

* cleanup

* keep simple

* just ignore
old-commit-hash: 7e187426c7f02b8d63bce394f08219307e5900ca
2022-07-07 18:00:07 +02:00
Adeeb Shihadeh d9d83b0225 Move camerad to system/ (#24836)
* mv camerad

* add hardware symlink

* fix unit tests
old-commit-hash: 6123ab3d1c901ed3763e1a7cb8e1aac3f6b8fda3
2022-06-19 14:43:49 -07:00
Adeeb Shihadeh 3c48bc47b7 Move a bunch of stuff to system/ part 3 (#24829)
* move swaglog.py

* timezoned

* logmessaged

* version.py

* fix linter
old-commit-hash: 0fce5d90459b77bf2cfa70f55f322f0e1fb8d01c
2022-06-11 23:19:27 -07:00
Adeeb Shihadeh 9a45a3b99b Move selfdrive/hardware/ to system/ (#24725)
* move hardware to system/

* fix mypy
old-commit-hash: 1139fe507b01f34de9714c99228f411558b44231
2022-06-11 16:38:24 -07:00
Maykon Pacheco fe24326066 test for the strip_bz2_extension method (#24826)
old-commit-hash: 1847a70a47b12ff5c7e7db9361f69d6d1e79a919
2022-06-11 11:08:24 -07:00
Willem Melching 753045c73d less TICI when not needed (#24698)
* less TICI when not needed

* fix process replay

* move reading voltages into hw abstraction layer

* Update selfdrive/hardware/tici/hardware.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/hardware/hw.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/hardware/base.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* rename init function

* Update selfdrive/athena/athenad.py

Co-authored-by: Robbe Derks <robbe.derks@gmail.com>

* Update selfdrive/boardd/boardd.cc

* Apply suggestions from code review

* Update selfdrive/thermald/thermald.py

* update ref

* fix alert width if all cameras are bad

* add ecam to test_loggerd

* bump cereal

* bump cereal

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Robbe Derks <robbe.derks@gmail.com>
old-commit-hash: f49a9c9fd2100a4f1401a93d88064ddbbbd935c1
2022-06-02 15:20:51 +02:00
Adeeb Shihadeh 402a0800dd athena: remove redirect detection for old prime (#24662)
This worked on old magenta, but doesn't work on new magenta
or blue prime.
old-commit-hash: e38b93fc31f3ff89a3806675bc090c2149434cf7
2022-05-26 16:09:22 -07:00
Dylan Herman 2dfba97175 add mypy check for return-any (#24379)
* add mypy check for return-any

* remove unused import

* typing

* remove unnecessary variable typing

* cleanup

* cleanup

* bump submodules

* small fixes

* only a problem on mac

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: d2eef1955c777ef6110c7a74d2a9ca9e4e1d958c
2022-05-12 14:59:00 +02:00
grekiki 7cd23af9b1 Mypy fixes for --check-untyped-defs (#24372)
more type fixes
old-commit-hash: 17e33978cdd9322918e9d5d388f7fa47aa30e9b7
2022-05-12 14:08:11 +02:00
Willem Melching be08693d5c delay bz2 compression from logging to uploading (#24392)
* remove log_name

* log without compression

* fix tests

* remove extension for bootlog

* another test fix

* uploader compresses

* also compress in athena

* only compress qlog

* more generic check in do_upload

* fix bootlog compression

* lower loggerd cpu usage

* dont link against bz2

* set core affinity to little cluster

* handle old files
old-commit-hash: 77a6f3d034a7b17d70b750b9ea454310f4b8edf1
2022-05-03 14:09:17 +02:00
grekiki cc943ac9a5 mypy: enable --warn-unreachable (#24282)
* Optional solves mypy assuming None is always None

* oops

* spaces

* Update registration.py

* fix local error

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: b8b15d5cf1d2507c89bbe0e3b1de1ff043d9078e
2022-04-21 18:27:45 +02:00
Adeeb Shihadeh 709e683bf4 athena: fix random test failure (#24278)
old-commit-hash: 0fbb7a4673ae02a5d2efafabfdccba194a0a8ebf
2022-04-20 13:20:50 -07:00
Willem Melching 85aa5e2706 uploader: azure returns 401 for files already uploaded (#24141)
old-commit-hash: e7f23ceca252fb4c62819b29c38eaaa62d66ce39
2022-04-05 20:09:05 -07:00
Willem Melching 34e0b26dc4 athena.upload_handler.upload_start log retry count (#24139)
old-commit-hash: 714efe828a10b1c9cf5f4b37b96af13c3d9333a1
2022-04-05 17:57:01 -07:00
Willem Melching 2eded18267 C3: detect metered Android hotspot (#23734)
* C3: detect metered networks

* show in ui

* fix text layout

* bump cereal

* revert ui changes

* set networkMetered

* add athena method

* add metered logging to uploader

* use in athena uploader

* remove param

* use networkmanager properties to set cell to unmetered

* fix indentation

* no need to check

* bump cereal

* review

* bump cereal
old-commit-hash: da5a0c41a00e4de076c2846ed54049c3dff3bc5a
2022-03-09 11:36:52 +01:00
Robbe Derks eda2f8c090 Fix unmetered upload over cell (fix #23811) (#23820)
old-commit-hash: a6214ff3b10114f727c9ec89dfa0ac275979f62f
2022-02-22 13:01:53 +01:00
Robbe Derks 7ef4b9115f Param to allow upload over cellular (#23811)
* param to allow upload over cellular

* brackets to be consistent

* move outside loop

* fix tests

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 5f64f1c08906a06889617e88aecc2f8ae74b4f21
2022-02-21 12:17:47 +01:00
Willem Melching 8788555f10 athena: expire items after 31 days (#23751)
* athena: expire items after 31 days

* add test
old-commit-hash: f03549c276e3877081273c713470e5184ef07ff5
2022-02-11 13:52:44 +01:00
Willem Melching a7fa889605 improve logging around uploading (#23750)
* improve logging around uploading

* use int

* use raw

* duplicate
old-commit-hash: 7765bc216677026d7b8919525eff310716fd3e7d
2022-02-11 12:58:41 +01:00
Willem Melching 48a9bd55c6 athena: specify network type for file uploads (#23687)
* athena: specify network type for file uploads

* add comment

* catch abort transfer

* fix tests

* put athena upload args in dict

* fix defaults

Co-authored-by: Joost Wooning <jwooning@gmail.com>
old-commit-hash: e9153fdb4b66c8175e4c0363ae74da6505348098
2022-02-09 17:28:49 +01:00
Willem Melching db3eef978f athena: add suport for download bandwith setting (#23727)
* athena: add suport for download bandwith setting

* clean ingress rules
old-commit-hash: 0d099e07aa1b16e2df5034835c1a6be3a721caa0
2022-02-08 12:07:11 +01:00
Willem Melching fb18d7b789 Add setUploadLimit method to athena (#23693)
* Add setUploadLimit method to athena

* add comments

* move to hw abstraction layer

* move to hw

* better errors
old-commit-hash: 4c406cdbe5c702d50a93410adf9d2d5b7f1b987a
2022-02-07 16:55:16 +01:00
Adeeb Shihadeh d17f80fa2d don't show registration error on pc
old-commit-hash: 52d4189e96e827d85ede52a557f7a8e940945a11
2022-01-29 18:10:41 -08:00
Adeeb Shihadeh eac8bd839a sentry improvements (#23627)
* sentry: filter out unregistered devices and PC

* add daemon name
old-commit-hash: 8b5b0ae3415e1ea44a1eecb6e70cec16b0fb1565
2022-01-26 11:44:10 -08:00
Ryan d947f4ff5b athena: unsuccessful upload handler requests. (#23620)
* Retry unsuccessful upload handler requests.

* test both cases

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 1d4191956b117483a32e5fe217ff78a612976a80
2022-01-26 15:23:59 +01:00
Robbe Derks 323febbf37 Statsd (#23413)
* device side of statsd

* need to start it

* enable in manager

* add sleep

* cleanup

* remove aggregates for now and standardize on industry terms

* manager needs main

* need to have a try/except

* atomic_write_on_fs_tmp does not work

* cleaner

* use dump

Co-authored-by: Willem Melching <willem.melching@gmail.com>

* one file at a time

* limit amount of files

* move to influx line protocol and cleanup

* needs to be a list

* fix timezone bug

* actually rate limit

* add to release

* normalized origin

* also log deviceType

* more stats

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 1b49ce6ec401966b0c56f62ea2b71209e9d4075b
2022-01-10 15:21:48 +01:00
Cameron Clough dbbdcb32d1 fix typo in athena/registration.py (#23434)
old-commit-hash: 1de0757c5e7dba1ea9696cbed80f871c8965c1c4
2022-01-06 16:11:32 -08:00
Joost Wooning ce94c27569 athena: uploads, show which items fail on error (#23412)
* athena: uploads, show which items fail on error

* fix upload-id

* no more 404

* Update selfdrive/athena/athenad.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 3ffebf4df5063e926ce530453f202cd7de4a6f99
2022-01-05 18:18:11 +01:00
Joost Wooning dd05972019 athena: methods for adding and cancelling multiple upload requests (#23366)
* multiple upload cancel

* multiple uploads athena method

* cleanup

* cleanup

* more cleanup

* isnt used

* fix test

* actually fix test
old-commit-hash: 47bb62b875fe2099a7a99482767bbd99b478f9e5
2022-01-04 17:01:33 +01:00
Ewout ter Hoeven f206ebd054 Pyupgrade 3.6: Update syntax with Python 3.6+ features (#23305)
Updated Python code with Python 3.6+ features:
- utf-8 encoding is now the default (PEP 3120)
- Replace list comprehensions by Generator Expressions (PEP 289)
- Replace yield loop by yield from (PEP 380)
- Remove the (object) subclass when defining a class
- Replace the IOError alias by OSError (PEP 3151)
- Define sets with curly braces {} instead of set()
- Remove "r" parameter from open function, which is default

Co-Authored-By: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-Authored-By: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>

Co-authored-by: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-authored-by: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>
old-commit-hash: 332f568a8241fba9459cb70c76840b9670e6993a
2021-12-24 11:18:39 -08:00
Willem Melching 4b13d224bd manage_athenad.py: add daemon argument to launcher
old-commit-hash: d09cc197d11efa2181ce68ef4212cb9df5ee285c
2021-12-15 17:03:12 +01:00
Devin Leamy 20a2b9d0a2 athena: persist upload queue to disk (#22659)
* revert submodules

* lowercase

* addressed comments

* add test for cancelled uploads

* formatting

* catch all exceptions

* handle empty param

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 8cb83b29a6b0cb1e5081651e96745c425e7e31f9
2021-12-14 13:29:17 +01:00
Shane Smiskol ef38c36435 version.py: make function names more clear (#23216)
* rename a few version functions

* and is_prebuilt

* and some formatting 😊
old-commit-hash: b745a14ff71622fe1fe4c9c043c7d7694d23dc08
2021-12-14 12:15:13 +01:00
Joost Wooning 969e181cff nav: athena destination with name and details (#23155)
old-commit-hash: 4a73445ee5b78549f28755110e068268d9e56cd8
2021-12-08 14:40:31 +01:00
Adeeb Shihadeh 4d43279023 registration: remove key generation (#23111)
* registration: remove key generation

* add test
old-commit-hash: 9ceb1a15eb87ba9ea5d7c8dbcc176f2466c82338
2021-12-03 22:58:34 -08:00
Willem Melching 28dd415759 Do not run code on version.py import (#23063)
* Do not run code on version.py import

* fix athena
old-commit-hash: f7c46c694985456e569ce2313b29e43607c62a12
2021-11-29 19:38:55 +01:00
Willem Melching e8b1e69ac1 remove broken athena test
old-commit-hash: 0e785021815cc573702ac537248bf53d24a7a18e
2021-11-11 17:32:26 +01:00
Willem Melching 8a0c57433e navigation: use mapbox proxy server (#22660)
* use mapbox proxy

* remove MapboxToken param

* do not log jwt

* remove import

* move to staging url

* put has_prime in ui state

* logic was flipped

* Update selfdrive/ui/qt/maps/map.h
old-commit-hash: cca07c0c730eb6e0d8dab0bde099a2ed706d14de
2021-10-26 18:59:17 +02:00
Willem Melching f78eacb3a5 athena: listUploadQueue, check None first
old-commit-hash: bde568205e5fe930d99fa4a7f16830077e962c15
2021-10-21 11:15:09 +02:00
Devin Leamy 5b5269cb06 athena: Do not show canceled upload items in listUploadQueue (#22627)
* do not show canceled upload items in listUploadQueue

* return item._asdict()

* athena: updated test for listUploadQueue
old-commit-hash: 12165c8865d8534e133af483cd15d7dfb3a0579a
2021-10-20 16:16:05 +02:00
Adeeb Shihadeh ad96d0d130 move athena tests to github actions (#22475)
* move athena tests to actions

* run

* fix timeout
old-commit-hash: de7a9e01160399228c30795769dc1ec6039cb8f4
2021-10-07 13:22:14 -07:00
Adeeb Shihadeh e9cd2a5561 need a jwt for those tests
old-commit-hash: f7bd264db888e2250a425c2a9249fc37ba539818
2021-08-19 15:48:26 -07:00
Adeeb Shihadeh 4da8cf4f31 athena: hardcode jwt for tests
old-commit-hash: 7e3d00206697ed552f8b3ae7018dd2c879906ee2
2021-08-19 14:19:14 -07:00
Greg Hogan 6cbe7f0749 athena: change test tearDown to setUp (#21935)
old-commit-hash: 1d64c18e499ee4ca51d8fca126488861e9943d4f
2021-08-14 22:27:22 -07:00
Greg Hogan b143411521 athena: split data into multiple frames (#21918)
old-commit-hash: a6debb999c1a4cd8e82ce71bdfb128d76f2923ef
2021-08-13 08:38:36 -07:00
Greg Hogan 9357587ace athena: add prefix param to listDataDirectory (#21906)
* athena: add prefix param to listDataDirectory

* only traverse directories that match prefix

* tests
old-commit-hash: 2dff84121e6966088c4941470a3d9fce12234d8f
2021-08-13 12:14:20 +02:00
Willem Melching dad760624a athena: return current upload status in listUploadQueue (#21903)
* return current upload item in listUploadQueue

* update progress on read

* move to file helpers

* use dict instead of global
old-commit-hash: 28823917ea87edebf66f568f15f965028b0a2ff5
2021-08-13 11:51:08 +02:00
Willem Melching 705988abb5 api_get: keep same api (#21838)
old-commit-hash: 267a0b343b0ad607ee0cddcf62de5e18966d4390
2021-08-03 15:33:41 +02:00
Dean Lee ceb40d8384 api: use API_HOST env variable everywhere (#21814)
* api base url to global constant

* update api/__init__.py
old-commit-hash: 543e019f715a7e485c7402d1e145dc8e81d43611
2021-08-03 13:49:49 +02:00
Willem Melching dd824ffe34 athenad: retry failed and timed out uploads (#21745)
* retry failed uploads

* test cleanup

* update comment

* also catch SSL error

* use defaults

* sleep in chunks
old-commit-hash: d5b6746ac55ec2947a7e7d14c12035168b73505d
2021-07-29 11:13:59 +02:00