Commit Graph

294 Commits

Author SHA1 Message Date
Willem Melching dfd521325d camerad: log debayer + yuv processing time (#23894)
* camerad: log debayer + yuv processing time

* bump cereal
old-commit-hash: aabd9a0dff9240fdf2beb2b83b9b35ec0d1bb0b6
2022-03-02 13:56:00 +01:00
George Hotz 10be5aad7a camerad: use open_v4l_by_name_and_index on c2 also (#23794)
* use open_v4l_by_name_and_index on c2 also

* remove open_v4l_by_name_and_index from qcom2
old-commit-hash: 51767c037b141d2100d51ce7406ead2a711676e6
2022-02-19 13:31:11 -08:00
George Hotz 5c766bea95 camerad: fast debayer on c2 cameras (#23795)
* fast debayer on c2 dcam

* add casts

* 128 local worksize on HDR debayer, 8 ms -> 3.5 ms

* width instead of saving rgb_width

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 92e9823cc0cf345d70530559e0e62d23c9b17be0
2022-02-18 20:15:20 -08:00
Adeeb Shihadeh 3e578e7409 camerad: adjust CL priority on comma three (#23790)
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: c6e0f1d84176cb4b65c81dff103b68b36862b700
2022-02-17 19:49:10 -08:00
George Hotz abfc6de831 Prepare for using the ISP (#23621)
* parse out isp packet with structs

* dsp mode

* support only driver / comments for yuv

* minor touchups

* DEBUG_FRAMES

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: cb6a68373b52f0bd867bef03d737b3125da7df74
2022-01-26 19:25:05 -08:00
Dean Lee 384e46644d camerad: cleanup main.cc (#23593)
* cleanup main.cc

* fix  ae_gray_test build errors
old-commit-hash: 61383dcc4289d3e976134495ad068fa4af5231a2
2022-01-21 16:19:50 -08:00
Dean Lee 120f43851d camerad: return 0 from main() (#23565)
old-commit-hash: e6a73e400ca0de3ce16e92d5f202caaff1e698cd
2022-01-18 14:21:30 +01:00
George Hotz 58a47e25bb camerad: Out of bounds memory write (#23534)
* parens were totally wrong

* cleaner
old-commit-hash: 7e83d9a61849950dcd7a4db3cd8ab7e9668dda2d
2022-01-14 15:22:28 -08:00
Ewout ter Hoeven a962365292 Convert format strings strings to f-strings (#23241)
* Convert all text strings to f-strings

Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.

Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.

F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale

* revert pyextra changes

* revert ublox.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 55390d273f36f49da3896e687ac5530f40c1d150
2021-12-16 14:58:17 +01:00
Willem Melching 0574f0f774 comma two: remove lens sag compensation (#23113)
old-commit-hash: 36db473babe445f0bae08065fb9d37197d5cd575
2021-12-14 13:29:59 +01:00
Dean Lee 88e6b990d4 LapConv: remove hardcoded FULL_STRIDE_X (#23200)
* fix wroing stride_x

* remove hardcoded rgb_stride
old-commit-hash: b519acd37bc4b160fc367bd690a326459f70109f
2021-12-13 15:53:44 +01:00
Dean Lee 4754da9fbc common_process_driver_camera: same parameters as process_thread_cb (#23202)
old-commit-hash: 5c6229f3df7ab499fa0a6121a327d9d538a2c847
2021-12-13 15:07:31 +01:00
Dean Lee dd7c226097 util: move all functions into util namespace (#23203)
old-commit-hash: 9decd3d8a20167de684322a458e28c8d1f746f83
2021-12-12 14:42:23 -08:00
Dean Lee 3ba7f99688 move struct LogCameraInfo into loggerd.h (#23129)
old-commit-hash: 480fb54926392b760944f5fafe7b4c96abee29ff
2021-12-04 21:08:55 -08:00
grekiki 9e4cc1f7ff Python: Replace more lists with generators (#23116)
* Replace lists with generators v2

* Replace set with {}

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Replace more set() with {}

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 6951b3271d1770297cb6bd53878463035ec0bf43
2021-12-03 22:57:19 -08:00
Dean Lee 9b0fcaeb84 rename yuv streams (#23071)
* rename yuv streams

* bump cereal

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 348d2d2b0d169b0d181a51eb0e23b1f9b8fc6793
2021-11-29 18:14:59 -08:00
Dean Lee 71c282d42a logreader: support reading from corrupt log (#23050)
* catch exception outside loop

* print decompress error

* add test case for corrupt log

* fix decompressbz2 stuck if log is corrupt

* recovered from corrupt data

* add output

* ass space

* std::endl

* override load(), load from buffer

* override FrameReader::load to load from the buffer

* replace NULL with nullptr

* fix test case for corrupt log

* Trigger Build

* check bzerror too

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: dbec761941c7599591fb59e9bb9f74b4f8cab2f7
2021-11-29 14:10:24 +01:00
Adeeb Shihadeh 94d30c6c5f camerad: remove camera frame stream (#23045)
* camerad: remove camera frame stream

* fix release files
old-commit-hash: 8160cf3c1b2726709db58219c33d1616b4a8acb5
2021-11-27 21:58:21 -08:00
Dean Lee e308bc0a3f framereader: replace swscale with libyuv, reduce cpu usage by half (#22992)
* use libyuv

* cleanup
old-commit-hash: 5ae51745097e3245c541456029fbc0fcce63a3c3
2021-11-26 14:41:14 +01:00
Willem Melching 935ce9edad navd: render map into VisionIPC (#22800)
* navd: render simple map

* render route

* offscreen rendering

* cleanup

* more cleanup

* render into visionIPC

* rename class

* split position update from route update

* stop broadcast if not active

* gate vipc server behind flag

* add python library

* faster

* no vipc from python

* put behind extras

* only send when loaded

* add glFlush just to be sure

* cleanup settings into helper function

* function ordering

* broadcast thumbnails

* put behind param

* adjust zoom level

* add route to python bindings

* revert that freq change

* add logging if map rendering is enabled

* use rlogs if available

* bump cereal
old-commit-hash: 5069852573128f40dd0174eb1d0f3f983367863d
2021-11-26 14:38:02 +01:00
Adeeb Shihadeh bef53929d7 camerad: lower YUV vipc buffer count to 40 on tici
old-commit-hash: 20d5c0c1cb73f848770bcf8b887eff9ccd43a442
2021-11-22 16:53:59 -08:00
Adeeb Shihadeh 4c7fbb1da0 Take camera snapshot using VisionIPC (#22070)
* vipc snapshot

* front and rear need to go

* remove SEND_*

* cleanup

* put that back

* fix duplicate code

* dont start camerad on pc

* fix rgb stride

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 0a1aaaa74cfb36a4c99b80698f0fd6c8b32e7044
2021-11-17 15:24:50 +01:00
Dean Lee 2f4b21b92f camera_qcom2: cleanup camera_open (#22085)
* cleanup camera_open

* remove static

* apply review
old-commit-hash: 3d0246eed55a5e1bcc97c56ac55afd12c98b56f6
2021-11-17 12:18:14 +01:00
Willem Melching 19deb98bc3 wshadow: fix comma three build
old-commit-hash: 45c506bb17e51d4f8592f74c75b9adcffa681201
2021-11-02 17:27:34 +01:00
Mayfield 43e1903073 enable wshadow (#22756)
* enable wshadow (#22714)

* fix replay

* more build fixes

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 5246f0231e6bfa572c0dd89c60e538910817faa2
2021-11-02 17:08:53 +01:00
Dean Lee a031b938b0 replay: improve segment download and merge (#22654)
* no-cache mode

* fix test cases build error

* space

* don't create cache dir in no-cache mode

* fix errors in test cases

* no_local_cache_

* set the number of connections by chunk_size

* use size_t instead of int64_t

* add test case for no-cache mode

* rename variables

* fix SIGSEGV

* cleanup

* faster decompressBZ2

* always decompress bz2

* add test cases

* prepare for python interface

* fix test cases build error

* continue

* camera_replay: cache remote file

* protected inheritance

* single option name

* TODO

* test_case for LogReader&FrameReader

* fix wrong require

* test case for FileReader

* cleanup test

* test:fix wrong filename

* check cached file's checksum

* fix mkdir permissions err

cleanup filereader

* remove initialize libav network libraries.

dd

* abort all loading if one failed

* cleanup tests

* use threadpool to limit concurrent downloads

* cache more segments

* merge 3 segments for replay

* one segment uses about 100M of memory

* use segments_need_merge.size()

* shutdown

* fix stuck if exit replay  before keyboard thread started

* load one segment at a time

* small cleanup

* cleanup filereader

* space

* tiny cleanup

* merge master

* cleanup test cases

* use util:create_directories

* cleanup framereader
old-commit-hash: 2b4a477fbc8fc02da5876aee4f955194916f19a3
2021-11-01 11:55:56 +01:00
Dean Lee 724fbebd42 camera_common: new class Debayer (#21854)
* new class Debayer

* merge master

* pass in cl_event

* needs define, class is not the same

* match clEnqueueCopyBuffer api

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: f7461ed9fc916113b20f5546bf41137f3328e524
2021-10-29 16:48:08 +02:00
Dean Lee 5e90a1bc79 config_isp: remove hardcoded buffer size (#22119)
old-commit-hash: b8be692c1a1ecfbaf1a70d06e4eb2e98d71508f9
2021-10-29 16:05:19 +02:00
Dean Lee c742e81f3e framereader: use swscale to handle the YUV conversion (#22710)
old-commit-hash: 921600b95c70d034f0c575673efce75533d3ba7a
2021-10-29 12:14:14 +02:00
Adeeb Shihadeh bb7751fdb0 don't set core affinity on PC (#22706)
old-commit-hash: 5b331fd6f5333e24a632839b64cdeb9129ef923b
2021-10-26 21:51:46 -07:00
Willem Melching 0f77dd4506 loggerd: set encodeIdx.valid if frame id is correct (#22634)
* loggerd: set encodeIdx.valid if frame id is correct

* check valid flag in test

* test cleanup

* bump cereal
old-commit-hash: 89f311714cf3606c386fd1de52b27e24780b8174
2021-10-26 16:00:43 +02:00
Adeeb Shihadeh 1dc45f978d camerad: allow core affinity setting to fail while offroad (#22664)
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: cd5bc565d38e66e31e2d9a91af331b4f13ef0b29
2021-10-22 20:57:21 -07:00
Adeeb Shihadeh b773f6711c loggerd: always run encoders (#22649)
* loggerd: always run encoders

* fix raw logger

* bump loggerd cpu usage
old-commit-hash: 76bd932cf555305a1e00d723bb5bb69d7fba39e9
2021-10-21 16:37:03 -07:00
Adeeb Shihadeh e761c4da22 fix snapshot camerad crash due to core affinity setting (#22648)
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: e9cb2104e6eb64a0a1d0e7dc342064523219dc74
2021-10-21 00:26:01 -07:00
Dean Lee 4f44e47aa1 replay: refactor FrameReader (#22438)
* decode from the previous keyframe after seek

* less memory

* some stream seems to contian no keyframes

* test random seek

* merge master

* continue

update test_cases

use fr

* merge master
old-commit-hash: 0189a19b8ebd18d9155328204da550c7ff1715f4
2021-10-18 14:19:23 -07:00
Adeeb Shihadeh 25cfa2b65a tici: affine loggerd to efficiency cores (#22581)
old-commit-hash: 84de248fa7cbee37ecb2fd6a9d4df50ba60a1e04
2021-10-18 13:21:24 -07:00
Dean Lee a8c5acb884 c++ replay: publish all frames in CameraServer (#22378)
* cameraserver

* support yuv

* init camera server in start()

* trigger ci
old-commit-hash: 1eb79d7a59a36e75ef2ccb1a8b1cea084784fc43
2021-10-04 16:45:28 +02:00
Dean Lee 5f63a9339c fix yuv420_to_jpeg: thumbnail_width & thumbnail_height must be aliged with 16 pixel. (#22287)
* aligned by 16px

* make buf big enough

* comment

* add comment

* comment
old-commit-hash: ae9305e7ff1bdba7afb94feb5e661ab2472321de
2021-09-23 13:20:10 -07:00
Dean Lee c0777f89bf webcam: add /usr/include/opencv4 to CPPPATH (#22320)
old-commit-hash: 5f9e0f7869eddcaff6e643cb7cf5031fcab82864
2021-09-23 11:43:51 -07:00
Willem Melching cfbee2bdca Script to extract thumbnails from segment (#22294)
* Script to extract thumnails from segment

* fix name

* whitespace
old-commit-hash: 8a836b7e1eb65b4a59a5be25f8b7dce9e2bedd0b
2021-09-21 10:14:16 -07:00
Dean Lee b7f3d92c9b camerad: added replay camera (#21241)
* start refactor

* remove camera_frame_stream from files_common

* rename camera_pc to camera_replay

* continue

* loop one segment

* rename cam_frame_id to stream_frame_id

* apply review

* continue

* more

* publish camera state

* cleanup

* cleanup

* better comment

* delete s->pm in cameras_close()

* add function getFrameCount

* refactor loop

* fix typo

* restore freame stream

* disable roadcam

* dd

* move file

* merge master

* fix test case

* add todo

* white space

* remove from release files

* add files back to relase

* move framereader back to ui/replay

* merge master

test_replay
old-commit-hash: 9b302488f98f7fcaf52a943e98de54196375b5b8
2021-09-19 14:43:54 -07:00
Dean Lee cd4e070b47 camerad: use the YUV buffer to create the jpeg thumbnail (#21936)
* yuv420_to_jpeg

continue

* add comments

* cleanup

* return ky::array
old-commit-hash: e0b5b4573ed34890a5e923012bf96fe311fd11dd
2021-09-17 17:00:21 -07:00
Dean Lee 4c17c06799 Handle EINTR for all syscalls that can return it (#21948)
old-commit-hash: f02c8d0e2c04e45ceeea9f2fe9851319d724a637
2021-08-31 16:22:56 -07:00
Dean Lee 94d1866ccd camera_qcom2: new functions device_acquire & device_config (#22077)
* new function device_acquire & device_config

* add space

* apply reviews

* use the specified struct for each command

* fix typo

* apply review

* remove static

* revert device_control
old-commit-hash: 0d3eec385cb65b9f962f6fbbc05e0f62b751239f
2021-08-31 07:21:40 -07:00
Dean Lee 5f64289206 camera_qcom2: use macros for link control operations (#22088)
old-commit-hash: ee5ef4dab25b638837fecae3fbb1c9acc615b65a
2021-08-30 22:10:18 -07:00
Dean Lee ecf3ee7603 loggerd: create encoder threads from cameras_logged (#21875)
old-commit-hash: d9baaf7eb90f0815c4d635d33244975ad71876d2
2021-08-30 19:33:00 -07:00
Dean Lee 7b9555df75 camera_qcom2: refactor open_v4l_by_name_and_index (#22067)
* refactor

* small cleanup

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: cf4ad99554ddbc7a96984f8876c733b4ee379656
2021-08-30 17:12:22 -07:00
Dean Lee 073898d8fd retry ioctls while errno == EINTR (#22054)
old-commit-hash: 31230e5b60aac7a7968a80d14e96f0a6fc9a9222
2021-08-30 16:07:34 -07:00
Dean Lee 6d054a0012 camera_qcom2: remove req_mgr_session_info from CameraState (#22059)
* remove req_mgr_session_info from CameraState

* handle should be int32_t
old-commit-hash: 37dda3d6bf98e86304bf19e88530cff60204fa45
2021-08-30 15:32:31 -07:00
Dean Lee 92627760de camera_qcom2: fix memory leaks in camera_open (#22061)
old-commit-hash: 7a80bf7fcc05b6955dd26dbbc1f6b44721f90781
2021-08-30 14:59:04 -07:00