Commit Graph

269 Commits

Author SHA1 Message Date
Dean Lee a031b938b0 replay: improve segment download and merge (#22654)
* no-cache mode

* fix test cases build error

* space

* don't create cache dir in no-cache mode

* fix errors in test cases

* no_local_cache_

* set the number of connections by chunk_size

* use size_t instead of int64_t

* add test case for no-cache mode

* rename variables

* fix SIGSEGV

* cleanup

* faster decompressBZ2

* always decompress bz2

* add test cases

* prepare for python interface

* fix test cases build error

* continue

* camera_replay: cache remote file

* protected inheritance

* single option name

* TODO

* test_case for LogReader&FrameReader

* fix wrong require

* test case for FileReader

* cleanup test

* test:fix wrong filename

* check cached file's checksum

* fix mkdir permissions err

cleanup filereader

* remove initialize libav network libraries.

dd

* abort all loading if one failed

* cleanup tests

* use threadpool to limit concurrent downloads

* cache more segments

* merge 3 segments for replay

* one segment uses about 100M of memory

* use segments_need_merge.size()

* shutdown

* fix stuck if exit replay  before keyboard thread started

* load one segment at a time

* small cleanup

* cleanup filereader

* space

* tiny cleanup

* merge master

* cleanup test cases

* use util:create_directories

* cleanup framereader
old-commit-hash: 2b4a477fbc8fc02da5876aee4f955194916f19a3
2021-11-01 11:55:56 +01:00
Dean Lee 724fbebd42 camera_common: new class Debayer (#21854)
* new class Debayer

* merge master

* pass in cl_event

* needs define, class is not the same

* match clEnqueueCopyBuffer api

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: f7461ed9fc916113b20f5546bf41137f3328e524
2021-10-29 16:48:08 +02:00
Dean Lee 5e90a1bc79 config_isp: remove hardcoded buffer size (#22119)
old-commit-hash: b8be692c1a1ecfbaf1a70d06e4eb2e98d71508f9
2021-10-29 16:05:19 +02:00
Dean Lee c742e81f3e framereader: use swscale to handle the YUV conversion (#22710)
old-commit-hash: 921600b95c70d034f0c575673efce75533d3ba7a
2021-10-29 12:14:14 +02:00
Adeeb Shihadeh bb7751fdb0 don't set core affinity on PC (#22706)
old-commit-hash: 5b331fd6f5333e24a632839b64cdeb9129ef923b
2021-10-26 21:51:46 -07:00
Willem Melching 0f77dd4506 loggerd: set encodeIdx.valid if frame id is correct (#22634)
* loggerd: set encodeIdx.valid if frame id is correct

* check valid flag in test

* test cleanup

* bump cereal
old-commit-hash: 89f311714cf3606c386fd1de52b27e24780b8174
2021-10-26 16:00:43 +02:00
Adeeb Shihadeh 1dc45f978d camerad: allow core affinity setting to fail while offroad (#22664)
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: cd5bc565d38e66e31e2d9a91af331b4f13ef0b29
2021-10-22 20:57:21 -07:00
Adeeb Shihadeh b773f6711c loggerd: always run encoders (#22649)
* loggerd: always run encoders

* fix raw logger

* bump loggerd cpu usage
old-commit-hash: 76bd932cf555305a1e00d723bb5bb69d7fba39e9
2021-10-21 16:37:03 -07:00
Adeeb Shihadeh e761c4da22 fix snapshot camerad crash due to core affinity setting (#22648)
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: e9cb2104e6eb64a0a1d0e7dc342064523219dc74
2021-10-21 00:26:01 -07:00
Dean Lee 4f44e47aa1 replay: refactor FrameReader (#22438)
* decode from the previous keyframe after seek

* less memory

* some stream seems to contian no keyframes

* test random seek

* merge master

* continue

update test_cases

use fr

* merge master
old-commit-hash: 0189a19b8ebd18d9155328204da550c7ff1715f4
2021-10-18 14:19:23 -07:00
Adeeb Shihadeh 25cfa2b65a tici: affine loggerd to efficiency cores (#22581)
old-commit-hash: 84de248fa7cbee37ecb2fd6a9d4df50ba60a1e04
2021-10-18 13:21:24 -07:00
Dean Lee a8c5acb884 c++ replay: publish all frames in CameraServer (#22378)
* cameraserver

* support yuv

* init camera server in start()

* trigger ci
old-commit-hash: 1eb79d7a59a36e75ef2ccb1a8b1cea084784fc43
2021-10-04 16:45:28 +02:00
Dean Lee 5f63a9339c fix yuv420_to_jpeg: thumbnail_width & thumbnail_height must be aliged with 16 pixel. (#22287)
* aligned by 16px

* make buf big enough

* comment

* add comment

* comment
old-commit-hash: ae9305e7ff1bdba7afb94feb5e661ab2472321de
2021-09-23 13:20:10 -07:00
Dean Lee c0777f89bf webcam: add /usr/include/opencv4 to CPPPATH (#22320)
old-commit-hash: 5f9e0f7869eddcaff6e643cb7cf5031fcab82864
2021-09-23 11:43:51 -07:00
Willem Melching cfbee2bdca Script to extract thumbnails from segment (#22294)
* Script to extract thumnails from segment

* fix name

* whitespace
old-commit-hash: 8a836b7e1eb65b4a59a5be25f8b7dce9e2bedd0b
2021-09-21 10:14:16 -07:00
Dean Lee b7f3d92c9b camerad: added replay camera (#21241)
* start refactor

* remove camera_frame_stream from files_common

* rename camera_pc to camera_replay

* continue

* loop one segment

* rename cam_frame_id to stream_frame_id

* apply review

* continue

* more

* publish camera state

* cleanup

* cleanup

* better comment

* delete s->pm in cameras_close()

* add function getFrameCount

* refactor loop

* fix typo

* restore freame stream

* disable roadcam

* dd

* move file

* merge master

* fix test case

* add todo

* white space

* remove from release files

* add files back to relase

* move framereader back to ui/replay

* merge master

test_replay
old-commit-hash: 9b302488f98f7fcaf52a943e98de54196375b5b8
2021-09-19 14:43:54 -07:00
Dean Lee cd4e070b47 camerad: use the YUV buffer to create the jpeg thumbnail (#21936)
* yuv420_to_jpeg

continue

* add comments

* cleanup

* return ky::array
old-commit-hash: e0b5b4573ed34890a5e923012bf96fe311fd11dd
2021-09-17 17:00:21 -07:00
Dean Lee 4c17c06799 Handle EINTR for all syscalls that can return it (#21948)
old-commit-hash: f02c8d0e2c04e45ceeea9f2fe9851319d724a637
2021-08-31 16:22:56 -07:00
Dean Lee 94d1866ccd camera_qcom2: new functions device_acquire & device_config (#22077)
* new function device_acquire & device_config

* add space

* apply reviews

* use the specified struct for each command

* fix typo

* apply review

* remove static

* revert device_control
old-commit-hash: 0d3eec385cb65b9f962f6fbbc05e0f62b751239f
2021-08-31 07:21:40 -07:00
Dean Lee 5f64289206 camera_qcom2: use macros for link control operations (#22088)
old-commit-hash: ee5ef4dab25b638837fecae3fbb1c9acc615b65a
2021-08-30 22:10:18 -07:00
Dean Lee ecf3ee7603 loggerd: create encoder threads from cameras_logged (#21875)
old-commit-hash: d9baaf7eb90f0815c4d635d33244975ad71876d2
2021-08-30 19:33:00 -07:00
Dean Lee 7b9555df75 camera_qcom2: refactor open_v4l_by_name_and_index (#22067)
* refactor

* small cleanup

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: cf4ad99554ddbc7a96984f8876c733b4ee379656
2021-08-30 17:12:22 -07:00
Dean Lee 073898d8fd retry ioctls while errno == EINTR (#22054)
old-commit-hash: 31230e5b60aac7a7968a80d14e96f0a6fc9a9222
2021-08-30 16:07:34 -07:00
Dean Lee 6d054a0012 camera_qcom2: remove req_mgr_session_info from CameraState (#22059)
* remove req_mgr_session_info from CameraState

* handle should be int32_t
old-commit-hash: 37dda3d6bf98e86304bf19e88530cff60204fa45
2021-08-30 15:32:31 -07:00
Dean Lee 92627760de camera_qcom2: fix memory leaks in camera_open (#22061)
old-commit-hash: 7a80bf7fcc05b6955dd26dbbc1f6b44721f90781
2021-08-30 14:59:04 -07:00
Dean Lee 75f686f728 camera_qcom2.cc : fix wrong LOGD order (#22072)
old-commit-hash: 862f93f7f13ea8d4a3fabf30f97f7f624358b22b
2021-08-30 09:21:35 -07:00
Dean Lee 88878cc2cd qcom2/enqueue_buffer: use macro CAM_MEM_FLAG_HW_READ_WRITE (#22060)
old-commit-hash: 0f4227f42b2a3610d6e26d5ad4f249fbf77f7dda
2021-08-28 17:43:50 -07:00
Dean Lee 2658ef5243 camera_qcom2.cc: use V4L_EVENT_CAM_REQ_MGR_EVENT (#22057)
old-commit-hash: ffb1dd019ecad9f2230cea501cec9007229ab102
2021-08-28 14:49:56 -07:00
Dean Lee eb17b4403e camera_qcom2: remove unused variables from MultiCameraState (#22058)
old-commit-hash: df6432c8cda96b0fe2d8e7d81a23295552fbc482
2021-08-28 14:44:30 -07:00
Adeeb Shihadeh 277bd43b15 Revert "tici: give loggerd rt priority (#21503)"
This reverts commit 0a50cf851e79342cf3db01033bc91516c504cbea.

old-commit-hash: 54a4eeb0fccd92c3b2f872175cfedf136a1cfc71
2021-08-22 16:57:53 -07:00
Adeeb Shihadeh 5c1d1ebe76 tici: give loggerd rt priority (#21503)
* give loggerd rt priority

* assert

* little more

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 7e2752c878a9e8c38639e4caa234bcabb979db2e
2021-08-20 21:27:31 -07:00
Dean Lee 5d77755f18 camerad: remove unused macros and global (#21970)
old-commit-hash: e646febaaebca8440510824a0b73c9281bf96f3e
2021-08-19 14:49:16 -07:00
Dean Lee f32b1655e4 cameras_qcom2.cc: add a missing newline (#21956)
old-commit-hash: 74ecbbed638c390beb50d3c932779492a5f0af97
2021-08-18 10:37:05 +02:00
Willem Melching ae4f3c98d5 refactor loggerd: trigger rotate in encoder thread (#21860)
* trigger rotate in eoncode thread

* rotate in time

* lgtm

* dcam trigger rotate on C3

* check trigger rotate field

* Use >=

* add rotator thread

* set cnt to 0

* test encodeIdx is increasing across segments

* test both segmentId and encodeId

* fix encodeIdx

* no thread needed

* add log for failed to encode

* stricter C3 test, dont check start encodeId on C2 dcam

* only update last_camera_seen_tms when considered for rotate

Co-authored-by: deanlee <deanlee3@gmail.com>
old-commit-hash: a39873872de5b2239e4b89119974e5aa961eba48
2021-08-06 11:10:33 +02:00
arfy slowy 63b3d6024b fix: spelling typos (#21861)
* fix: typo spelling grammar

* revert

* Update selfdrive/locationd/calibrationd.py

* more revert

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: d74def61f88937302f7423eea67895d5f4c596b5
2021-08-05 12:05:49 -07:00
Willem Melching 9038e1b86f C3 AE improvements (#21698)
* AE improvements

* boardd: also use gain to compute IR brightness

* use default rect for DM autoexposure

* more smooth

* whitespace

* camerad cpu usage

* hcg on is slightly better for noise

* hysteris around high conversion gain

* improve i2c timing

* use AB contexts

* HCG default off

* run at 10 hz

* stay in context A for now

* remove ae thread

* wait till next frame is started before sending i2c

* back at 20fps

* add comment and filter across 3 evs

* remove context switch code

* tuning

* recomened without HCG is 0.8x
old-commit-hash: 25de93d133c386c353155c2b782e3807c46150cd
2021-08-05 16:34:36 +02:00
Dean Lee 94d534afea hw.h: move Hardware-related paths to namespace Path (#21826)
* refactor getenv and path

* apply review
old-commit-hash: fefc70b4cab71a11b3c5ed1b8680606ee2013a48
2021-08-04 13:29:03 +02:00
Adeeb Shihadeh 4d0543d92d 100 yuv buffers for all
old-commit-hash: 4c0caf7f7ea3a0127e6b573a78850004aab8e187
2021-07-26 14:02:49 -07:00
Willem Melching d479b9a5de Revert AE changes (#21682)
* Revert AE changes

* keep YUV buffer count
old-commit-hash: b6b0393e2dc3aba054d55884fcfef192b2a28694
2021-07-22 11:22:19 -07:00
Willem Melching 7a9659a9f7 AE: revert changes to min grey target (#21648)
old-commit-hash: b35f6871cb9ed2049ce1f31fcaeb1914677b8177
2021-07-19 10:37:43 +02:00
Willem Melching afd96fb083 only 100 yuv buffers on C2 (#21622)
old-commit-hash: 0d9e7ad24149217cf9e5e215cc5b60a60a77231c
2021-07-16 11:59:30 +02:00
Willem Melching 6dc20d7c75 tici: AE sane default values (#21610)
* tici: AE sane default values

* set exposure time

* remove comment

* set dc gain variable

* whitespace
old-commit-hash: 3ad40963cc5d30d4afd9693653d1f10a9bee61e8
2021-07-15 16:11:27 +02:00
Willem Melching 7198224cf2 tici: improve AE (#21493)
* simplify histogram calculation

* add debug prints

* faster AE

* revert that

* lower log level

* use minimum recomended gains from datasheet

* use define

* fix min gain

* dynamic grey target

* fix build

* add low pass

* more stable rounding

* increase max gain

* always enable HCG

* correctly set min ev

* fix analog gain

* remove unused variable

* fix build

* move recomended gain into cost function

* tweak cost function a bit

* small LPF on desired ev

* fix typo

* lower ts

* clean up defines

* that was used in common

* add cast

* less cost is fine

* fix tests

* whitespace

* filtering is broken

* put back

* clip gain changes instead

* cost function tweaking

* Only go below recomended gain when absolutely necessary

* small penalty on changing gains
old-commit-hash: 63c9b5606358c435b7291a1b35ee4faf6a7870ba
2021-07-13 23:33:50 +02:00
Adeeb Shihadeh 92e236e7ad increase yuv buffer count to hold 5s (#21528)
* increase yuv buffer count to hold 5s

* bump cereal
old-commit-hash: 4738fe0e45a9fded16333b08d27538aeedf044cf
2021-07-10 14:46:20 -07:00
Willem Melching e0378d41cb Refactor camerad exposure logging (#21506)
* simplify gain logging

* clean up metadata

* log grey fractions

* those are not used

* set grey fracs in message

* fix qcom
old-commit-hash: 47be6c751ec69d2f8957c9a1c6f7776a8bde4f7e
2021-07-08 15:22:00 +02:00
Willem Melching 3c5a97251f tici: Switch to 1D dead pixel correction (#21467)
old-commit-hash: 8d075048b30ea00a9d202165f63445d389426814
2021-07-02 14:01:43 +02:00
Dean Lee def897a2c7 fix webcam crash when failed to read video frame (#21272)
old-commit-hash: 73d741b7be4337b4e8dd39e150ee253b62b86153
2021-06-15 12:25:05 +02:00
Dean Lee 147b355dde add space after function name and if statement (#21225)
old-commit-hash: e4e669bb0fc1a919dfb38800099f8e3ad52346f1
2021-06-11 10:17:52 +02:00
Maxime Desroches 426df0435b Convert C++ while loops from int to bool (#21214)
old-commit-hash: 03df1b08bbe6b4156e9aa0bfbc5a85717094c877
2021-06-09 21:09:21 -07:00
Adeeb Shihadeh edd2f1c1a6 fix qcom and qcom2 builds
old-commit-hash: 9e94fb17cfd95dfc5ad85534e50748d5e4225dac
2021-06-08 16:44:05 -07:00