replay: improve segment download and merge (#22654)

* no-cache mode

* fix test cases build error

* space

* don't create cache dir in no-cache mode

* fix errors in test cases

* no_local_cache_

* set the number of connections by chunk_size

* use size_t instead of int64_t

* add test case for no-cache mode

* rename variables

* fix SIGSEGV

* cleanup

* faster decompressBZ2

* always decompress bz2

* add test cases

* prepare for python interface

* fix test cases build error

* continue

* camera_replay: cache remote file

* protected inheritance

* single option name

* TODO

* test_case for LogReader&FrameReader

* fix wrong require

* test case for FileReader

* cleanup test

* test:fix wrong filename

* check cached file's checksum

* fix mkdir permissions err

cleanup filereader

* remove initialize libav network libraries.

dd

* abort all loading if one failed

* cleanup tests

* use threadpool to limit concurrent downloads

* cache more segments

* merge 3 segments for replay

* one segment uses about 100M of memory

* use segments_need_merge.size()

* shutdown

* fix stuck if exit replay  before keyboard thread started

* load one segment at a time

* small cleanup

* cleanup filereader

* space

* tiny cleanup

* merge master

* cleanup test cases

* use util:create_directories

* cleanup framereader
This commit is contained in:
Dean Lee
2021-11-01 18:55:56 +08:00
committed by GitHub
parent 5b93d44459
commit 2b4a477fbc
19 changed files with 350 additions and 240 deletions
+6 -2
View File
@@ -21,8 +21,12 @@ else:
if USE_FRAME_STREAM:
cameras = ['cameras/camera_frame_stream.cc']
else:
libs += ['avutil', 'avcodec', 'avformat', 'swscale']
cameras = ['cameras/camera_replay.cc', env.Object('camera-framereader', '#/selfdrive/ui/replay/framereader.cc')]
libs += ['avutil', 'avcodec', 'avformat', 'swscale', 'bz2', 'ssl', 'curl', 'crypto']
# TODO: import replay_lib from root SConstruct
cameras = ['cameras/camera_replay.cc',
env.Object('camera-util', '#/selfdrive/ui/replay/util.cc'),
env.Object('camera-framereader', '#/selfdrive/ui/replay/framereader.cc'),
env.Object('camera-filereader', '#/selfdrive/ui/replay/filereader.cc')]
if arch == "Darwin":
del libs[libs.index('OpenCL')]