add 2018 skoda kodiaq fp

dragonpilot 0.7.5.4
======================== * Removed Steering monitor timer adjustment in code (timer in settings will no longer work.) * Updated Dynamic Follow model. (Special Thanks to @ShaneSmiskol for the feature and @cgw1968 for testing)
2026-06-20 05:22:03 +08:00 · 2020-07-03 11:35:47 +10:00 · 2020-06-22 11:50:30 +10:00 · 2020-05-31 16:36:15 +10:00 · 2020-05-25 22:29:03 +10:00 · 2020-05-19 23:00:20 +10:00
1672 changed files with 340489 additions and 33840 deletions
@@ -0,0 +1,25 @@
+---
+name: Bug report
+about: Create a report to help us improve openpilot
+title: ''
+labels: 'bug'
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**How to reproduce or log data**
+Steps to reproduce the behavior, or a explorer/cabana link to the exact drive and timestamp of when the bug occurred.
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Device/Version information (please complete the following information):**
+ - Device: [e.g. EON/EON Gold]
+ - Version: [e.g. 0.6.4], or commit hash when on devel
+ - Car make/model [e.g. Toyota Prius 2016]
+
+**Additional context**
+Add any other context about the problem here.
@@ -0,0 +1,21 @@
+Choose one of the templates below:
+
+# Fingerprint
+This pull requests adds a fingerprint for <Make - Model - Year - Trim>.
+
+This is an explorer link to a drive with the stock system enabled: ...
+
+# Car support
+This pull requests adds support for <Make - Model - Year - Trim>.
+
+This is an explorer link to a drive with the stock system enabled: ...
+This is an explorer link to a drive with openpilot system enabled: ...
+
+# Feature
+This pull requests adds feature X
+
+## Description
+Explain what the feature does
+
+## Testing
+Explain how the feature was tested. Either by the added unit tests, or what tests were performed while driving.
@@ -54,15 +54,9 @@ openpilot
 notebooks
 xx
 panda_jungle
-apks
-openpilot-apks

 .coverage*
 coverage.xml
 htmlcov
 pandaextra

-.mypy_cache/
-flycheck_*
-
-cppcheck_report.txt
@@ -1,168 +1,9 @@
-dragonpilot 0.7.10.1
-========================
-* HYUNDAI_GENESIS 使用 INDI 控制器。(感謝 @donfyffe 提供)
-* HYUNDAI_GENESIS uses INDI controller. (Thanks to @donfyffe)
-* HYUNDAI_GENESIS 加入 Cruise 按紐 和 lkMode 支援。(感謝 @donfyffe 建議)
-* HYUNDAI_GENESIS added Cruise button event and lkMode feature. (Thanks to @donfyffe)
-* 支援台灣版 2018 Huyndai IONIQ + smart MDPS (dp_hkg_smart_mdps) (感謝 @andy741217 提供)
-* Support 2018 Taiwan Hyundai IONIQ + smart MDPS (dp_hkg_smart_mdps) (Thanks to @andy741217)
-* 使用 openpilot v0.8 的模型。(感謝 @eisenheim)
-* Use openpilot v0.8 model. (Thanks to @eisenheim)
-* 加入 0.8 測試版的部分優化。
-* Added optimizations from pre-0.8.
-* 加入 dp_honda_eps_mod 設定來使用更高的扭力 (需 eps mod)。(感謝 @Wuxl_369 提供)
-* Added dp_honda_eps_mod setting to enable higher torque (eps mod required). (Thanks to @Wuxl_369)
-* 修正 VW 對白/灰熊的支援 (感謝 @lirudy 提供)
-* Fixed issue with white/grey panda support for VW (Thanks to @lirudy)
-* GENESIS_G70 優化 (感謝 @sebastian4k 提供)
-* GENESIS_G70 Optimisation (Thanks to @sebastian4k)
-* HYUNDAI_GENESIS 優化 (感謝 @donfyffe 提供)
-* HYUNDAI_GENESIS Optimisation (Thanks to @donfyffe)
-* 加入 Dynamic gas Lite。(感謝 @toyboxZ 提供)
-* Added Dynamic Gas Lite. (Thanks to @toyboxZ)
-* 加入來自 afa 的 Honda inspire, accord, crv SnG 優化。(感謝 @menwenliang 提供)
-* Added Honda inspire, accord, crv SnG optimisation from afa fork. (Thanks to @menwenliang) 
-* 加入 dp_toyota_lowest_cruise_override_vego。(感謝 @toyboxZ 提供)
-* Added dp_toyota_lowest_cruise_override_vego. (Thanks to @toyboxZ)
-
-dragonpilot 0.7.10.0
-========================
-* 基於最新 openpilot 0.7.10 devel.
-* Based on latest openpilot 0.7.10 devel.
-* 修正 Prius 特定情況下無法操控方向盤的問題。
-* Fixed unable to regain Prius steering control under certain condition.
-* 更新 VW MQB 的支援。(需執行 scripts/vw.sh 腳本)
-* Updated support of VW MQB. (scripts/vw.sh script required)
-* 新增 2018 China Toyota CHR 指紋v2。（感謝 @xiaohongcheung 提供)
-* Added 2018 China Toyota CHR FPv2. (Thanks to @xiaohongcheung)
-* 加入 Headunit Reloaded Android Auto App 支援。(感謝 @Ninjaa 提供)
-* Added Headunit Reloaded Android Auto App Support. (Thanks to @Ninjaa)
-* 優化 nanovg。(感謝 @piggy 提供)
-* Optomized nanovg. (Thanks to @piggy)
-* 加入 complete_setup.sh (感謝 @深鲸希西 提供)
-* Added complete_setup.sh (Thanks to @深鲸希西)
-* Based on latest openpilot 0.7.10 devel.
-* 修正 EON 接 PC/USB 充電器時仍會自動關機的錯誤。(感謝 @小愛 回報)
-* Fixed auto shutdown issue when EON connect to PC/USB Charger. (Thanks to @LOVEChen)
-* HYUNDAI_GENESIS 使用 INDI 控制器。(感謝 @donfyffe 提供)
-* HYUNDAI_GENESIS uses INDI controller. (Thanks to @donfyffe)
-
-dragonpilot 0.7.8.3
-========================
-* VW 加入 6 分鐘時間方向盤控制限制輔助方案。(特別感謝 @actuallylemoncurd 提供代碼)
-* VW added 6 minutes timebomb assist. (dp_timebomb_assist, special thanks to @actuallylemoncurd)
-
-dragonpilot 0.7.8.2
-========================
-* 修正在沒網路的情況下，開機超過五分鐘的問題。
-* Fixed 5+ minutes boot time issue when there is no internet connection.
-* 錯誤回傳改使用 dp 的主機。
-* Used dp server for error reporting.
-* 更新服務改使用 gitee 的 IP 檢查連線狀態。
-* updated service uses gitee IP address instead.
-
-dragonpilot 0.7.8.1
-========================
-* 加入 ko-KR 翻譯。
-* Added ko-KR translation.
-* 加入 Honda Jade 支援。(感謝 @李俊灝)
-* Added Honda Jade support. (Thanks to @lijunhao731)
-* 修正 ui.cc 內存越界的問題。(感謝 @piggy 提供)
-* Fixed ui.cc memory out of bound issue. (Thanks to @piggy)
-* gpxd 記錄改自動存成 zip 格式。
-* gpxd now store in zip format.
-* 強制關閉 panda 檢查 DOS 硬體。
-* Force disabled DOS hardware check in panda.
-
-dragonpilot 0.7.8.0
-========================
-* 基於最新 openpilot 0.7.8 devel.
-* Based on latest openpilot 0.7.8 devel.
-* 加入重置 DP 設定按鈕。(感謝 @LOVEChen 建議)
-* Added "Reset DP Settings" button. (Thanks to @LOVEChen)
-* 將警示訊息更改為類似於概念 UI 的設計。
-* Alert messages changed to concept UI alike design.
-* 當 manager 出現錯誤後，按 Exit 按鈕會執行 reset_update 腳本。
-* Added ability to execute reset_update.sh when press "Exit" button once manager returned errors. 
-
-dragonpilot 0.7.7.3
-========================
-* 修正方向盤監控。
-* Fixed steering monitor timer param.
-* 修正行駛時關閉畫面導致當機的錯誤。(感謝 @salmankhan, @stevej99, @bobbydough 回報)
-* Fixed screen frozen issue when "screen off while driving" toggle is enabled. (Thanks to @salmankhan, @stevej99, @bobbydough)
-* 加回 Dev Mini UI 開關。(感謝 @Ninjaa 建議)
-* Re-added Dev Mini UI. (Thanks to @Ninjaa)
-* 新增 (dp_reset_live_parameters_on_start) 每次發車重設 LiveParameters 值。(感謝 @eisenheim)
-* Added ability (dp_reset_live_param_on_start) to reset LiveParameters on each start. (Thanks @eisenheim)
-* 修正同時開啟 dp_toyota_zss 和 dp_lqr 產生的錯誤。(感謝 @bobbydough)
-* Fixed error cuased by enabling both dp_toyota_zss and dp_lqr at the same time. (Thanks to @bobbydough)
-* 新增 (dp_gpxd) 將 GPS 軌跡導出至 GPX 格式 (/sdcard/gpx_logs/）的功能。 （感謝 @mageymoo1）
-* Added ability (dp_gpxd) to export GPS track into GPX files (/sdcard/gpx_logs/). (Thanks to @mageymoo1)
-* 使用德國的車道寬度估算值。 （感謝 @arne182）
-* Used lane width estimate value from Germany. (Thanks to @arne182)
-
-dragonpilot 0.7.7.2
-========================
-* 加入 d_poly offset。 (感謝 @ShaneSmiskol)
-* Added d_poly offset. (Thanks to @ShaneSmiskol)
-* 加入 ZSS 支援。(感謝 @bobbydough, @WilliamPrius 建議, @bobbydough 測試)
-* Added ZSS support. (Thanks to @bobbydough, @WilliamPrius for recommendation, @bobbydough for testing)
-* 加入錯誤記錄至 /sdcard/crash_logs/ (感謝 @ShaneSmiskol 提供代碼)
-* Added error logs to /sdcard/crash_logs/ (Special Thanks to @ShaneSmiskol)
-* 加入 LQR 控制器開關進設定畫面。
-* Added LQR Controller toggle to settings.
-
-dragonpilot 0.7.7.1
-========================
-* 加入 C2 風扇靜音模式。(感謝 @dingliangxue)
-* Added C2 quiet fan mode. (Thanks to @dingliangxue)
-* 加入「輔助換道最低啟動速度」、「自動換道最低啟動速度」設定。
-* Added "Assisted Lane Change Min Engage Speed" and "Auto Lane Change Min Engage Speed" settings.
-* 加入回調校介面。(感謝 @Kent)
-* Re-added Dev UI. (Thanks to @Kent)
-* 加入 "dp_lqr" 設定來強制使用 RAV4 的 lqr 調校。(感謝 @eisenheim)
-* Added "dp_lqr" setting to force enable lqr tuning from RAV4. (Thanks to eisenheim) 
-
-dragonpilot 0.7.7.0
-========================
-* 基於最新 openpilot 0.7.7 devel.
-* Based on latest openpilot 0.7.7 devel.
-* 當 Manager 出現錯誤時，顯示 IP 位置。(感謝 @dingliangxue)
-* When Manager failed, display IP address. (Thanks to  @dingliangxue)
-* 加回 sr learner 開關。
-* Re-added sr learner toggle.
-* 加回 加速模式 開關。
-* Re-added Accel Profile toggle.
-* Toyota 加入改寫最低巡航速度功能。(感謝 @Mojo)
-* Added Toyota to override lowerest cruise speed. (Thanks to @Mojo)
-* 介面加入盲點偵測顯示。(感謝 @wabes)
-* Added BSM indicator to UI. (Thanks to @wabes)
-* 加回彎道減速功能。(感謝 @Mojo)
-* re-added Slow On Curve functionality. (Thanks to @Mojo)
-
-dragonpilot 0.7.6.2
-========================
-* 修正無法正確關閉駕駛監控的問題。
-* Fixed unable to properly turn off driver monitor issue.
-
-dragonpilot 0.7.6.1
-========================
-* 基於最新 openpilot 0.7.6.1 devel.
-* Based on latest openpilot 0.7.6.1 devel.
-* 優化並整合 dp 服務。 (所有的設定檔已改名，請重新設定所有的功能)
-* Optimized and integrated several dp services. (Settings have been renamed, please re-config all settings)
-* 完全關閉 steer ratio learner。
-* Completely disabled steer ratio learner.
-* 移除「加速模式」。
-* Removed Accel Profile.
-* 加入本田皓影混電版指紋v1。(感謝 @劉駿)
-* Added Honda Breeze Hybrid FPv1. (Thanks to @劉駿)
-* 加入台灣版 Toyota Prius 4.5 指紋v1。(感謝 @jeekid)
-* Added Taiwan Toyota Prius 4.5 FPv1. (Thanks to @jeekid)
-
 dragonpilot 0.7.5.4
 ========================
+* 加入 2018 skoda kodiaq 指紋v1。(感謝 @Gold 提供)
+* Added 2018 skoda kodiaq FPv1. (Thanks to @Gold)
+* 移除方向盤監控設定。(無法取消/關閉方向盤監控)
+* Removed Steering monitor timer adjustment in code (timer in settings will no longer work.)
 * Dynamic Follow 更新模型。(感謝 @ShaneSmiskol 提供代碼、 @cgw1968 測試)
 * Updated Dynamic Follow model. (Special Thanks to @ShaneSmiskol for the feature and @cgw1968 for testing)

@@ -1,106 +1,7 @@
-dragonpilot 0.7.10.0
-========================
-* 基於最新 openpilot 0.7.10 devel.
-* Based on latest openpilot 0.7.10 devel.
-* 修正 Prius 特定情況下無法操控方向盤的問題。
-* Fixed unable to regain Prius steering control under certain condition.
-* 更新 VW MQB 的支援。(需執行 scripts/vw.sh 腳本)
-* Updated support of VW MQB. (scripts/vw.sh script required)
-* 新增 2018 China Toyota CHR 指紋v2。（感謝 @xiaohongcheung 提供)
-* Added 2018 China Toyota CHR FPv2. (Thanks to @xiaohongcheung)
-* 加入 Headunit Reloaded Android Auto App 支援。(感謝 @Ninjaa 提供)
-* Added Headunit Reloaded Android Auto App Support. (Thanks to @Ninjaa)
-* 優化 nanovg。(感謝 @piggy 提供)
-* Optomized nanovg. (Thanks to @piggy)
-* 加入 complete_setup.sh (感謝 @深鲸希西 提供)
-* Added complete_setup.sh (Thanks to @深鲸希西)
-* Based on latest openpilot 0.7.10 devel.
-* 修正 EON 接 PC/USB 充電器時仍會自動關機的錯誤。(感謝 @小愛 回報)
-* Fixed auto shutdown issue when EON connect to PC/USB Charger. (Thanks to @LOVEChen)
-* HYUNDAI_GENESIS 使用 INDI 控制器。(感謝 @donfyffe 提供)
-* HYUNDAI_GENESIS uses INDI controller. (Thanks to @donfyffe)
-* HYUNDAI_GENESIS 加入 Cruise 按紐 和 lkMode 支援。(感謝 @donfyffe 建議)
-* HYUNDAI_GENESIS added Cruise button event and lkMode feature. (Thanks to @donfyffe) 
-* 支援台灣版 2018 Huyndai IONIQ + smart MDPS (dp_hkg_smart_mdps) (感謝 @andy741217 提供)
-* Support 2018 Taiwan Hyundai IONIQ + smart MDPS (dp_hkg_smart_mdps) (Thanks to @andy741217)
-* 使用 openpilot v0.8 的模型。(感謝 @eisenheim)
-* Use openpilot v0.8 model. (Thanks to @eisenheim)
-* 加入 0.8 測試版的部分優化。
-* Added optimizations from pre-0.8.
-* 加入 dp_honda_eps_mod 設定來使用更高的扭力 (需 eps mod)。(感謝 @Wuxl_369 提供)
-* Added dp_honda_eps_mod setting to enable higher torque (eps mod required). (Thanks to @Wuxl_369)
-* 修正 VW 對白/灰熊的支援 (感謝 @lirudy 提供)
-* Fixed issue with white/grey panda support for VW (Thanks to @lirudy)
-* GENESIS_G70 優化 (感謝 @sebastian4k 提供)
-* GENESIS_G70 Optimisation (Thanks to @sebastian4k)
-* HYUNDAI_GENESIS 優化 (感謝 @donfyffe 提供)
-* HYUNDAI_GENESIS Optimisation (Thanks to @donfyffe)
-* 加入 Dynamic gas Lite。(感謝 @toyboxZ 提供)
-* Added Dynamic Gas Lite. (Thanks to @toyboxZ)
-* 加入來自 afa 的 Honda inspire, accord, crv SnG 優化。(感謝 @menwenliang 提供)
-* Added Honda inspire, accord, crv SnG optimisation from afa fork. (Thanks to @menwenliang) 
-* 加入 dp_toyota_lowest_cruise_override_vego。(感謝 @toyboxZ 提供)
-* Added dp_toyota_lowest_cruise_override_vego. (Thanks to @toyboxZ)
-
-dragonpilot 0.7.8
-========================
-* 基於最新 openpilot 0.7.8 devel.
-* Based on latest openpilot 0.7.8 devel.
-* 加入重置 DP 設定按鈕。(感謝 @LOVEChen 建議)
-* Added "Reset DP Settings" button. (Thanks to @LOVEChen)
-* 將警示訊息更改為類似於概念 UI 的設計。
-* Alert messages changed to concept UI alike design.
-* 當 manager 出現錯誤後，按 Exit 按鈕會執行 reset_update 腳本。
-* Added ability to execute reset_update.sh when press "Exit" button once manager returned errors. 
-* 加入 ko-KR 翻譯。
-* Added ko-KR translation.
-* 加入 Honda Jade 支援。(感謝 @李俊灝)
-* Added Honda Jade support. (Thanks to @lijunhao731)
-* 修正 ui.cc 內存越界的問題。(感謝 @piggy 提供)
-* Fixed ui.cc memory out of bound issue. (Thanks to @piggy)
-* gpxd 記錄改自動存成 zip 格式。
-* gpxd now store in zip format.
-* 強制關閉 panda 檢查 DOS 硬體。
-* Force disabled DOS hardware check in panda.
-* 修正在沒網路的情況下，開機超過五分鐘的問題。
-* Fixed 5+ minutes boot time issue when there is no internet connection.
-* 錯誤回傳改使用 dp 的主機。
-* Used dp server for error reporting.
-* 更新服務改使用 gitee 的 IP 檢查連線狀態。
-* updated service uses gitee IP address instead.
-* VW 加入 6 分鐘時間方向盤控制限制輔助方案。(特別感謝 @actuallylemoncurd 提供代碼)
-* VW added 6 minutes timebomb assist. (dp_timebomb_assist, special thanks to @actuallylemoncurd)
-
-dragonpilot 0.7.7.0
-========================
-* 基於最新 openpilot 0.7.7 devel.
-* Based on latest openpilot 0.7.7 devel.
-* 當 Manager 出現錯誤時，顯示 IP 位置。(感謝 @dingliangxue)
-* When Manager failed, display IP address. (Thanks to  @dingliangxue)
-* 加回 sr learner 開關。
-* Re-added sr learner toggle.
-
-dragonpilot 0.7.6
-========================
-* 基於最新 openpilot 0.7.6.1 devel.
-* Based on latest openpilot 0.7.6.1 devel.
-* 優化並整合 dp 服務。 (所有的設定檔已改名，請重新設定所有的功能)
-* Optimized and integrated several dp services. (Settings have been renamed, please re-config all settings)
-* 完全關閉 steer ratio learner。
-* Completely disabled steer ratio learner.
-* 移除「加速模式」。
-* Removed Accel Profile.
-* 加入本田皓影混電版指紋v1。(感謝 @劉駿)
-* Added Honda Breeze Hybrid FPv1. (Thanks to @劉駿)
-* 加入台灣版 Toyota Prius 4.5 指紋v1。(感謝 @jeekid)
-* Added Taiwan Toyota Prius 4.5 FPv1. (Thanks to @jeekid)
-* 加入 2020 Toyota Prius 指紋v2。(感謝 @Trae)
-* Added Toyota Prius 2020 FPv2. (Thanks to @Trae)
-* 優化 Honda CR-V Hybrid 轉向。(感謝 @martint1980)
-× Optomised Honda CR-V Hybrid lateral control. (Thanks to @martint1980) 
-
 dragonpilot 0.7.5
 ========================
+* 移除方向盤監控設定。(無法取消/關閉方向盤監控)
+* Removed Steering monitor timer adjustment in code (timer in settings will no longer work.)
 * 優化 Lexus GSH 轉向。(感謝 @簡銘佑 測試)
 * Optimize Lexus GSH steering. (Thanks to @簡銘佑)
 * C2 支援自動關機「DragonAutoShutdownAt」參數。(感謝 @cgw1968 建議)
@@ -1,286 +1,7 @@
-2020-11-28 (0.7.10.0)
+2020-06-22 (0.7.5)
 ========================
-* 加入來自 afa 的 Honda inspire, accord, crv SnG 優化。(感謝 @menwenliang 提供)
-* Added Honda inspire, accord, crv SnG optimisation from afa fork. (Thanks to @menwenliang) 
-* 加入 dp_toyota_lowest_cruise_override_vego。(感謝 @toyboxZ 提供)
-* Added dp_toyota_lowest_cruise_override_vego. (Thanks to @toyboxZ)
-
-2020-11-20 (0.7.10.0)
-========================
-* 加入 Dynamic gas Lite。(感謝 @toyboxZ 提供)
-* Added Dynamic Gas Lite. (Thanks to @toyboxZ)
-
-2020-11-19 (0.7.10.0)
-========================
-* 更新所有 honda/hyunda/toyota 指紋。
-* Updated all honda/hyunda/toyota fingerprints.
-
-2020-11-18 (0.7.10.0)
-========================
-* 支援台灣版 2018 Huyndai IONIQ + smart MDPS (dp_hkg_smart_mdps) (感謝 @andy741217 提供)
-* Support 2018 Taiwan Hyundai IONIQ + smart MDPS (dp_hkg_smart_mdps) (Thanks to @andy741217)
-* 使用 openpilot v0.8 的模型。(感謝 @eisenheim)
-* Use openpilot v0.8 model. (Thanks to @eisenheim)
-* 加入 0.8 測試版的部分優化。
-* Added optimizations from pre-0.8.
-* 加入 dp_honda_eps_mod 設定來使用更高的扭力 (需 eps mod)。(感謝 @Wuxl_369 提供)
-* Added dp_honda_eps_mod setting to enable higher torque (eps mod required). (Thanks to @Wuxl_369)
-* 修正 VW 對白/灰熊的支援 (感謝 @lirudy 提供)
-* Fixed issue with white/grey panda support for VW (Thanks to @lirudy)
-* GENESIS_G70 優化 (感謝 @sebastian4k 提供)
-* GENESIS_G70 Optimisation (Thanks to @sebastian4k)
-* HYUNDAI_GENESIS 優化 (感謝 @donfyffe 提供)
-* HYUNDAI_GENESIS Optimisation (Thanks to @donfyffe)
-
-2020-11-05 (0.7.10.0)
-========================
-* HYUNDAI_GENESIS 加入 Cruise 按紐 和 lkMode 支援。(感謝 @donfyffe 建議)
-* HYUNDAI_GENESIS added Cruise button event and lkMode feature. (Thanks to @donfyffe) 
- 
-2020-11-04 (0.7.10.0)
-========================
-* HYUNDAI_GENESIS 使用 INDI 控制器。(感謝 @donfyffe 提供)
-* HYUNDAI_GENESIS uses INDI controller. (Thanks to @donfyffe)
-
-2020-10-30 (0.7.10.0)
-========================
-* 基於最新 openpilot 0.7.10 devel.
-* Based on latest openpilot 0.7.10 devel.
-* 修正 EON 接 PC/USB 充電器時仍會自動關機的錯誤。(感謝 @小愛 回報)
-* Fixed auto shutdown issue when EON connect to PC/USB Charger. (Thanks to @LOVEChen)
-* 新增大陸版 2018 Inspire 指紋。(感謝 @)
-* Added China Camry Hybrid FPv2. (Thanks to @杜子腾) 
-
-2020-10-23 (0.7.9.0)
-========================
-* 加入 Headunit Reloaded Android Auto 斷線偵側。
-* Added Disconnect detection for Headunit Reloaded Android Auto.
-* 加入 complete_setup.sh (感謝 @深鲸希西 提供)
-* Added complete_setup.sh (Thanks to @深鲸希西)
-
-2020-10-21 (0.7.9.0)
-========================
-* 基於最新 openpilot 0.7.9 devel.
-* Based on latest openpilot 0.7.9 devel.
-* 修正 Prius 特定情況下無法操控方向盤的問題。
-* Fixed unable to regain Prius steering control under certain condition.
-* 更新 VW MQB 的支援。(需執行 scripts/vw.sh 腳本)
-* Updated support of VW MQB. (scripts/vw.sh script required)
-* 加入 HKG mdps/sas 的支援。(需執行 scripts/hkg.sh 腳本)
-* Added support to HKG mdps/sas. (scripts/hkg.sh script required)
-* 新增 2018 China Toyota CHR 指紋v2。（感謝 @xiaohongcheung 提供)
-* Added 2018 China Toyota CHR FPv2. (Thanks to @xiaohongcheung)
-* 加入 Headunit Reloaded Android Auto App 支援。(感謝 @Ninjaa 提供)
-* Added Headunit Reloaded Android Auto App Support. (Thanks to @Ninjaa)
-* 優化 nanovg。(感謝 @piggy 提供)
-* Optomized nanovg. (Thanks to @piggy)
-
-2020-09-25 (0.7.8.0)
-========================
-* VW 加入 6 分鐘時間方向盤控制限制輔助方案。(特別感謝 @actuallylemoncurd 提供代碼)
-* VW added 6 minutes timebomb assist. (dp_timebomb_assist, special thanks to @actuallylemoncurd)
-
-2020-09-23 (0.7.8.0)
-========================
-* 修正在沒網路的情況下，開機超過五分鐘的問題。
-* Fixed 5+ minutes boot time issue when there is no internet connection.
-* 錯誤回傳改使用 dp 的主機。
-* Used dp server for error reporting.
-* 更新服務改使用 gitee 的 IP 檢查連線狀態。
-* updated service uses gitee IP address instead.
-
-2020-09-21 (0.7.8.0)
-========================
-* 強制關閉 panda 檢查 DOS 硬體。
-* Force disabled DOS hardware check in panda.
-
-2020-09-18 (0.7.8.0)
-========================
-* gpxd 記錄改自動存成 zip 格式。
-* gpxd now store in zip format.
-
-2020-09-15 (0.7.8.0)
-========================
-* 加入 Honda Jade 支援。(感謝 @李俊灝)
-* Added Honda Jade support. (Thanks to @lijunhao731)
-* 修正 ui.cc 內存越界的問題。(感謝 @piggy 提供)
-* Fixed ui.cc memory out of bound issue. (Thanks to @piggy)
-
-2020-09-01 (0.7.8.0)
-========================
-* 加入 ko-KR 翻譯。
-* Added ko-KR translation.
-
-2020-08-21 (0.7.8.0)
-========================
-* 基於最新 openpilot 0.7.8 devel.
-* Based on latest openpilot 0.7.8 devel.
-* 加入重置 DP 設定按鈕。(感謝 @LOVEChen 建議)
-* Added "Reset DP Settings" button. (Thanks to @LOVEChen)
-* 將警示訊息更改為類似於概念 UI 的設計。
-* Alert messages changed to concept UI alike design.
-* 當 manager 出現錯誤後，按 Exit 按鈕會執行 reset_update 腳本。
-* Added ability to execute reset_update.sh when press "Exit" button once manager returned errors. 
-
-2020-08-18 (0.7.7.0)
-========================
-* gpxd 不再切換至 GCJ-02 格式。(感謝 @arne182 建議)
-* gpxd no longer switch to GCJ-02 format automatically. (Thanks to @arne182)
-* 修正方向盤監控。
-* Fixed steering monitor timer param.
-* 修正行駛時關閉畫面導致當機的錯誤。(感謝 @salmankhan, @stevej99, @bobbydough 回報)
-* Fixed screen frozen issue when "screen off while driving" toggle is enabled. (Thanks to @salmankhan, @stevej99, @bobbydough)
-* 加回 Dev Mini UI 開關。(感謝 @Ninjaa 建議)
-* Re-added Dev Mini UI. (Thanks to @Ninjaa)
-
-2020-08-17 (0.7.7.0)
-========================
-* gpxd 只儲存高精度數據。(感謝 @arne182)
-* gpxd now only stored high accuracy data. (Thanks to @arne182)
-* gpxd 加入自動切換成 GCJ-02 格式。
-* added ability to switch to GCJ-02 format in gpxd.
-* 新增 (dp_reset_live_parameters_on_start) 每次發車重設 LiveParameters 值。(感謝 @eisenheim)
-* Added ability (dp_reset_live_param_on_start) to reset LiveParameters on each start. (Thanks @eisenheim)
-
-2020-08-12 (0.7.7.0)
-========================
-* 修正同時開啟 dp_toyota_zss 和 dp_lqr 產生的錯誤。(感謝 @bobbydough)
-* Fixed error cuased by enabling both dp_toyota_zss and dp_lqr at the same time. (Thanks to @bobbydough)
-
-2020-08-12 (0.7.7.0)
-========================
-* 新增 (dp_gpxd) 將 GPS 軌跡導出至 GPX 格式 (/sdcard/gpx_logs/）的功能。 （感謝 @mageymoo1）
-* Added ability (dp_gpxd) to export GPS track into GPX files (/sdcard/gpx_logs/). (Thanks to @mageymoo1)
-* 使用德國的車道寬度估算值。 （感謝 @arne182）
-* Used lane width estimate value from Germany. (Thanks to @arne182)
-
-2020-08-11 (0.7.7.0)
-========================
-* 加入 d_poly offset。 (感謝 @ShaneSmiskol)
-* Added d_poly offset. (Thanks to @ShaneSmiskol)
-
-2020-08-05 (0.7.7.0)
-========================
-* 修正 Dev UI 顯示。
-* Fixed Dev UI display.
-* 加入 LQR 控制器開關進設定畫面。
-* Added LQR Controller toggle to settings.
-
-2020-08-04 (0.7.7.0)
-========================
-* 嘗試修正非 Toyota 使用 lqr 產生的錯誤。
-* Attempted to fix lqr issue on non-Toyota Cars.
-* 加入錯誤記錄至 /sdcard/crash_logs/ (感謝 @ShaneSmiskol 提供代碼)
-* Added error logs to /sdcard/crash_logs/ (Special Thanks to @ShaneSmiskol)
-
-2020-08-02 (0.7.7.0)
-========================
-* 加入 ZSS 支援。(感謝 @bobbydough, @WilliamPrius 建議, @bobbydough 測試)
-* Added ZSS support. (Thanks to @bobbydough, @WilliamPrius for recommendation, @bobbydough for testing)
-
-2020-07-28 (0.7.7.0)
-========================
-* 修正 steer ratio learner 關閉。(感謝 @Mojo 回報, @ShaneSmiskol 提供代碼)
-* Fixed steer ratio learner toggle. (Thanks to @Mojo, @ShaneSmiskol)
-* 加入 "dp_lqr" 設定來強制使用 RAV4 的 lqr 調校。(感謝 @eisenheim)
-* Added "dp_lqr" setting to force enable lqr tuning from RAV4. (Thanks to eisenheim) 
-
-2020-07-28 (0.7.7.0)
-========================
-* 修正無法上傳記錄的問題。(感謝 @Mojo)
-* Fixed unable to upload log issue. (Thanks to @Mojo)
-* 修正無法關閉警示音的問題。(感謝 @Mojo)
-* Fixed unable to disable audio alert (-100%) issue. ($Thanks to @Mojo)
-
-2020-07-27 (0.7.7.0)
-========================
-* 加入回調校介面。(感謝 @Kent)
-* Re-added Dev UI. (Thanks to @Kent)
-
-2020-07-27 (0.7.7.0)
-========================
-* 加入 C2 風扇靜音模式。(感謝 @dingliangxue)
-* Added C2 quiet fan mode. (Thanks to @dingliangxue)
-* 加入「輔助換道最低啟動速度」、「自動換道最低啟動速度」設定。
-* Added "Assisted Lane Change Min Engage Speed" and "Auto Lane Change Min Engage Speed" settings.
-
-2020-07-23 (0.7.7.0)
-========================
-* 修正 appd。(感謝 @cgw1968)
-* Fixed appd. (Thanks to @cgw1968)
-
-2020-07-22 (0.7.7.0)
-========================
-* 修正 waze 顯示。(感謝 @Mojo)
-* Fixed waze display. (Thanks to @Mojo)
-* 加回彎道減速功能。(感謝 @Mojo)
-* re-added Slow On Curve functionality. (Thanks to @Mojo)
-* 加入日文支援。(特別感謝 @ponzu07 提供)
-* Added Japanese support. (Special thanks to @ponzu07)
-* 刪除部分設定對 dp_steering_on_signal 的依賴。
-* Removed dp_steering_on_signal dependencies.
-* 介面加入盲點偵測顯示。(感謝 @wabes)
-* Added BSM indicator to UI. (Thanks to @wabes)
-
-2020-07-21 (0.7.7.0)
-========================
-* Toyota 加入改寫最低巡航速度功能。(感謝 @Mojo)
-* Added Toyota to override lowest cruise speed. (Thanks to @Mojo)
-* 修正 gm 錯誤。
-* Fixed bugs in gm cars.
-
-2020-07-20 (0.7.7.0)
-========================
-* 加回 加速模式 開關。
-* Re-added Accel Profile toggle.
-* 修正 gm 錯誤。
-* Fixed bugs in gm cars.
-
-2020-07-19 (0.7.7.0)
-========================
-* 限制 dp_conf int / float 範圍。
-* Limited dp_conf int/float range.
-* 修復行車記錄文件夾不存在的錯誤。
-* Fixed dashcam folder not exist error.
-
-2020-07-18 (0.7.7.0)
-========================
-* 優化 camera offset 讀取。
-* optomised loading camera offset value.
-* 更換模型延遲警示為一般警示。
-* Replaced model lagging loud alert to normal alert.
-
-2020-07-17 (0.7.7.0)
-========================
-* 更新至最新 openpilot 0.7.7 devel。
-* Updated to latest openpilot 0.7.7 devel.
-
-2020-07-17 (0.7.7.0)
-========================
-* 當 Manager 出現錯誤時，顯示 IP 位置。(感謝 @dingliangxue)
-* When Manager failed, display IP address. (Thanks to  @dingliangxue)
-* 加回 sr learner 開關。
-* Re-added sr learner toggle.
-
-2020-07-13 (0.7.7.0)
-========================
-* 基於最新 openpilot 0.7.7 devel.
-* Based on latest openpilot 0.7.7 devel.
-
-2020-06-22 (0.7.6.1)
-========================
-* 更新至 openpilot 0.7.6.1。 (特別感謝 @rockindy 協助更新)
-* Updated to openpilot 0.7.6.1. (Special thanks to @rockindy for initial merge)
-* 優化並整合 dp 服務。 (所有的設定檔已改名，請重新設定所有的功能)
-* Optimized and integrated several dp services. (Settings have been renamed, please re-config all settings)
-* 完全關閉 steer ratio learner。
-* Completely disabled steer ratio learner.
-* 移除「加速模式」。
-* Removed Accel Profile.
-* 加入本田皓影混電版指紋v1。(感謝 @劉駿)
-* Added Honda Breeze Hybrid FPv1. (Thanks to @劉駿)
-* 加入台灣版 Toyota Prius 4.5 指紋v1。(感謝 @jeekid)
-* Added Taiwan Toyota Prius 4.5 FPv1. (Thanks to @jeekid)
+* 移除方向盤監控設定。(無法取消/關閉方向盤監控)
+* Removed Steering monitor timer adjustment in code (timer in settings will no longer work.)

 2020-06-18 (0.7.5)
 ========================
@@ -2,7 +2,7 @@

 Our software is open source so you can solve your own problems without needing help from others. And if you solve a problem and are so kind, you can upstream it for the rest of the world to use.

-Most open source development activity is coordinated through our [Discord](https://discord.comma.ai). A lot of documentation is available on our [medium](https://medium.com/@comma_ai/).
+Most open source development activity is coordinated through our [Discord](https://discord.comma.ai). A lot of documentation is available on our [medium](https://medium.com/@comma_ai/)

 ## Getting Started

@@ -18,17 +18,17 @@ You can test your changes on your machine by running `run_docker_tests.sh`. This

 ### Automated Testing

-All PRs and commits are automatically checked by Github Actions. Check out `.github/workflows/` for what Github Actions runs. Any new tests should be added to Github Actions.
+All PRs are automatically checked by Github Actions. Check out `.github/workflows/` for what Github Actions runs. Any new tests sould be added to Github Actions.

 ### Code Style and Linting

-Code is automatically checked for style by Github Actions as part of the automated tests. You can also run these tests yourself by running `pre-commit run --all`.
+Code is automatically checked for style by Github Actions as part of the automated tests. You can also run these tests yourself by running `pylint_openpilot.sh` and `flake8_openpilot.sh`.

 ## Car Ports (openpilot)

 We've released a [Model Port guide](https://medium.com/@comma_ai/openpilot-port-guide-for-toyota-models-e5467f4b5fe6) for porting to Toyota/Lexus models.

-If you port openpilot to a substantially new car brand, see this more generic [Brand Port guide](https://medium.com/@comma_ai/how-to-write-a-car-port-for-openpilot-7ce0785eda84). You might also be eligible for a bounty. 
+If you port openpilot to a substantially new car brand, see this more generic [Brand Port guide](https://medium.com/@comma_ai/how-to-write-a-car-port-for-openpilot-7ce0785eda84). You might also be eligible for a bounty. See our bounties at [comma.ai/bounties.html](https://comma.ai/bounties.html)

 ## Pull Requests

@@ -48,4 +48,3 @@ Modules that are in seperate repositories include:
 * laika
 * opendbc
 * panda
-* rednose
@@ -1,160 +0,0 @@
-def phone(String ip, String step_label, String cmd) {
-  def ci_env = "CI=1 TEST_DIR=${env.TEST_DIR} GIT_BRANCH=${env.GIT_BRANCH} GIT_COMMIT=${env.GIT_COMMIT}"
-
-  withCredentials([file(credentialsId: 'id_rsa_public', variable: 'key_file')]) {
-    sh label: step_label,
-        script: """
-                ssh -tt -o StrictHostKeyChecking=no -i ${key_file} -p 8022 root@${ip} '${ci_env} /usr/bin/bash -le' <<'EOF'
-echo \$\$ > /dev/cpuset/app/tasks || true
-echo \$PPID > /dev/cpuset/app/tasks || true
-mkdir -p /dev/shm
-chmod 777 /dev/shm
-cd ${env.TEST_DIR} || true
-${cmd}
-exit 0
-EOF"""
-  }
-}
-
-def phone_steps(String device_type, steps) {
-  lock(resource: "", label: device_type, inversePrecedence: true, variable: 'device_ip', quantity: 1) {
-    timeout(time: 60, unit: 'MINUTES') {
-      phone(device_ip, "kill old processes", "pkill -f comma || true")
-      phone(device_ip, "git checkout", readFile("selfdrive/test/setup_device_ci.sh"),)
-      steps.each { item ->
-        phone(device_ip, item[0], item[1])
-      }
-    }
-  }
-}
-
-pipeline {
-  agent none
-  environment {
-    COMMA_JWT = credentials('athena-test-jwt')
-    TEST_DIR = "/data/openpilot"
-  }
-  options {
-      timeout(time: 1, unit: 'HOURS')
-  }
-
-  stages {
-
-    stage('Release Build') {
-      agent {
-        docker {
-          image 'python:3.7.3'
-          args '--user=root'
-        }
-      }
-      when {
-        branch 'devel-staging'
-      }
-      steps {
-        phone_steps("eon-build", [
-          ["build release2-staging and dashcam-staging", "cd release && PUSH=1 ./build_release2.sh"],
-        ])
-      }
-    }
-
-    stage('openpilot tests') {
-      when {
-        not {
-          anyOf {
-            branch 'master-ci'; branch 'devel'; branch 'devel-staging'; branch 'release2'; branch 'release2-staging'; branch 'dashcam'; branch 'dashcam-staging'; branch 'testing-closet*'
-          }
-        }
-      }
-
-
-      stages {
-
-        /*
-        stage('PC tests') {
-          agent {
-            dockerfile {
-              filename 'Dockerfile.openpilotci'
-              args '--privileged --shm-size=1G --user=root'
-            }
-          }
-          stages {
-            stage('Build') {
-              steps {
-                sh 'scons -j$(nproc)'
-              }
-            }
-          }
-          post {
-            always {
-              // fix permissions since docker runs as another user
-              sh "chmod -R 777 ."
-            }
-          }
-        }
-        */
-
-        stage('On-device Tests') {
-          agent {
-            docker {
-              image 'python:3.7.3'
-              args '--user=root'
-            }
-          }
-
-          stages {
-            stage('parallel tests') {
-              parallel {
-
-                stage('Devel Build') {
-                  environment {
-                    CI_PUSH = "${env.BRANCH_NAME == 'master' ? 'master-ci' : ' '}"
-                  }
-                  steps {
-                    phone_steps("eon", [
-                      ["build devel", "cd release && CI_PUSH=${env.CI_PUSH} ./build_devel.sh"],
-                      ["test openpilot", "nosetests -s selfdrive/test/test_openpilot.py"],
-                      ["test cpu usage", "cd selfdrive/test/ && ./test_cpu_usage.py"],
-                      ["test car interfaces", "cd selfdrive/car/tests/ && ./test_car_interfaces.py"],
-                      ["test spinner build", "cd selfdrive/ui/spinner && make clean && make"],
-                      ["test text window build", "cd selfdrive/ui/text && make clean && make"],
-                    ])
-                  }
-                }
-
-                stage('Replay Tests') {
-                  steps {
-                    phone_steps("eon2", [
-                      ["camerad/modeld replay", "cd selfdrive/test/process_replay && ./camera_replay.py"],
-                    ])
-                  }
-                }
-
-                stage('HW + Unit Tests') {
-                  steps {
-                    phone_steps("eon", [
-                      ["build cereal", "SCONS_CACHE=1 scons -j4 cereal/"],
-                      ["test sounds", "nosetests -s selfdrive/test/test_sounds.py"],
-                      ["test boardd loopback", "nosetests -s selfdrive/boardd/tests/test_boardd_loopback.py"],
-                      ["test loggerd", "CI=1 python selfdrive/loggerd/tests/test_loggerd.py"],
-                      //["test camerad", "CI=1 python selfdrive/camerad/test/test_camerad.py"], // wait for shelf refactor
-                      //["test updater", "python installer/updater/test_updater.py"],
-                    ])
-                  }
-                }
-
-              }
-            }
-          }
-
-          post {
-            always {
-              cleanWs()
-            }
-          }
-
-        }
-
-      }
-    }
-  }
-}
@@ -66,35 +66,33 @@ Supported Cars
 | ----------| ------------------------------| ------------------| -----------------| -------------------| ------------------|
 | Acura     | ILX 2016-18                   | AcuraWatch Plus   | openpilot        | 25mph<sup>1</sup>  | 25mph             |
 | Acura     | RDX 2016-18                   | AcuraWatch Plus   | openpilot        | 25mph<sup>1</sup>  | 12mph             |
-| Acura     | RDX 2020                      | AcuraWatch        | Stock            | 0mph               | 3mph              |
-| Honda     | Accord 2018-20                | All               | Stock            | 0mph               | 3mph              |
-| Honda     | Accord Hybrid 2018-20         | All               | Stock            | 0mph               | 3mph              |
+| Honda     | Accord 2018-19                | All               | Stock            | 0mph               | 3mph              |
+| Honda     | Accord Hybrid 2018-19         | All               | Stock            | 0mph               | 3mph              |
 | Honda     | Civic Hatchback 2017-19       | Honda Sensing     | Stock            | 0mph               | 12mph             |
 | Honda     | Civic Sedan/Coupe 2016-18     | Honda Sensing     | openpilot        | 0mph               | 12mph             |
-| Honda     | Civic Sedan/Coupe 2019-20     | All               | Stock            | 0mph               | 2mph<sup>2</sup>  |
+| Honda     | Civic Sedan/Coupe 2019-20     | Honda Sensing     | Stock            | 0mph               | 2mph<sup>2</sup>  |
 | Honda     | CR-V 2015-16                  | Touring           | openpilot        | 25mph<sup>1</sup>  | 12mph             |
-| Honda     | CR-V 2017-20                  | Honda Sensing     | Stock            | 0mph               | 12mph             |
+| Honda     | CR-V 2017-19                  | Honda Sensing     | Stock            | 0mph               | 12mph             |
 | Honda     | CR-V Hybrid 2017-2019         | Honda Sensing     | Stock            | 0mph               | 12mph             |
 | Honda     | Fit 2018-19                   | Honda Sensing     | openpilot        | 25mph<sup>1</sup>  | 12mph             |
 | Honda     | HR-V 2019                     | Honda Sensing     | openpilot        | 25mph<sup>1</sup>  | 12mph             |
-| Honda     | Insight 2019-20               | All               | Stock            | 0mph               | 3mph              |
+| Honda     | Insight 2019                  | Honda Sensing     | Stock            | 0mph               | 3mph              |
 | Honda     | Odyssey 2018-20               | Honda Sensing     | openpilot        | 25mph<sup>1</sup>  | 0mph              |
 | Honda     | Passport 2019                 | All               | openpilot        | 25mph<sup>1</sup>  | 12mph             |
-| Honda     | Pilot 2016-19                 | Honda Sensing     | openpilot        | 25mph<sup>1</sup>  | 12mph             |
+| Honda     | Pilot 2016-18                 | Honda Sensing     | openpilot        | 25mph<sup>1</sup>  | 12mph             |
+| Honda     | Pilot 2019                    | All               | openpilot        | 25mph<sup>1</sup>  | 12mph             |
 | Honda     | Ridgeline 2017-20             | Honda Sensing     | openpilot        | 25mph<sup>1</sup>  | 12mph             |
-| Hyundai   | Palisade 2020                 | All               | Stock            | 0mph               | 0mph         |
-| Hyundai   | Sonata 2020                   | All               | Stock            | 0mph               | 0mph              |
-| Lexus     | CT Hybrid 2017-18             | LSS               | Stock<sup>3</sup>| 0mph               | 0mph              |
+| Lexus     | CT Hybrid 2017-18             | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
 | Lexus     | ES 2019                       | All               | openpilot        | 0mph               | 0mph              |
 | Lexus     | ES Hybrid 2019                | All               | openpilot        | 0mph               | 0mph              |
 | Lexus     | IS 2017-2019                  | All               | Stock            | 22mph              | 0mph              |
 | Lexus     | IS Hybrid 2017                | All               | Stock            | 0mph               | 0mph              |
 | Lexus     | NX Hybrid 2018                | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
-| Lexus     | RX 2016-18                    | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
+| Lexus     | RX 2016-17                    | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
 | Lexus     | RX 2020                       | All               | openpilot        | 0mph               | 0mph              |
 | Lexus     | RX Hybrid 2016-19             | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
-| Lexus     | RX Hybrid 2020                | All               | openpilot        | 0mph               | 0mph              |
-| Toyota    | Avalon 2016-18                | TSS-P             | Stock<sup>3</sup>| 20mph<sup>1</sup>  | 0mph              |
+| Toyota    | Avalon 2016                   | TSS-P             | Stock<sup>3</sup>| 20mph<sup>1</sup>  | 0mph              |
+| Toyota    | Avalon 2017-18                | All               | Stock<sup>3</sup>| 20mph<sup>1</sup>  | 0mph              |
 | Toyota    | Camry 2018-20                 | All               | Stock            | 0mph<sup>4</sup>   | 0mph              |
 | Toyota    | Camry Hybrid 2018-19          | All               | Stock            | 0mph<sup>4</sup>   | 0mph              |
 | Toyota    | C-HR 2017-19                  | All               | Stock            | 0mph               | 0mph              |
@@ -102,22 +100,25 @@ Supported Cars
 | Toyota    | Corolla 2017-19               | All               | Stock<sup>3</sup>| 20mph<sup>1</sup>  | 0mph              |
 | Toyota    | Corolla 2020                  | All               | openpilot        | 0mph               | 0mph              |
 | Toyota    | Corolla Hatchback 2019-20     | All               | openpilot        | 0mph               | 0mph              |
-| Toyota    | Corolla Hybrid 2020-21        | All               | openpilot        | 0mph               | 0mph              |
+| Toyota    | Corolla Hybrid 2020           | All               | openpilot        | 0mph               | 0mph              |
 | Toyota    | Highlander 2017-19            | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
-| Toyota    | Highlander 2020               | All               | openpilot        | 0mph               | 0mph              |
 | Toyota    | Highlander Hybrid 2017-19     | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
+| Toyota    | Highlander 2020               | All               | openpilot        | 0mph               | 0mph              |
 | Toyota    | Highlander Hybrid 2020        | All               | openpilot        | 0mph               | 0mph              |
-| Toyota    | Prius 2016-20                 | TSS-P             | Stock<sup>3</sup>| 0mph               | 0mph              |
+| Toyota    | Prius 2016                    | TSS-P             | Stock<sup>3</sup>| 0mph               | 0mph              |
+| Toyota    | Prius 2017-19                 | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
 | Toyota    | Prius Prime 2017-20           | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
-| Toyota    | Rav4 2016-18                  | TSS-P             | Stock<sup>3</sup>| 20mph<sup>1</sup>  | 0mph              |
-| Toyota    | Rav4 2019-21                  | All               | openpilot        | 0mph               | 0mph              |
-| Toyota    | Rav4 Hybrid 2016-18           | TSS-P             | Stock<sup>3</sup>| 0mph               | 0mph              |
+| Toyota    | Rav4 2016                     | TSS-P             | Stock<sup>3</sup>| 20mph<sup>1</sup>  | 0mph              |
+| Toyota    | Rav4 2017-18                  | All               | Stock<sup>3</sup>| 20mph<sup>1</sup>  | 0mph              |
+| Toyota    | Rav4 2019-20                  | All               | openpilot        | 0mph               | 0mph              |
+| Toyota    | Rav4 Hybrid 2016              | TSS-P             | Stock<sup>3</sup>| 0mph               | 0mph              |
+| Toyota    | Rav4 Hybrid 2017-18           | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
 | Toyota    | Rav4 Hybrid 2019-20           | All               | openpilot        | 0mph               | 0mph              |
-| Toyota    | Sienna 2018-20                | All               | Stock<sup>3</sup>| 0mph               | 0mph              |
+| Toyota    | Sienna 2018                   | All               | Stock<sup>3</sup>| 0mph               | 0mph              |

-<sup>1</sup>[Comma Pedal](https://github.com/commaai/openpilot/wiki/comma-pedal) is used to provide stop-and-go capability to some of the openpilot-supported cars that don't currently support stop-and-go. ***NOTE: The Comma Pedal is not officially supported by [comma](https://comma.ai).*** <br />
+<sup>1</sup>[Comma Pedal](https://community.comma.ai/wiki/index.php/Comma_Pedal) is used to provide stop-and-go capability to some of the openpilot-supported cars that don't currently support stop-and-go. Here is how to [build a Comma Pedal](https://medium.com/@jfrux/comma-pedal-building-with-macrofab-6328bea791e8). ***NOTE: The Comma Pedal is not officially supported by [comma](https://comma.ai).*** <br />
 <sup>2</sup>2019 Honda Civic 1.6L Diesel Sedan does not have ALC below 12mph. <br />
-<sup>3</sup>When disconnecting the Driver Support Unit (DSU), openpilot ACC will replace stock ACC. ***NOTE: disconnecting the DSU disables Automatic Emergency Braking (AEB).*** <br />
+<sup>3</sup>When disconnecting the Driver Support Unit (DSU), openpilot ACC will replace stock ACC. For DSU locations, see [Toyota Wiki page](https://community.comma.ai/wiki/index.php/Toyota). ***NOTE: disconnecting the DSU disables Automatic Emergency Braking (AEB).*** <br />
 <sup>4</sup>28mph for Camry 4CYL L, 4CYL LE and 4CYL SE which don't have Full-Speed Range Dynamic Radar Cruise Control. <br />

 Community Maintained Cars and Features
@@ -133,37 +134,35 @@ Community Maintained Cars and Features
 | Chrysler  | Pacifica 2020                 | Adaptive Cruise   | Stock            | 0mph               | 39mph        |
 | Chrysler  | Pacifica Hybrid 2017-18       | Adaptive Cruise   | Stock            | 0mph               | 9mph         |
 | Chrysler  | Pacifica Hybrid 2019-20       | Adaptive Cruise   | Stock            | 0mph               | 39mph        |
-| Genesis   | G70 2018                      | All               | Stock            | 0mph               | 0mph         |
-| Genesis   | G80 2018                      | All               | Stock            | 0mph               | 0mph         |
-| Genesis   | G90 2018                      | All               | Stock            | 0mph               | 0mph         |
-| GMC       | Acadia Denali 2018<sup>2</sup>| Adaptive Cruise   | openpilot        | 0mph               | 7mph         |
+| Genesis   | G80 2018<sup>2</sup>          | All               | Stock            | 0mph               | 0mph         |
+| Genesis   | G90 2018<sup>2</sup>          | All               | Stock            | 0mph               | 0mph         |
+| GMC       | Acadia Denali 2018<sup>3</sup>| Adaptive Cruise   | openpilot        | 0mph               | 7mph         |
 | Holden    | Astra 2017<sup>1</sup>        | Adaptive Cruise   | openpilot        | 0mph               | 7mph         |
-| Hyundai   | Elantra 2017-19               | SCC + LKAS        | Stock            | 19mph              | 34mph        |
-| Hyundai   | Genesis 2015-16               | SCC + LKAS        | Stock            | 19mph              | 37mph        |
-| Hyundai   | Ioniq Electric 2019-20        | SCC + LKAS        | Stock            | 0mph               | 32mph        |
-| Hyundai   | Kona 2020                     | SCC + LKAS        | Stock            | 0mph               | 0mph         |
-| Hyundai   | Kona EV 2019                  | SCC + LKAS        | Stock            | 0mph               | 0mph         |
-| Hyundai   | Santa Fe 2019                 | All               | Stock            | 0mph               | 0mph         |
-| Hyundai   | Sonata 2019                   | SCC + LKAS        | Stock            | 0mph               | 0mph         |
-| Hyundai   | Veloster 2019                 | SCC + LKAS        | Stock            | 5mph               | 0mph         |
+| Hyundai   | Elantra 2017-19<sup>2</sup>   | SCC + LKAS        | Stock            | 19mph              | 34mph        |
+| Hyundai   | Genesis 2015-16<sup>2</sup>   | SCC + LKAS        | Stock            | 19mph              | 37mph        |
+| Hyundai   | Ioniq 2017<sup>2</sup>        | SCC + LKAS        | Stock            | 0mph               | 32mph        |
+| Hyundai   | Ioniq 2019 EV<sup>2</sup>     | SCC + LKAS        | Stock            | 0mph               | 32mph        |
+| Hyundai   | Kona 2017-19<sup>2</sup>      | SCC + LKAS        | Stock            | 22mph              | 0mph         |
+| Hyundai   | Kona 2019 EV<sup>2</sup>      | SCC + LKAS        | Stock            | 0mph               | 0mph         |
+| Hyundai   | Palisade 2020<sup>2</sup>     | All               | Stock            | 0mph               | 0mph         |
+| Hyundai   | Santa Fe 2019<sup>2</sup>     | All               | Stock            | 0mph               | 0mph         |
+| Hyundai   | Sonata 2020<sup>2</sup>       | All               | Stock            | 0mph               | 0mph         |
 | Jeep      | Grand Cherokee 2016-18        | Adaptive Cruise   | Stock            | 0mph               | 9mph         |
-| Jeep      | Grand Cherokee 2019-20        | Adaptive Cruise   | Stock            | 0mph               | 39mph        |
-| Kia       | Forte 2018-19                 | SCC + LKAS        | Stock            | 0mph               | 0mph         |
-| Kia       | Optima 2017                   | SCC + LKAS        | Stock            | 0mph               | 32mph        |
-| Kia       | Optima 2019                   | SCC + LKAS        | Stock            | 0mph               | 0mph         |
-| Kia       | Sorento 2018                  | SCC + LKAS        | Stock            | 0mph               | 0mph         |
-| Kia       | Stinger 2018                  | SCC + LKAS        | Stock            | 0mph               | 0mph         |
-| Nissan    | Leaf 2018-20                  | ProPILOT          | Stock            | 0mph               | 0mph         |
-| Nissan    | Rogue 2018-19                 | ProPILOT          | Stock            | 0mph               | 0mph         |
-| Nissan    | X-Trail 2017                  | ProPILOT          | Stock            | 0mph               | 0mph         |
-| Subaru    | Ascent 2019                   | EyeSight          | Stock            | 0mph               | 0mph         |
+| Jeep      | Grand Cherokee 2019           | Adaptive Cruise   | Stock            | 0mph               | 39mph        |
+| Kia       | Forte 2018<sup>2</sup>        | SCC + LKAS        | Stock            | 0mph               | 0mph         |
+| Kia       | Optima 2017<sup>2</sup>       | SCC + LKAS/LDWS   | Stock            | 0mph               | 32mph        |
+| Kia       | Optima 2019<sup>2</sup>       | SCC + LKAS        | Stock            | 0mph               | 0mph         |
+| Kia       | Sorento 2018<sup>2</sup>      | SCC + LKAS        | Stock            | 0mph               | 0mph         |
+| Kia       | Stinger 2018<sup>2</sup>      | SCC + LKAS        | Stock            | 0mph               | 0mph         |
+| Nissan    | Leaf 2019                     | Propilot          | Stock            | 0mph               | 0mph         |
+| Nissan    | X-Trail 2018                  | Propilot          | Stock            | 0mph               | 0mph         |
 | Subaru    | Crosstrek 2018-19             | EyeSight          | Stock            | 0mph               | 0mph         |
-| Subaru    | Forester 2019                 | EyeSight          | Stock            | 0mph               | 0mph         |
-| Subaru    | Impreza 2017-19               | EyeSight          | Stock            | 0mph               | 0mph         |
-| Volkswagen| Golf 2015-19                  | Driver Assistance | Stock            | 0mph               | 0mph         |
+| Subaru    | Impreza 2018-20               | EyeSight          | Stock            | 0mph               | 0mph         |
+| Volkswagen| Golf 2016-19<sup>3</sup>      | Driver Assistance | Stock            | 0mph               | 0mph         |

-<sup>1</sup>Requires an [OBD-II car harness](https://comma.ai/shop/products/comma-car-harness) and [community built giraffe](https://github.com/commaai/openpilot/wiki/GM). ***NOTE: disconnecting the ASCM disables Automatic Emergency Braking (AEB).*** <br />
-<sup>2</sup>Requires a custom connector for the developer [car harness](https://comma.ai/shop/products/car-harness) <br />
+<sup>1</sup>Requires a [panda](https://comma.ai/shop/products/panda-obd-ii-dongle) and [community built giraffe](https://zoneos.com/volt/). ***NOTE: disconnecting the ASCM disables Automatic Emergency Braking (AEB).*** <br />
+<sup>2</sup>Requires a [panda](https://comma.ai/shop/products/panda-obd-ii-dongle) and open sourced [Hyundai giraffe](https://github.com/commaai/neo/tree/master/giraffe/hyundai), designed for the 2019 Sante Fe; pinout may differ for other Hyundai and Kia models. <br />
+<sup>3</sup>Requires a [custom connector](https://community.comma.ai/wiki/index.php/Volkswagen#Integration_at_R242_Camera) for the [car harness](https://comma.ai/shop/products/car-harness) <br />

 Although it's not upstream, there's a community of people getting openpilot to run on Tesla's [here](https://tinkla.us/)

@@ -176,7 +175,7 @@ Installation Instructions

 Install openpilot on an EON or comma two by entering ``https://openpilot.comma.ai`` during the installer setup.

-Follow these [video instructions](https://youtu.be/lcjqxCymins) to properly mount the device on the windshield. Note: openpilot features an automatic pose calibration routine and openpilot performance should not be affected by small pitch and yaw misalignments caused by imprecise device mounting.
+Follow these [video instructions](https://youtu.be/3nlkomHathI) to properly mount the device on the windshield. Note: openpilot features an automatic pose calibration routine and openpilot performance should not be affected by small pitch and yaw misalignments caused by imprecise device mounting.

 Before placing the device on your windshield, check the state and local laws and ordinances where you drive. Some state laws prohibit or restrict the placement of objects on the windshield of a motor vehicle.

@@ -269,23 +268,8 @@ Safety and Testing

 Testing on PC
 ------
-For simplified development and experimentation, openpilot can be run in the CARLA driving simulator, which allows you to develop openpilot without a car. The whole setup should only take a few minutes. 
-
-Steps:
-1) Start the CARLA server on first terminal
-```
-bash -c "$(curl https://raw.githubusercontent.com/commaai/openpilot/master/tools/sim/start_carla.sh)"
-```
-2) Start openpilot on second terminal
-```
-bash -c "$(curl https://raw.githubusercontent.com/commaai/openpilot/master/tools/sim/start_openpilot_docker.sh)"
-```
-3) Press 1 to engage openpilot
-
-See the full [README](tools/sim/README.md)
-
-You should also take a look at the tools directory in master: lots of tools you can use to replay driving data, test, and develop openpilot from your PC.

+Check out the tools directory in master: lots of tools you can use to replay driving data, test and develop openpilot from your pc.

 Community and Contributing
 ------
@@ -294,7 +278,7 @@ openpilot is developed by [comma](https://comma.ai/) and by users like you. We w

 You can add support for your car by following guides we have written for [Brand](https://medium.com/@comma_ai/how-to-write-a-car-port-for-openpilot-7ce0785eda84) and [Model](https://medium.com/@comma_ai/openpilot-port-guide-for-toyota-models-e5467f4b5fe6) ports. Generally, a car with adaptive cruise control and lane keep assist is a good candidate. [Join our Discord](https://discord.comma.ai) to discuss car ports: most car makes have a dedicated channel.

-Want to get paid to work on openpilot? [comma is hiring](https://comma.ai/jobs/).
+Want to get paid to work on openpilot? [comma is hiring](https://comma.ai/jobs/). We also have a [bounty program](https://comma.ai/bounties.html).

 And [follow us on Twitter](https://twitter.com/comma_ai).

@@ -310,7 +294,7 @@ Directory Structure
    ├── phonelibs           # Libraries used on NEOS devices
    ├── pyextra             # Libraries used on NEOS devices
    └── selfdrive           # Code needed to drive the car
-        ├── assets          # Fonts, images and sounds for UI
+        ├── assets          # Fonts, images, and sounds for UI
        ├── athena          # Allows communication with the app
        ├── boardd          # Daemon to talk to the board
        ├── camerad         # Driver to capture images from the camera sensors
@@ -344,7 +328,6 @@ NO WARRANTY EXPRESSED OR IMPLIED.**

 <img src="https://d1qb2nb5cznatu.cloudfront.net/startups/i/1061157-bc7e9bf3b246ece7322e6ffe653f6af8-medium_jpg.jpg?buster=1458363130" width="75"></img> <img src="https://cdn-images-1.medium.com/max/1600/1*C87EjxGeMPrkTuVRVWVg4w.png" width="225"></img>

-[![openpilot tests](https://github.com/commaai/openpilot/workflows/openpilot%20tests/badge.svg?event=push)](https://github.com/commaai/openpilot/actions)
 [![Total alerts](https://img.shields.io/lgtm/alerts/g/commaai/openpilot.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/commaai/openpilot/alerts/)
 [![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/commaai/openpilot.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/commaai/openpilot/context:python)
 [![Language grade: C/C++](https://img.shields.io/lgtm/grade/cpp/g/commaai/openpilot.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/commaai/openpilot/context:cpp)
@@ -1,62 +1,12 @@
-Version 0.7.10 (2020-10-29)
-========================
- * Grey panda is deprecated, upgrade to comma two or black panda
- * NEOS update: update to Python 3.8.2 and lower CPU frequency
- * Improved thermals due to reduced CPU frequency
- * Update SNPE to 1.41.0
- * Reduced offroad power consumption
- * Various system stability improvements
- * Acura RDX 2020 support thanks to csouers!
-
-Version 0.7.9 (2020-10-09)
-========================
- * Improved car battery power management
- * Improved updater robustness
- * Improved realtime performance
- * Reduced UI and modeld lags
- * Increased torque on 2020 Hyundai Sonata and Palisade
-
-Version 0.7.8 (2020-08-19)
-========================
- * New driver monitoring model: improved face detection and better compatibility with sunglasses
- * Download NEOS operating system updates in the background
- * Improved updater reliability and responsiveness
- * Hyundai Kona 2020, Veloster 2019, and Genesis G70 2018 support thanks to xps-genesis!
-
-Version 0.7.7 (2020-07-20)
-========================
- * White panda is no longer supported, upgrade to comma two or black panda
- * Improved vehicle model estimation using high precision localizer
- * Improved thermal management on comma two
- * Improved autofocus for road-facing camera
- * Improved noise performance for driver-facing camera
- * Block lane change start using blindspot monitor on select Toyota, Hyundai, and Subaru
- * Fix GM ignition detection
- * Code cleanup and smaller release sizes
- * Hyundai Sonata 2020 promoted to officially supported car
- * Hyundai Ioniq Electric Limited 2019 and Ioniq SE 2020 support thanks to baldwalker!
- * Subaru Forester 2019 and Ascent 2019 support thanks to martinl!
-
-Version 0.7.6.1 (2020-06-16)
-========================
- * Hotfix: update kernel on some comma twos (orders #8570-#8680)
-
-Version 0.7.6 (2020-06-05)
-========================
- * White panda is deprecated, upgrade to comma two or black panda
- * 2017 Nissan X-Trail, 2018-19 Leaf and 2019 Rogue support thanks to avolmensky!
- * 2017 Mazda CX-5 support in dashcam mode thanks to Jafaral!
- * Huge CPU savings in modeld by using thneed!
- * Lots of code cleanup and refactors
-
 Version 0.7.5 (2020-05-13)
 ========================
- * Right-Hand Drive support for both driving and driver monitoring!
- * New driving model: improved at sharp turns and lead speed estimation
- * New driver monitoring model: overall improvement on comma two
- * Driver camera preview in settings to improve mounting position
- * Added support for many Hyundai, Kia, Genesis models thanks to xx979xx!
- * Improved lateral tuning for 2020 Toyota Rav 4 (hybrid)
+* Right-Hand Drive support for both driving and driver monitoring!
+* New driving model: improved at sharp turns and lead speed estimation
+* New driver monitoring model: overall improvement on comma two
+* Driver camera preview in settings to improve mounting position
+* Added support for many Hyundai, Kia, Genesis models thanks to xx979xx!
+* 2019 Nissan X-Trail and 2018 Nissan Leaf support thanks to avolmensky!
+* Improved lateral tuning for 2020 Toyota Rav 4 (hybrid)

 Version 0.7.4 (2020-03-20)
 ========================
@@ -492,96 +442,96 @@ Version 0.3.4 (2017-07-28)

 Version 0.3.3  (2017-06-28)
 ===========================
- * Improved model trained on more data
- * Alpha CR-V support thanks to energee and johnnwvs!
- * Using the opendbc project for DBC files
- * Minor performance improvements
- * UI update thanks to pjlao307
- * Power off button
- * 6% more torque on the Civic
+  * Improved model trained on more data
+  * Alpha CR-V support thanks to energee and johnnwvs!
+  * Using the opendbc project for DBC files
+  * Minor performance improvements
+  * UI update thanks to pjlao307
+  * Power off button
+  * 6% more torque on the Civic

 Version 0.3.2  (2017-05-22)
 ===========================
- * Minor stability bugfixes
- * Added metrics and rear view mirror disable to settings
- * Update model with more crowdsourced data
+  * Minor stability bugfixes
+  * Added metrics and rear view mirror disable to settings
+  * Update model with more crowdsourced data

 Version 0.3.1  (2017-05-17)
 ===========================
- * visiond stability bugfix
- * Add logging for angle and flashing
+  * visiond stability bugfix
+  * Add logging for angle and flashing

 Version 0.3.0  (2017-05-12)
 ===========================
- * Add CarParams struct to improve the abstraction layer
- * Refactor visiond IPC to support multiple clients
- * Add raw GPS and beginning support for navigation
- * Improve model in visiond using crowdsourced data
- * Add improved system logging to diagnose instability
- * Rewrite baseui in React Native
- * Moved calibration to the cloud
+  * Add CarParams struct to improve the abstraction layer
+  * Refactor visiond IPC to support multiple clients
+  * Add raw GPS and beginning support for navigation
+  * Improve model in visiond using crowdsourced data
+  * Add improved system logging to diagnose instability
+  * Rewrite baseui in React Native
+  * Moved calibration to the cloud

 Version 0.2.9  (2017-03-01)
 ===========================
- * Retain compatibility with NEOS v1
+  * Retain compatibility with NEOS v1

 Version 0.2.8  (2017-02-27)
 ===========================
- * Fix bug where frames were being dropped in minute 71
+  * Fix bug where frames were being dropped in minute 71

 Version 0.2.7  (2017-02-08)
 ===========================
- * Better performance and pictures at night
- * Fix ptr alignment issue in boardd
- * Fix brake error light, fix crash if too cold
+  * Better performance and pictures at night
+  * Fix ptr alignment issue in boardd
+  * Fix brake error light, fix crash if too cold

 Version 0.2.6  (2017-01-31)
 ===========================
- * Fix bug in visiond model execution
+  * Fix bug in visiond model execution

 Version 0.2.5  (2017-01-30)
 ===========================
- * Fix race condition in manager
+  * Fix race condition in manager

 Version 0.2.4  (2017-01-27)
 ===========================
- * OnePlus 3T support
- * Enable installation as NEOS app
- * Various minor bugfixes
+  * OnePlus 3T support
+  * Enable installation as NEOS app
+  * Various minor bugfixes

 Version 0.2.3  (2017-01-11)
 ===========================
- * Reduce space usage by 80%
- * Add better logging
- * Add Travis CI
+  * Reduce space usage by 80%
+  * Add better logging
+  * Add Travis CI

 Version 0.2.2  (2017-01-10)
 ===========================
- * Board triggers started signal on CAN messages
- * Improved autoexposure
- * Handle out of space, improve upload status
+  * Board triggers started signal on CAN messages
+  * Improved autoexposure
+  * Handle out of space, improve upload status

 Version 0.2.1  (2016-12-14)
 ===========================
- * Performance improvements, removal of more numpy
- * Fix boardd process priority
- * Make counter timer reset on use of steering wheel
+  * Performance improvements, removal of more numpy
+  * Fix boardd process priority
+  * Make counter timer reset on use of steering wheel

 Version 0.2  (2016-12-12)
 =========================
- * Car/Radar abstraction layers have shipped, see cereal/car.capnp
- * controlsd has been refactored
- * Shipped plant model and testing maneuvers
- * visiond exits more gracefully now
- * Hardware encoder in visiond should always init
- * ui now turns off the screen after 30 seconds
- * Switch to openpilot release branch for future releases
- * Added preliminary Docker container to run tests on PC
+  * Car/Radar abstraction layers have shipped, see cereal/car.capnp
+  * controlsd has been refactored
+  * Shipped plant model and testing maneuvers
+  * visiond exits more gracefully now
+  * Hardware encoder in visiond should always init
+  * ui now turns off the screen after 30 seconds
+  * Switch to openpilot release branch for future releases
+  * Added preliminary Docker container to run tests on PC

 Version 0.1  (2016-11-29)
 =========================
- * Initial release of openpilot
- * Adaptive cruise control is working
- * Lane keep assist is working
- * Support for Acura ILX 2016 with AcuraWatch Plus
- * Support for Honda Civic 2016 Touring Edition
+  * Initial release of openpilot
+  * Adaptive cruise control is working
+  * Lane keep assist is working
+  * Support for Acura ILX 2016 with AcuraWatch Plus
+  * Support for Honda Civic 2016 Touring Edition
@@ -1,7 +1,7 @@
 openpilot Safety
 ======

-openpilot is an Adaptive Cruise Control (ACC) and Automated Lane Centering (ALC) system.
+openpilot is an Adaptive Cruise Control (ACC) and Automated Lane Centering (ALC) system. 
 Like other ACC and ALC systems, openpilot is a failsafe passive system and it requires the
 driver to be alert and to pay attention at all times.

@@ -22,7 +22,7 @@ hardware-in-the-loop and in-vehicle tests before each software release.
 Following Hazard and Risk Analysis and FMEA, at a very high level, we have designed openpilot
 ensuring two main safety requirements.

-1. The driver must always be capable to immediately retake manual control of the vehicle,
+1. The driver must always be capable to immediately retake manual control of the vehicle, 
   by stepping on either pedal or by pressing the cancel button.
 2. The vehicle must not alter its trajectory too quickly for the driver to safely
   react. This means that while the system is engaged, the actuators are constrained
@@ -1,14 +1,8 @@
-import Cython
-import distutils
 import os
-import shutil
 import subprocess
 import sys
 import platform

-TICI = os.path.isfile('/TICI')
-Decider('MD5-timestamp')
-
 AddOption('--test',
          action='store_true',
          help='build test files')
@@ -17,19 +11,13 @@ AddOption('--asan',
          action='store_true',
          help='turn on ASAN')

-# Rebuild cython extensions if python, distutils, or cython change
-cython_dependencies = [Value(v) for v in (sys.version, distutils.__version__, Cython.__version__)]
-Export('cython_dependencies')
-
-real_arch = arch = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()
+arch = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()
 if platform.system() == "Darwin":
  arch = "Darwin"
-
-if arch == "aarch64" and TICI:
+if arch == "aarch64" and not os.path.isdir("/system"):
  arch = "larch64"

-USE_WEBCAM = os.getenv("USE_WEBCAM") is not None
-QCOM_REPLAY = arch == "aarch64" and os.getenv("QCOM_REPLAY") is not None
+webcam = bool(ARGUMENTS.get("use_webcam", 0))

 if arch == "aarch64" or arch == "larch64":
  lenv = {
@@ -48,59 +36,53 @@ if arch == "aarch64" or arch == "larch64":

  libpath = [
    "/usr/lib",
+    "/data/data/com.termux/files/usr/lib",
    "/system/vendor/lib64",
    "/system/comma/usr/lib",
    "#phonelibs/nanovg",
  ]

  if arch == "larch64":
-    libpath += [
-      "#phonelibs/snpe/larch64",
-      "#phonelibs/libyuv/larch64/lib",
-      "/usr/lib/aarch64-linux-gnu"
-    ]
+    cpppath += ["#phonelibs/capnp-cpp/include"]
+    libpath += ["#phonelibs/snpe/larch64"]
+    libpath += ["#phonelibs/libyuv/larch64/lib"]
+    libpath += ["#external/capnparm/lib", "/usr/lib/aarch64-linux-gnu"]
    cflags = ["-DQCOM2", "-mcpu=cortex-a57"]
    cxxflags = ["-DQCOM2", "-mcpu=cortex-a57"]
    rpath = ["/usr/local/lib"]
  else:
-    libpath += [
-      "#phonelibs/snpe/aarch64",
-      "#phonelibs/libyuv/lib",
-      "/system/vendor/lib64"
-    ]
+    libpath += ["#phonelibs/snpe/aarch64"]
+    libpath += ["#phonelibs/libyuv/lib"]
    cflags = ["-DQCOM", "-mcpu=cortex-a57"]
    cxxflags = ["-DQCOM", "-mcpu=cortex-a57"]
-    rpath = []
-
-    if QCOM_REPLAY:
-      cflags += ["-DQCOM_REPLAY"]
-      cxxflags += ["-DQCOM_REPLAY"]
+    rpath = ["/system/vendor/lib64"]

 else:
-  cflags = []
-  cxxflags = []
-
  lenv = {
    "PATH": "#external/bin:" + os.environ['PATH'],
  }
  cpppath = [
+    "#phonelibs/capnp-cpp/include",
+    "#phonelibs/zmq/x64/include",
    "#external/tensorflow/include",
  ]

  if arch == "Darwin":
    libpath = [
+      "#phonelibs/capnp-cpp/mac/lib",
      "#phonelibs/libyuv/mac/lib",
      "#cereal",
      "#selfdrive/common",
      "/usr/local/lib",
      "/System/Library/Frameworks/OpenGL.framework/Libraries",
    ]
-    cflags += ["-DGL_SILENCE_DEPRECATION"]
-    cxxflags += ["-DGL_SILENCE_DEPRECATION"]
  else:
    libpath = [
+      "#phonelibs/capnp-cpp/x64/lib",
      "#phonelibs/snpe/x86_64-linux-clang",
+      "#phonelibs/zmq/x64/lib",
      "#phonelibs/libyuv/x64/lib",
+      "#external/zmq/lib",
      "#external/tensorflow/lib",
      "#cereal",
      "#selfdrive/common",
@@ -108,22 +90,20 @@ else:
      "/usr/local/lib",
    ]

-  rpath = [
-    "phonelibs/snpe/x86_64-linux-clang",
-    "external/tensorflow/lib",
-    "cereal",
-    "selfdrive/common"
-  ]
+  rpath = ["phonelibs/capnp-cpp/x64/lib",
+           "phonelibs/zmq/x64/lib",
+           "external/tensorflow/lib",
+           "cereal",
+           "selfdrive/common"]

  # allows shared libraries to work globally
  rpath = [os.path.join(os.getcwd(), x) for x in rpath]

-if GetOption('asan'):
-  ccflags_asan = ["-fsanitize=address", "-fno-omit-frame-pointer"]
-  ldflags_asan = ["-fsanitize=address"]
-else:
-  ccflags_asan = []
-  ldflags_asan = []
+  cflags = []
+  cxxflags = []
+
+ccflags_asan = ["-fsanitize=address", "-fno-omit-frame-pointer"] if GetOption('asan') else []
+ldflags_asan = ["-fsanitize=address"] if GetOption('asan') else []

 # change pythonpath to this
 lenv["PYTHONPATH"] = Dir("#").path
@@ -134,14 +114,11 @@ env = Environment(
    "-g",
    "-fPIC",
    "-O2",
-    "-Wunused",
-    "-Werror",
-    "-Wno-unknown-warning-option",
-    "-Wno-deprecated-register",
-    "-Wno-register",
-    "-Wno-inconsistent-missing-override",
-    "-Wno-c99-designator",
-    "-Wno-reorder-init-list",
+    "-Werror=implicit-function-declaration",
+    "-Werror=incompatible-pointer-types",
+    "-Werror=int-conversion",
+    "-Werror=return-type",
+    "-Werror=format-extra-args",
  ] + cflags + ccflags_asan,

  CPPPATH=cpppath + [
@@ -151,7 +128,9 @@ env = Environment(
    "#phonelibs/libyuv/include",
    "#phonelibs/openmax/include",
    "#phonelibs/json11",
+    "#phonelibs/eigen",
    "#phonelibs/curl/include",
+    #"#phonelibs/opencv/include", # use opencv4 instead
    "#phonelibs/libgralloc/include",
    "#phonelibs/android_frameworks_native/include",
    "#phonelibs/android_hardware_libhardware/include",
@@ -164,8 +143,6 @@ env = Environment(
    "#selfdrive/camerad/include",
    "#selfdrive/loggerd/include",
    "#selfdrive/modeld",
-    "#selfdrive/sensord",
-    "#selfdrive/ui",
    "#cereal/messaging",
    "#cereal",
    "#opendbc/can",
@@ -178,67 +155,17 @@ env = Environment(
  RPATH=rpath,

  CFLAGS=["-std=gnu11"] + cflags,
-  CXXFLAGS=["-std=c++1z"] + cxxflags,
-  LIBPATH=libpath + [
+  CXXFLAGS=["-std=c++14"] + cxxflags,
+  LIBPATH=libpath +
+  [
    "#cereal",
    "#selfdrive/common",
    "#phonelibs",
  ]
 )

-qt_env = None
-if arch in ["x86_64", "Darwin", "larch64"]:
-  qt_env = env.Clone()
-
-  if arch == "Darwin":
-    qt_env['QTDIR'] = "/usr/local/opt/qt"
-    QT_BASE = "/usr/local/opt/qt/"
-    qt_dirs = [
-      QT_BASE + "include/",
-      QT_BASE + "include/QtWidgets",
-      QT_BASE + "include/QtGui",
-      QT_BASE + "include/QtCore",
-      QT_BASE + "include/QtDBus",
-      QT_BASE + "include/QtMultimedia",
-    ]
-    qt_env["LINKFLAGS"] += ["-F" + QT_BASE + "lib"]
-  else:
-    qt_env['QTDIR'] = "/usr"
-    qt_dirs = [
-      f"/usr/include/{real_arch}-linux-gnu/qt5",
-      f"/usr/include/{real_arch}-linux-gnu/qt5/QtWidgets",
-      f"/usr/include/{real_arch}-linux-gnu/qt5/QtGui",
-      f"/usr/include/{real_arch}-linux-gnu/qt5/QtCore",
-      f"/usr/include/{real_arch}-linux-gnu/qt5/QtDBus",
-      f"/usr/include/{real_arch}-linux-gnu/qt5/QtMultimedia",
-      f"/usr/include/{real_arch}-linux-gnu/qt5/QtGui/5.5.1/QtGui",
-    ]
-
-  qt_env.Tool('qt')
-  qt_env['CPPPATH'] += qt_dirs
-  qt_flags = [
-    "-D_REENTRANT",
-    "-DQT_NO_DEBUG",
-    "-DQT_WIDGETS_LIB",
-    "-DQT_GUI_LIB",
-    "-DQT_CORE_LIB"
-  ]
-  qt_env['CXXFLAGS'] += qt_flags
-
 if os.environ.get('SCONS_CACHE'):
-  cache_dir = '/tmp/scons_cache'
-
-  if os.getenv('CI'):
-    branch = os.getenv('GIT_BRANCH')
-
-    if QCOM_REPLAY:
-      cache_dir = '/tmp/scons_cache_qcom_replay'
-    elif branch is not None and branch != 'master':
-      cache_dir_branch = '/tmp/scons_cache_' + branch
-      if not os.path.isdir(cache_dir_branch) and os.path.isdir(cache_dir):
-        shutil.copytree(cache_dir, cache_dir_branch)
-      cache_dir = cache_dir_branch
-  CacheDir(cache_dir)
+  CacheDir('/tmp/scons_cache')

 node_interval = 5
 node_count = 0
@@ -262,8 +189,11 @@ def abspath(x):
    return x[0].path.rsplit("/", 1)[1][:-3]

 # still needed for apks
-zmq = 'zmq'
-Export('env', 'qt_env', 'arch', 'zmq', 'SHARED', 'USE_WEBCAM', 'QCOM_REPLAY')
+if arch == 'larch64':
+  zmq = 'zmq'
+else:
+  zmq = FindFile("libzmq.a", libpath)
+Export('env', 'arch', 'zmq', 'SHARED', 'webcam')

 # cereal and messaging are shared with the system
 SConscript(['cereal/SConscript'])
@@ -291,11 +221,11 @@ SConscript(['opendbc/can/SConscript'])

 SConscript(['common/SConscript'])
 SConscript(['common/kalman/SConscript'])
-SConscript(['common/transformations/SConscript'])
 SConscript(['phonelibs/SConscript'])

-SConscript(['selfdrive/camerad/SConscript'])
-SConscript(['selfdrive/modeld/SConscript'])
+if arch != "Darwin":
+  SConscript(['selfdrive/camerad/SConscript'])
+  SConscript(['selfdrive/modeld/SConscript'])

 SConscript(['selfdrive/controls/lib/cluster/SConscript'])
 SConscript(['selfdrive/controls/lib/lateral_mpc/SConscript'])
@@ -304,18 +234,15 @@ SConscript(['selfdrive/controls/lib/longitudinal_mpc_model/SConscript'])

 SConscript(['selfdrive/boardd/SConscript'])
 SConscript(['selfdrive/proclogd/SConscript'])
-SConscript(['selfdrive/clocksd/SConscript'])

+SConscript(['selfdrive/ui/SConscript'])
 SConscript(['selfdrive/loggerd/SConscript'])

-SConscript(['selfdrive/locationd/SConscript'])
-SConscript(['selfdrive/locationd/models/SConscript'])
-SConscript(['selfdrive/sensord/SConscript'])
-SConscript(['selfdrive/ui/SConscript'])
-
-if arch != "Darwin":
+if arch == "aarch64":
  SConscript(['selfdrive/logcatd/SConscript'])
+  SConscript(['selfdrive/sensord/SConscript'])
+  SConscript(['selfdrive/clocksd/SConscript'])

-
-if arch == "x86_64":
-  SConscript(['tools/lib/index_log/SConscript'])
+SConscript(['selfdrive/locationd/SConscript'])
+SConscript(['selfdrive/locationd/kalman/SConscript'])
+SConscript(['tools/lib/index_log/SConscript'])
@@ -0,0 +1 @@
+.sconsign.dblite
@@ -0,0 +1,23 @@
+name: Tests
+
+on: [push, pull_request]
+
+jobs:
+  test:
+
+    runs-on: ubuntu-16.04
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Build docker image
+      run: docker build -t cereal .
+    - name: Unit Tests
+      run: |
+        docker run cereal bash -c "scons --test --asan -j$(nproc) && messaging/test_runner"
+    - name: Test ZMQ
+      run: |
+        docker run cereal bash -c "ZMQ=1 python -m unittest discover ."
+    - name: Test MSGQ
+      run: |
+        docker run cereal bash -c "MSGQ=1 python -m unittest discover ."
+
@@ -11,4 +11,4 @@ libmessaging_shared.*
 services.h
 .sconsign.dblite
 libcereal_shared.*
-.mypy_cache/
+
@@ -0,0 +1,19 @@
+from ubuntu:16.04
+
+RUN apt-get update && apt-get install -y libzmq3-dev clang wget git autoconf libtool curl make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl
+
+RUN curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
+ENV PATH="/root/.pyenv/bin:/root/.pyenv/shims:${PATH}"
+RUN pyenv install 3.7.3
+RUN pyenv global 3.7.3
+RUN pyenv rehash
+RUN pip3 install pyyaml==5.1.2 Cython==0.29.14 scons==3.1.1 pycapnp==0.6.4
+
+WORKDIR /project/cereal
+COPY install_capnp.sh .
+RUN ./install_capnp.sh
+
+ENV PYTHONPATH=/project
+
+COPY . .
+RUN scons -c && scons -j$(nproc)
@@ -0,0 +1,41 @@
+What is cereal?
+----
+
+cereal is both a messaging spec for robotics systems as well as generic high performance IPC pub sub messaging with a single publisher and multiple subscribers.
+
+Imagine this use case:
+* A sensor process reads gyro measurements directly from an IMU and publishes a sensorEvents packet
+* A calibration process subscribes to the sensorEvents packet to use the IMU
+* A localization process subscribes to the sensorEvents packet to use the IMU also
+
+
+Messaging Spec
+----
+
+You'll find the message types in [log.capnp](log.capnp). It uses [Cap'n proto](https://capnproto.org/capnp-tool.html) and defines one struct called Event.
+
+All Events have a logMonoTime and a valid. Then a big union defines the packet type.
+
+
+Pub Sub Backends
+----
+
+cereal supports two backends, one based on [zmq](https://zeromq.org/), the other called msgq, a custom pub sub based on shared memory that doesn't require the bytes to pass through the kernel.
+
+Example
+---
+```python
+import cereal.messaging as messaging
+
+# in subscriber
+sm = messaging.SubMaster(['sensorEvents'])
+while 1:
+  sm.update()
+  print(sm['sensorEvents'])
+
+# in publisher
+pm = messaging.PubMaster(['sensorEvents'])
+dat = messaging.new_message('sensorEvents', size=1)
+dat.sensorEvents[0] = {"gyro": {"v": [0.1, -0.1, 0.1]}}
+pm.send('sensorEvents', dat)
+```
@@ -1,41 +1,41 @@
-Import('env', 'arch', 'zmq', 'cython_dependencies')
-
-import shutil
+Import('env', 'arch', 'zmq')

 gen_dir = Dir('gen')
 messaging_dir = Dir('messaging')

 # TODO: remove src-prefix and cereal from command string. can we set working directory?
 env.Command(["gen/c/include/c++.capnp.h", "gen/c/include/java.capnp.h"], [], "mkdir -p " + gen_dir.path + "/c/include && touch $TARGETS")
-env.Command(['gen/cpp/car.capnp.c++', 'gen/cpp/log.capnp.c++', 'gen/cpp/car.capnp.h', 'gen/cpp/log.capnp.h'],
-            ['car.capnp', 'log.capnp'],
-            'capnpc $SOURCES --src-prefix=cereal -o c++:' + gen_dir.path + '/cpp/')
-
+env.Command(
+  ['gen/cpp/car.capnp.c++', 'gen/cpp/log.capnp.c++', 'gen/cpp/car.capnp.h', 'gen/cpp/log.capnp.h'],
+  ['car.capnp', 'log.capnp'],
+  'capnpc $SOURCES --src-prefix=cereal -o c++:' + gen_dir.path + '/cpp/')
+import shutil
 if shutil.which('capnpc-java'):
-  env.Command(['gen/java/Car.java', 'gen/java/Log.java'],
-              ['car.capnp', 'log.capnp'],
-              'capnpc $SOURCES --src-prefix=cereal -o java:' + gen_dir.path + '/java/')
+  env.Command(
+    ['gen/java/Car.java', 'gen/java/Log.java'],
+    ['car.capnp', 'log.capnp'],
+    'capnpc $SOURCES --src-prefix=cereal -o java:' + gen_dir.path + '/java/')

 # TODO: remove non shared cereal and messaging
 cereal_objects = env.SharedObject([
-  'gen/cpp/car.capnp.c++',
-  'gen/cpp/log.capnp.c++',
-])
+    'gen/cpp/car.capnp.c++',
+    'gen/cpp/log.capnp.c++',
+  ])

 env.Library('cereal', cereal_objects)
 env.SharedLibrary('cereal_shared', cereal_objects)

 cereal_dir = Dir('.')
-services_h = env.Command(['services.h'],
-                          ['service_list.yaml', 'services.py'],
-                          'python3 ' + cereal_dir.path + '/services.py > $TARGET')
+services_h = env.Command(
+  ['services.h'],
+  ['service_list.yaml', 'services.py'],
+  'python3 ' + cereal_dir.path + '/services.py > $TARGET')

 messaging_objects = env.SharedObject([
  'messaging/messaging.cc',
  'messaging/impl_zmq.cc',
  'messaging/impl_msgq.cc',
  'messaging/msgq.cc',
-  'messaging/socketmaster.cc',
 ])

 messaging_lib = env.Library('messaging', messaging_objects)
@@ -43,10 +43,8 @@ Depends('messaging/impl_zmq.cc', services_h)

 # note, this rebuilds the deps shared, zmq is statically linked to make APK happy
 # TODO: get APK to load system zmq to remove the static link
-if arch == "aarch64":
-  zmq_static = FindFile("libzmq.a", "/usr/lib")
-  shared_lib_shared_lib = [zmq_static, 'm', 'stdc++', "gnustl_shared", "kj", "capnp"]
-  env.SharedLibrary('messaging_shared', messaging_objects, LIBS=shared_lib_shared_lib)
+shared_lib_shared_lib = [zmq, 'm', 'stdc++'] + ["gnustl_shared"] if arch == "aarch64" else [zmq]
+env.SharedLibrary('messaging_shared', messaging_objects, LIBS=shared_lib_shared_lib)

 env.Program('messaging/bridge', ['messaging/bridge.cc'], LIBS=[messaging_lib, 'zmq'])
 Depends('messaging/bridge.cc', services_h)
@@ -55,9 +53,9 @@ Depends('messaging/bridge.cc', services_h)
 #env.Program('messaging/demo', ['messaging/demo.cc'], LIBS=[messaging_lib, 'zmq'])


-env.Command(['messaging/messaging_pyx.so', 'messaging/messaging_pyx.cpp'],
-            cython_dependencies + [messaging_lib, 'messaging/messaging_pyx_setup.py', 'messaging/messaging_pyx.pyx', 'messaging/messaging.pxd'],
-            "cd " + messaging_dir.path + " && python3 messaging_pyx_setup.py build_ext --inplace")
+env.Command(['messaging/messaging_pyx.so'],
+  [messaging_lib, 'messaging/messaging_pyx_setup.py', 'messaging/messaging_pyx.pyx', 'messaging/messaging.pxd'],
+  "cd " + messaging_dir.path + " && python3 messaging_pyx_setup.py build_ext --inplace")


 if GetOption('test'):
@@ -0,0 +1,49 @@
+import os
+import subprocess
+
+zmq = 'zmq'
+arch = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()
+
+cereal_dir = Dir('.')
+
+cpppath = [
+    cereal_dir,
+    '/usr/lib/include',
+]
+
+AddOption('--test',
+          action='store_true',
+          help='build test files')
+
+AddOption('--asan',
+          action='store_true',
+          help='turn on ASAN')
+
+ccflags_asan = ["-fsanitize=address", "-fno-omit-frame-pointer"] if GetOption('asan') else []
+ldflags_asan = ["-fsanitize=address"] if GetOption('asan') else []
+
+env = Environment(
+  ENV=os.environ,
+  CC='clang',
+  CXX='clang++',
+  CCFLAGS=[
+    "-g",
+    "-fPIC",
+    "-O2",
+    "-Werror=implicit-function-declaration",
+    "-Werror=incompatible-pointer-types",
+    "-Werror=int-conversion",
+    "-Werror=return-type",
+    "-Werror=format-extra-args",
+  ] + ccflags_asan,
+  LDFLAGS=ldflags_asan,
+  LINKFLAGS=ldflags_asan,
+
+  CFLAGS="-std=gnu11",
+  CXXFLAGS="-std=c++14",
+  CPPPATH=cpppath,
+)
+
+
+Export('env', 'zmq', 'arch')
+SConscript(['SConscript'])
@@ -1,4 +1,3 @@
-# pylint: skip-file
 import os
 import capnp

@@ -13,18 +13,19 @@ struct CarEvent @0x9b1657f34caf3ad3 {
  name @0 :EventName;
  enable @1 :Bool;
  noEntry @2 :Bool;
-  warning @3 :Bool;   # alerts presented only when  enabled or soft disabling
+  warning @3 :Bool;
  userDisable @4 :Bool;
  softDisable @5 :Bool;
  immediateDisable @6 :Bool;
  preEnable @7 :Bool;
-  permanent @8 :Bool; # alerts presented regardless of openpilot state
+  permanent @8 :Bool;

  enum EventName @0xbaa8c5d505f727de {
    # TODO: copy from error list
    canError @0;
    steerUnavailable @1;
    brakeUnavailable @2;
+    gasUnavailable @3;
    wrongGear @4;
    doorOpen @5;
    seatbeltNotLatched @6;
@@ -37,6 +38,7 @@ struct CarEvent @0x9b1657f34caf3ad3 {
    pedalPressed @13;
    cruiseDisabled @14;
    radarCanError @15;
+    dataNeeded @16;
    speedTooLow @17;
    outOfSpace @18;
    overheat @19;
@@ -47,22 +49,29 @@ struct CarEvent @0x9b1657f34caf3ad3 {
    pcmDisable @24;
    noTarget @25;
    radarFault @26;
+    modelCommIssueDEPRECATED @27;
    brakeHold @28;
    parkBrake @29;
    manualRestart @30;
    lowSpeedLockout @31;
    plannerError @32;
+    ipasOverride @33;
    debugAlert @34;
    steerTempUnavailableMute @35;
    resumeRequired @36;
    preDriverDistracted @37;
    promptDriverDistracted @38;
    driverDistracted @39;
+    geofence @40;
+    driverMonitorOn @41;
+    driverMonitorOff @42;
    preDriverUnresponsive @43;
    promptDriverUnresponsive @44;
    driverUnresponsive @45;
    belowSteerSpeed @46;
+    calibrationProgress @47;
    lowBattery @48;
+    invalidGiraffeHonda @49;
    vehicleModelInvalid @50;
    controlsFailed @51;
    sensorDataInvalid @52;
@@ -88,47 +97,14 @@ struct CarEvent @0x9b1657f34caf3ad3 {
    relayMalfunction @72;
    gasPressed @73;
    stockFcw @74;
-    startup @75;
-    startupNoCar @76;
-    startupNoControl @77;
-    startupMaster @78;
-    fcw @79;
-    steerSaturated @80;
-    whitePandaUnsupported @81;
-    startupGreyPanda @82;
-    belowEngageSpeed @84;
-    noGps @85;
-    wrongCruiseMode @87;
-    modeldLagging @89;
-    deviceFalling @90;
-    fanMalfunction @91;
-    cameraMalfunction @92;
-
-    gasUnavailableDEPRECATED @3;
-    dataNeededDEPRECATED @16;
-    modelCommIssueDEPRECATED @27;
-    ipasOverrideDEPRECATED @33;
-    geofenceDEPRECATED @40;
-    driverMonitorOnDEPRECATED @41;
-    driverMonitorOffDEPRECATED @42;
-    calibrationProgressDEPRECATED @47;
-    invalidGiraffeHondaDEPRECATED @49;
-    canErrorPersistentDEPRECATED @83;
-    focusRecoverActiveDEPRECATED @86;
-    neosUpdateRequiredDEPRECATED @88;
-    modelLagWarningDEPRECATED @93;
-
-    #dp
-    preLaneChangeLeftALC @94;
-    preLaneChangeRightALC @95;
-    manualSteeringRequired @96;
-    manualSteeringRequiredBlinkersOn @97;
-    leadCarMoving @98;
-
-    # timebomb assist
-    timebombWarn @99;
-    timebombBypassing @100;
-    timebombBypassed @101;
+    # dp
+    preLaneChangeLeftALC @75;
+    preLaneChangeRightALC @76;
+    laneChangeALC @77;
+    manualSteeringRequired @78;
+    manualSteeringRequiredBlinkersOn @79;
+    leadCarMoving @80;
+    leadCarDetected @81;
  }
 }

@@ -189,6 +165,9 @@ struct CarState {
  # clutch (manual transmission only)
  clutchPressed @28 :Bool;

+  # dp
+  lkMode @37 :Bool;
+
  # which packets this state came from
  canMonoTimes @12: List(UInt64);

@@ -196,10 +175,6 @@ struct CarState {
  leftBlindspot @33 :Bool; # Is there something blocking the left lane change
  rightBlindspot @34 :Bool; # Is there something blocking the right lane change

-  # dp
-  lkMode @37 :Bool;
-  stopSteering @38 :Bool; # timebomb - stopSteering
-
  struct WheelSpeeds {
    # optional wheel speeds
    fl @0 :Float32;
@@ -214,7 +189,6 @@ struct CarState {
    available @2 :Bool;
    speedOffset @3 :Float32;
    standstill @4 :Bool;
-    nonAdaptive @5 :Bool;
  }

  enum GearShifter {
@@ -494,8 +468,6 @@ struct CarParams {
    hondaBoschHarness @20;
    volkswagenPq @21;
    subaruLegacy @22;  # pre-Global platform
-    hyundaiLegacy @23;
-    hyundaiCommunity @24;
  }

  enum SteerControlType {
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+rm -r gen/ts
+rm -r gen/js
+
+mkdir gen/ts
+mkdir gen/js
+
+echo "Installing needed npm modules"
+npm i capnpc-ts capnp-ts
+
+capnpc -o node_modules/.bin/capnpc-ts:gen/ts log.capnp car.capnp
+capnpc -o node_modules/.bin/capnpc-ts:gen/ts car.capnp
+
+cat log.capnp | egrep '\([a-zA-Z]*\.[^\s]+\.[^s]+\)' | sed 's/^.*([a-zA-Z]*\.\([a-zA-Z.]*\)).*/\1/' | while read line
+do
+  TOKEN=`echo $line | sed 's/\./_/g'`
+  ROOT=`echo $line | sed 's/\..*$//g'`
+  cat gen/ts/log.capnp.ts | grep '^import.*'${TOKEN}
+  if [[ "$?" == "1" ]]
+  then
+    sed -i 's/^\(import {.*\)'${ROOT}'\(,*\) \(.*\)$/\1'${ROOT}', '${TOKEN}'\2 \3/' ./gen/ts/log.capnp.ts
+  fi
+done
+
+tsc ./gen/ts/* --lib es2015 --outDir ./gen/js
@@ -0,0 +1,40 @@
+set -e
+echo "Installing capnp"
+
+cd /tmp
+VERSION=0.6.1
+wget https://capnproto.org/capnproto-c++-${VERSION}.tar.gz
+tar xvf capnproto-c++-${VERSION}.tar.gz
+cd capnproto-c++-${VERSION}
+CXXFLAGS="-fPIC" ./configure
+
+make -j$(nproc)
+make install
+
+# manually build binaries statically
+g++ -std=gnu++11 -I./src -I./src -DKJ_HEADER_WARNINGS -DCAPNP_HEADER_WARNINGS -DCAPNP_INCLUDE_DIR=\"/usr/local/include\" -pthread -O2 -DNDEBUG -pthread -pthread -o .libs/capnp src/capnp/compiler/module-loader.o src/capnp/compiler/capnp.o  ./.libs/libcapnpc.a ./.libs/libcapnp.a ./.libs/libkj.a -lpthread -pthread
+
+g++ -std=gnu++11 -I./src -I./src -DKJ_HEADER_WARNINGS -DCAPNP_HEADER_WARNINGS -DCAPNP_INCLUDE_DIR=\"/usr/local/include\" -pthread -O2 -DNDEBUG -pthread -pthread -o .libs/capnpc-c++ src/capnp/compiler/capnpc-c++.o  ./.libs/libcapnp.a ./.libs/libkj.a -lpthread -pthread
+
+g++ -std=gnu++11 -I./src -I./src -DKJ_HEADER_WARNINGS -DCAPNP_HEADER_WARNINGS -DCAPNP_INCLUDE_DIR=\"/usr/local/include\" -pthread -O2 -DNDEBUG -pthread -pthread -o .libs/capnpc-capnp src/capnp/compiler/capnpc-capnp.o  ./.libs/libcapnp.a ./.libs/libkj.a -lpthread -pthread
+
+cp .libs/capnp /usr/local/bin/
+cp .libs/capnpc-c++ /usr/local/bin/
+cp .libs/capnpc-capnp /usr/local/bin/
+cp .libs/*.a /usr/local/lib
+
+cd /tmp
+echo "Installing c-capnp"
+git clone https://github.com/commaai/c-capnproto.git
+cd c-capnproto
+git submodule update --init --recursive
+autoreconf -f -i -s
+CXXFLAGS="-fPIC" ./configure
+make -j$(nproc)
+make install
+
+# manually build binaries statically
+gcc -fPIC -o .libs/capnpc-c compiler/capnpc-c.o compiler/schema.capnp.o compiler/str.o  ./.libs/libcapnp_c.a
+
+cp .libs/capnpc-c /usr/local/bin/
+cp .libs/*.a /usr/local/lib
@@ -130,7 +130,6 @@ struct FrameData {
  focusVal @16 :List(Int16);
  focusConf @17 :List(UInt8);
  sharpnessScore @18 :List(UInt16);
-  recoverState @19 :Int32;

  frameType @7 :FrameType;
  timestampSof @8 :UInt64;
@@ -186,7 +185,6 @@ struct SensorEventData {
    gyroUncalibrated @12 :SensorVec;
    proximity @13: Float32;
    light @14: Float32;
-    temperature @15: Float32;
  }
  source @8 :SensorSource;

@@ -204,8 +202,6 @@ struct SensorEventData {
    lsm6ds3 @5;   # accelerometer (c2)
    bmp280 @6;    # barometer (c2)
    mmc3416x @7;  # magnetometer (c2)
-    bmx055 @8;
-    rpr0521 @9;
  }
 }

@@ -270,15 +266,14 @@ struct CanData {
 }

 struct ThermalData {
-  # Deprecated
-  cpu0DEPRECATED @0 :UInt16;
-  cpu1DEPRECATED @1 :UInt16;
-  cpu2DEPRECATED @2 :UInt16;
-  cpu3DEPRECATED @3 :UInt16;
-  memDEPRECATED @4 :UInt16;
-  gpuDEPRECATED @5 :UInt16;
-  batDEPRECATED @6 :UInt32;
-  pa0DEPRECATED @21 :UInt16;
+  cpu0 @0 :UInt16;
+  cpu1 @1 :UInt16;
+  cpu2 @2 :UInt16;
+  cpu3 @3 :UInt16;
+  mem @4 :UInt16;
+  gpu @5 :UInt16;
+  bat @6 :UInt32;
+  pa0 @21 :UInt16;

  # not thermal
  freeSpace @7 :Float32;
@@ -290,7 +285,6 @@ struct ThermalData {
  networkType @22 :NetworkType;
  offroadPowerUsage @23 :UInt32;  # Power usage since going offroad in uWh
  networkStrength @24 :NetworkStrength;
-  carBatteryCapacity @25 :UInt32;  # Estimated remaining car battery capacity in uWh

  fanSpeed @10 :UInt16;
  started @11 :Bool;
@@ -303,11 +297,7 @@ struct ThermalData {
  memUsedPercent @19 :Int8;
  cpuPerc @20 :Int8;

-  cpu @26 :List(Float32);
-  gpu @27 :List(Float32);
-  mem @28 :Float32;
-  bat @29 :Float32;
-  ambient @30 :Float32;
+  ipAddr @25 :Text; # dp

  enum ThermalStatus {
    green @0;   # all processes run
@@ -383,9 +373,6 @@ struct HealthData {
    interruptRateTim1 @16;
    interruptRateTim3 @17;
    registerDivergent @18;
-    interruptRateKlineInit @19;
-    interruptRateClockSource @20;
-    interruptRateTim9 @21;
    # Update max fault type in boardd when adding faults
  }

@@ -396,7 +383,6 @@ struct HealthData {
    blackPanda @3;
    pedal @4;
    uno @5;
-    dos @6;
  }

  enum UsbPowerMode {
@@ -454,22 +440,18 @@ struct RadarState @0x9a185389d6fdd05f {
 struct LiveCalibrationData {
  # deprecated
  warpMatrix @0 :List(Float32);
-
  # camera_frame_from_model_frame
  warpMatrix2 @5 :List(Float32);
  warpMatrixBig @6 :List(Float32);
-
  calStatus @1 :Int8;
  calCycle @2 :Int32;
  calPerc @3 :Int8;
-  validBlocks @9 :Int32;

  # view_frame_from_road_frame
  # ui's is inversed needs new
  extrinsicMatrix @4 :List(Float32);
  # the direction of travel vector in device frame
  rpyCalib @7 :List(Float32);
-  rpyCalibSpread @8 :List(Float32);
 }

 struct LiveTracks {
@@ -571,7 +553,7 @@ struct ControlsState @0x97ff69c53601abf1 {

  enum AlertStatus {
    normal @0;       # low priority alert for user's convenience
-    userPrompt @1;   # mid priority alert that might require user intervention
+    userPrompt @1;   # mid piority alert that might require user intervention
    critical @2;     # high priority alert that needs immediate user intervention
  }

@@ -615,7 +597,9 @@ struct ControlsState @0x97ff69c53601abf1 {
    output @3 :Float32;
    lqrOutput @4 :Float32;
    saturated @5 :Bool;
-  }
+   }
+
+
 }

 struct LiveEventData {
@@ -625,11 +609,7 @@ struct LiveEventData {

 struct ModelData {
  frameId @0 :UInt32;
-  frameAge @12 :UInt32;
-  frameDropPerc @13 :Float32;
  timestampEof @9 :UInt64;
-  modelExecutionTime @14 :Float32;
-  rawPred @15 :Data;

  path @1 :PathData;
  leftLane @2 :PathData;
@@ -649,7 +629,6 @@ struct ModelData {
    std @2 :Float32;
    stds @3 :List(Float32);
    poly @4 :List(Float32);
-    validLen @5 :Float32;
  }

  struct LeadData {
@@ -690,56 +669,6 @@ struct ModelData {
  }
 }

-
-struct ModelDataV2 {
-  frameId @0 :UInt32;
-  frameAge @1 :UInt32;
-  frameDropPerc @2 :Float32;
-  timestampEof @3 :UInt64;
-  modelExecutionTime @15 :Float32;
-  rawPred @16 :Data;
-
-  position @4 :XYZTData;
-  orientation @5 :XYZTData;
-  velocity @6 :XYZTData;
-  orientationRate @7 :XYZTData;
-  laneLines @8 :List(XYZTData);
-  laneLineProbs @9 :List(Float32);
-  laneLineStds @13 :List(Float32);
-  roadEdges @10 :List(XYZTData);
-  roadEdgeStds @14 :List(Float32);
-  leads @11 :List(LeadDataV2);
-
-  meta @12 :MetaData;
-
-  struct XYZTData {
-    x @0 :List(Float32);
-    y @1 :List(Float32);
-    z @2 :List(Float32);
-    t @3 :List(Float32);
-    xStd @4 :List(Float32);
-    yStd @5 :List(Float32);
-    zStd @6 :List(Float32);
-  }
-
-  struct LeadDataV2 {
-    prob @0 :Float32;
-    t @1 :Float32;
-    xyva @2 :List(Float32);
-    xyvaStd @3 :List(Float32);
-  }
-
-  struct MetaData {
-    engagedProb @0 :Float32;
-    desirePrediction @1 :List(Float32);
-    brakeDisengageProb @2 :Float32;
-    gasDisengageProb @3 :Float32;
-    steerOverrideProb @4 :Float32;
-    desireState @5 :List(Float32);
-  }
-}
-
-
 struct CalibrationFeatures {
  frameId @0 :UInt32;

@@ -760,8 +689,6 @@ struct EncodeIndex {
  segmentId @4 :UInt32;
  # index into camera file in segment in encode order
  segmentIdEncode @5 :UInt32;
-  timestampSof @6 :UInt64;
-  timestampEof @7 :UInt64;

  enum Type {
    bigBoxLossless @0;   # rcamera.mkv
@@ -874,8 +801,8 @@ struct PathPlan {
  laneChangeState @18 :LaneChangeState;
  laneChangeDirection @19 :LaneChangeDirection;

-  # dp
-  dpALCAllowed @20 :Bool;
+  # dragonpilot
+  alcAllowed @20 :Bool;

  enum Desire {
    none @0;
@@ -917,7 +844,6 @@ struct LiveLocationKalman {
  # These angles are all eulers and roll, pitch, yaw
  # orientationECEF transforms to rot matrix: ecef_from_device
  orientationECEF @6 : Measurement;
-  calibratedOrientationECEF @20 : Measurement;
  orientationNED @7 : Measurement;
  angularVelocityDevice @8 : Measurement;

@@ -934,11 +860,6 @@ struct LiveLocationKalman {
  gpsTimeOfWeek @14 :Float64;
  status @15 :Status;
  unixTimestampMillis @16 :Int64;
-  inputsOK @17 :Bool = true;
-  posenetOK @18 :Bool = true;
-  gpsOK @19 :Bool = true;
-  sensorsOK @21 :Bool = true;
-  deviceStable @22 :Bool = true;

  enum Status {
    uninitialized @0;
@@ -1954,9 +1875,6 @@ struct OrbKeyFrame {

 struct DriverState {
  frameId @0 :UInt32;
-  modelExecutionTime @14 :Float32;
-  rawPred @15 :Data;
-
  descriptorDEPRECATED @1 :List(Float32);
  stdDEPRECATED @2 :Float32;
  faceOrientation @3 :List(Float32);
@@ -1969,7 +1887,6 @@ struct DriverState {
  irPwrDEPRECATED @10 :Float32;
  faceOrientationStd @11 :List(Float32);
  facePositionStd @12 :List(Float32);
-  sgProb @13 :Float32;
 }

 struct DMonitoringState {
@@ -1979,6 +1896,7 @@ struct DMonitoringState {
  isDistracted @2 :Bool;
  awarenessStatus @3 :Float32;
  isRHD @4 :Bool;
+  rhdChecked @5 :Bool;
  posePitchOffset @6 :Float32;
  posePitchValidCount @7 :UInt32;
  poseYawOffset @8 :Float32;
@@ -1989,15 +1907,12 @@ struct DMonitoringState {
  isLowStd @13 :Bool;
  hiStdCount @14 :UInt32;
  isPreview @15 :Bool;
-
-  rhdCheckedDEPRECATED @5 :Bool;
 }

 struct Boot {
  wallTimeNanos @0 :UInt64;
  lastKmsg @1 :Data;
  lastPmsg @2 :Data;
-  launchLog @3 :Text;
 }

 struct LiveParametersData {
@@ -2133,92 +2048,9 @@ struct Event {
    thumbnail @66: Thumbnail;
    carEvents @68: List(Car.CarEvent);
    carParams @69: Car.CarParams;
-    frontFrame @70: FrameData; # driver facing camera
+    frontFrame @70: FrameData;
    dMonitoringState @71: DMonitoringState;
    liveLocationKalman @72 :LiveLocationKalman;
    sentinel @73 :Sentinel;
-    wideFrame @74: FrameData;
-    modelV2 @75 :ModelDataV2;
-    frontEncodeIdx @76 :EncodeIndex; # driver facing camera
-    wideEncodeIdx @77 :EncodeIndex;
-    dragonConf @78 :DragonConf;
  }
 }
-
-struct DragonConf {
-  dpThermalStarted @0 :Bool;
-  dpThermalOverheat @1 :Bool;
-  dpVw @2 :Bool;
-  dpAtl @3 :Bool;
-  dpAppWaze @4 :Bool;
-  dpAppWazeManual @5 :Int8;
-  dpAppHr @6 :Bool;
-  dpAppHrManual @7 :Int8;
-  dpDashcam @8 :Bool;
-  dpDashcamHoursStored @9 :UInt8;
-  dpAutoShutdown @10 :Bool;
-  dpAthenad @11 :Bool;
-  dpUploader @12 :Bool;
-  dpLatCtrl @13 :Bool;
-  dpSteeringLimitAlert @14 :Bool;
-  dpSteeringOnSignal @15 :Bool;
-  dpSignalOffDelay @16 :UInt8;
-  dpAssistedLcMinMph @17 :Float32;
-  dpAutoLc @18 :Bool;
-  dpAutoLcCont @19 :Bool;
-  dpAutoLcMinMph @20 :Float32;
-  dpAutoLcDelay @21 :Float32;
-  dpSlowOnCurve @22 :Bool;
-  dpAllowGas @23 :Bool;
-  dpMaxCtrlSpeed @24 :Float32;
-  dpLeadCarAlert @25 :Bool;
-  dpDynamicFollow @26 :UInt8;
-  dpAccelProfile @27 :UInt8;
-  dpDriverMonitor @28 :Bool;
-  dpSteeringMonitor @29 :Bool;
-  dpSteeringMonitorTimer @30 :UInt8;
-  dpGearCheck @31 :Bool;
-  dpDrivingUi @32 :Bool;
-  dpUiScreenOffReversing @33 :Bool;
-  dpUiScreenOffDriving @34 :Bool;
-  dpUiSpeed @35 :Bool;
-  dpUiEvent @36 :Bool;
-  dpUiMaxSpeed @37 :Bool;
-  dpUiFace @38 :Bool;
-  dpUiLane @39 :Bool;
-  dpUiPath @40 :Bool;
-  dpUiLead @41 :Bool;
-  dpUiDev @42 :Bool;
-  dpUiDevMini @43 :Bool;
-  dpUiBlinker @44 :Bool;
-  dpUiBrightness @45 :UInt8;
-  dpUiVolumeBoost @46 :Int8;
-  dpAppAutoUpdate @47 :Bool;
-  dpAppExtGps @48 :Bool;
-  dpAppTomtom @49 :Bool;
-  dpAppTomtomAuto @50 :Bool;
-  dpAppTomtomManual @51 :Int8;
-  dpAppAutonavi @52 :Bool;
-  dpAppAutonaviAuto @53 :Bool;
-  dpAppAutonaviManual @54 :Int8;
-  dpAppAegis @55 :Bool;
-  dpAppAegisAuto @56 :Bool;
-  dpAppAegisManual @57 :Int8;
-  dpAppMixplorer @58 :Bool;
-  dpAppMixplorerManual @59 :Int8;
-  dpCarDetected @60 :Text;
-  dpToyotaLdw @61 :Bool;
-  dpToyotaSng @62 :Bool;
-  dpToyotaLowestCruiseOverride @63 :Bool;
-  dpToyotaLowestCruiseOverrideVego @64 :Bool;
-  dpToyotaLowestCruiseOverrideAt @65 :Float32;
-  dpToyotaLowestCruiseOverrideSpeed @66 :Float32;
-  dpIpAddr @67 :Text;
-  dpCameraOffset @68 :Int8;
-  dpLocale @69 :Text;
-  dpChargingCtrl @70 :Bool;
-  dpChargingAt @71 :UInt8;
-  dpDischargingAt @72 :UInt8;
-  dpIsUpdating @73 :Bool;
-  dpTimebombAssist @74 :Bool;
-}
@@ -0,0 +1,53 @@
+using Cxx = import "./include/c++.capnp";
+$Cxx.namespace("cereal");
+
+using Java = import "./include/java.capnp";
+$Java.package("ai.comma.openpilot.cereal");
+$Java.outerClassname("Map");
+
+@0xa086df597ef5d7a0;
+
+# Geometry
+struct Point {
+  x @0: Float64;
+  y @1: Float64;
+  z @2: Float64;
+}
+
+struct PolyLine {
+  points @0: List(Point);
+}
+
+# Map features
+struct Lane {
+  id @0 :Text;
+
+  leftBoundary @1 :LaneBoundary;
+  rightBoundary @2 :LaneBoundary;
+
+  leftAdjacentId @3 :Text;
+  rightAdjacentId @4 :Text;
+
+  inboundIds @5 :List(Text);
+  outboundIds @6 :List(Text);
+
+  struct LaneBoundary {
+    polyLine @0 :PolyLine;
+    startHeading @1 :Float32; # WRT north
+  }
+}
+
+# Map tiles
+struct TileSummary {
+  version @0 :Text;
+  updatedAt @1 :UInt64; # Millis since epoch
+
+  level @2 :UInt8;
+  x @3 :UInt16;
+  y @4 :UInt16;
+}
+
+struct MapTile {
+  summary @0 :TileSummary;
+  lanes @1 :List(Lane);
+}
@@ -3,14 +3,12 @@ from .messaging_pyx import Context, Poller, SubSocket, PubSocket  # pylint: disa
 from .messaging_pyx import MultiplePublishersError, MessagingError  # pylint: disable=no-name-in-module, import-error
 import capnp

-from typing import Optional, List, Union
+assert MultiplePublishersError
+assert MessagingError

 from cereal import log
 from cereal.services import service_list

-assert MultiplePublishersError
-assert MessagingError
-
 # sec_since_boot is faster, but allow to run standalone too
 try:
  from common.realtime import sec_since_boot
@@ -21,7 +19,7 @@ except ImportError:

 context = Context()

-def new_message(service: Optional[str] = None, size: Optional[int] = None) -> capnp.lib.capnp._DynamicStructBuilder:
+def new_message(service=None, size=None):
  dat = log.Event.new_message()
  dat.logMonoTime = int(sec_since_boot() * 1e9)
  dat.valid = True
@@ -32,15 +30,15 @@ def new_message(service: Optional[str] = None, size: Optional[int] = None) -> ca
      dat.init(service, size)
  return dat

-def pub_sock(endpoint: str) -> PubSocket:
+def pub_sock(endpoint):
  sock = PubSocket()
  sock.connect(context, endpoint)
  return sock

-def sub_sock(endpoint: str, poller: Optional[Poller] = None, addr: str = "127.0.0.1",
-             conflate: bool = False, timeout: Optional[int] = None) -> SubSocket:
+def sub_sock(endpoint, poller=None, addr="127.0.0.1", conflate=False, timeout=None):
  sock = SubSocket()
-  sock.connect(context, endpoint, addr.encode('utf8'), conflate)
+  addr = addr.encode('utf8')
+  sock.connect(context, endpoint, addr, conflate)

  if timeout is not None:
    sock.setTimeout(timeout)
@@ -50,9 +48,9 @@ def sub_sock(endpoint: str, poller: Optional[Poller] = None, addr: str = "127.0.
  return sock


-def drain_sock_raw(sock: SubSocket, wait_for_one: bool = False) -> List[bytes]:
+def drain_sock_raw(sock, wait_for_one=False):
  """Receive all message currently available on the queue"""
-  ret: List[bytes] = []
+  ret = []
  while 1:
    if wait_for_one and len(ret) == 0:
      dat = sock.receive()
@@ -66,16 +64,16 @@ def drain_sock_raw(sock: SubSocket, wait_for_one: bool = False) -> List[bytes]:

  return ret

-def drain_sock(sock: SubSocket, wait_for_one: bool = False) -> List[capnp.lib.capnp._DynamicStructReader]:
+def drain_sock(sock, wait_for_one=False):
  """Receive all message currently available on the queue"""
-  ret: List[capnp.lib.capnp._DynamicStructReader] = []
+  ret = []
  while 1:
    if wait_for_one and len(ret) == 0:
      dat = sock.receive()
    else:
      dat = sock.receive(non_blocking=True)

-    if dat is None:  # Timeout hit
+    if dat is None: # Timeout hit
      break

    dat = log.Event.from_bytes(dat)
@@ -85,7 +83,7 @@ def drain_sock(sock: SubSocket, wait_for_one: bool = False) -> List[capnp.lib.ca


 # TODO: print when we drop packets?
-def recv_sock(sock: SubSocket, wait: bool = False) -> Union[None, capnp.lib.capnp._DynamicStructReader]:
+def recv_sock(sock, wait=False):
  """Same as drain sock, but only returns latest message. Consider using conflate instead."""
  dat = None

@@ -95,7 +93,7 @@ def recv_sock(sock: SubSocket, wait: bool = False) -> Union[None, capnp.lib.capn
    else:
      rcv = sock.receive(non_blocking=True)

-    if rcv is None:  # Timeout hit
+    if rcv is None: # Timeout hit
      break

    dat = rcv
@@ -105,42 +103,45 @@ def recv_sock(sock: SubSocket, wait: bool = False) -> Union[None, capnp.lib.capn

  return dat

-def recv_one(sock: SubSocket) -> Union[None, capnp.lib.capnp._DynamicStructReader]:
+def recv_one(sock):
  dat = sock.receive()
  if dat is not None:
    dat = log.Event.from_bytes(dat)
  return dat

-def recv_one_or_none(sock: SubSocket) -> Union[None, capnp.lib.capnp._DynamicStructReader]:
+def recv_one_or_none(sock):
  dat = sock.receive(non_blocking=True)
  if dat is not None:
    dat = log.Event.from_bytes(dat)
  return dat

-def recv_one_retry(sock: SubSocket) -> capnp.lib.capnp._DynamicStructReader:
+def recv_one_retry(sock):
  """Keep receiving until we get a message"""
  while True:
    dat = sock.receive()
    if dat is not None:
      return log.Event.from_bytes(dat)

+# TODO: This does not belong in messaging
+def get_one_can(logcan):
+  while True:
+    can = recv_one_retry(logcan)
+    if len(can.can) > 0:
+      return can
+
 class SubMaster():
-  def __init__(self, services: List[str], poll: Optional[List[str]] = None,
-               ignore_alive: Optional[List[str]] = None, addr:str ="127.0.0.1"):
+  def __init__(self, services, ignore_alive=None, addr="127.0.0.1"):
+    self.poller = Poller()
    self.frame = -1
-    self.updated = {s: False for s in services}
-    self.rcv_time = {s: 0. for s in services}
-    self.rcv_frame = {s: 0 for s in services}
-    self.alive = {s: False for s in services}
+    self.updated = {s : False for s in services}
+    self.rcv_time = {s : 0. for s in services}
+    self.rcv_frame = {s : 0 for s in services}
+    self.alive = {s : False for s in services}
    self.sock = {}
    self.freq = {}
    self.data = {}
-    self.valid = {}
    self.logMonoTime = {}
-
-    self.poller = Poller()
-    self.non_polled_services = [s for s in services if poll is not None and
-                                len(poll) and s not in poll]
+    self.valid = {}

    if ignore_alive is not None:
      self.ignore_alive = ignore_alive
@@ -149,33 +150,30 @@ class SubMaster():

    for s in services:
      if addr is not None:
-        p = self.poller if s not in self.non_polled_services else None
-        self.sock[s] = sub_sock(s, poller=p, addr=addr, conflate=True)
+        self.sock[s] = sub_sock(s, poller=self.poller, addr=addr, conflate=True)
      self.freq[s] = service_list[s].frequency

      try:
        data = new_message(s)
-      except capnp.lib.capnp.KjException:  # pylint: disable=c-extension-no-member
-        data = new_message(s, 0) # lists
+      except capnp.lib.capnp.KjException:
+        # lists
+        data = new_message(s, 0)

      self.data[s] = getattr(data, s)
      self.logMonoTime[s] = 0
      self.valid[s] = data.valid

-  def __getitem__(self, s: str) -> capnp.lib.capnp._DynamicStructReader:
+  def __getitem__(self, s):
    return self.data[s]

-  def update(self, timeout: int = 1000) -> None:
+  def update(self, timeout=1000):
    msgs = []
    for sock in self.poller.poll(timeout):
      msgs.append(recv_one_or_none(sock))
-
-    # non-blocking receive for non-polled sockets
-    for s in self.non_polled_services:
-      msgs.append(recv_one_or_none(self.sock[s]))
    self.update_msgs(sec_since_boot(), msgs)

-  def update_msgs(self, cur_time: float, msgs: List[capnp.lib.capnp._DynamicStructReader]) -> None:
+  def update_msgs(self, cur_time, msgs):
+    # TODO: add optional input that specify the service to wait for
    self.frame += 1
    self.updated = dict.fromkeys(self.updated, False)
    for msg in msgs:
@@ -198,28 +196,30 @@ class SubMaster():
      else:
        self.alive[s] = True

-  def all_alive(self, service_list=None) -> bool:
+  def all_alive(self, service_list=None):
    if service_list is None:  # check all
      service_list = self.alive.keys()
    return all(self.alive[s] for s in service_list if s not in self.ignore_alive)

-  def all_valid(self, service_list=None) -> bool:
+  def all_valid(self, service_list=None):
    if service_list is None:  # check all
      service_list = self.valid.keys()
    return all(self.valid[s] for s in service_list)

-  def all_alive_and_valid(self, service_list=None) -> bool:
+  def all_alive_and_valid(self, service_list=None):
    if service_list is None:  # check all
      service_list = self.alive.keys()
    return self.all_alive(service_list=service_list) and self.all_valid(service_list=service_list)

+
 class PubMaster():
-  def __init__(self, services: List[str]):
+  def __init__(self, services):
    self.sock = {}
    for s in services:
      self.sock[s] = pub_sock(s)

-  def send(self, s: str, dat: Union[bytes, capnp.lib.capnp._DynamicStructBuilder]) -> None:
+  def send(self, s, dat):
+    # accept either bytes or capnp builder
    if not isinstance(dat, bytes):
      dat = dat.to_bytes()
    self.sock[s].send(dat)
@@ -0,0 +1,50 @@
+#include <iostream>
+#include <cstddef>
+#include <chrono>
+#include <thread>
+#include <cassert>
+
+#include "messaging.hpp"
+#include "impl_zmq.hpp"
+
+#define MSGS 1e5
+
+int main() {
+  Context * c = Context::create();
+  SubSocket * sub_sock = SubSocket::create(c, "controlsState");
+  PubSocket * pub_sock = PubSocket::create(c, "controlsState");
+
+  char data[8];
+
+  Poller * poller = Poller::create({sub_sock});
+
+  auto start = std::chrono::steady_clock::now();
+
+  for (uint64_t i = 0; i < MSGS; i++){
+    *(uint64_t*)data = i;
+    pub_sock->send(data, 8);
+
+    auto r = poller->poll(100);
+
+    for (auto p : r){
+      Message * m = p->receive();
+      uint64_t ii = *(uint64_t*)m->getData();
+      assert(i == ii);
+      delete m;
+    }
+  }
+
+
+  auto end = std::chrono::steady_clock::now();
+  double elapsed = std::chrono::duration_cast<std::chrono::nanoseconds>(end - start).count() / 1e9;
+  double throughput = ((double) MSGS / (double) elapsed);
+  std::cout << throughput << " msg/s" << std::endl;
+
+  delete poller;
+  delete sub_sock;
+  delete pub_sock;
+  delete c;
+
+
+  return 0;
+}
@@ -0,0 +1,30 @@
+import time
+
+from messaging_pyx import Context, Poller, SubSocket, PubSocket # pylint: disable=no-name-in-module, import-error
+
+MSGS = 1e5
+
+if __name__ == "__main__":
+  c = Context()
+  sub_sock = SubSocket()
+  pub_sock = PubSocket()
+
+  sub_sock.connect(c, "controlsState")
+  pub_sock.connect(c, "controlsState")
+
+
+  poller = Poller()
+  poller.registerSocket(sub_sock)
+
+  t = time.time()
+  for i in range(int(MSGS)):
+    bts = i.to_bytes(4, 'little')
+    pub_sock.send(bts)
+
+    for s in poller.poll(100):
+      dat = s.receive()
+      ii = int.from_bytes(dat, 'little')
+      assert(i == ii)
+
+  dt = time.time() - t
+  print("%.1f msg/s" % (MSGS / dt))
@@ -15,18 +15,6 @@ void sig_handler(int signal) {
  msgq_do_exit = 1;
 }

-static size_t get_size(std::string endpoint){
-  size_t sz = DEFAULT_SEGMENT_SIZE;
-
-#if !defined(QCOM) && !defined(QCOM2)
-  if (endpoint == "frame" || endpoint == "frontFrame" || endpoint == "wideFrame"){
-    sz *= 10;
-  }
-#endif
-
-  return sz;
-}
-

 MSGQContext::MSGQContext() {
 }
@@ -61,12 +49,13 @@ MSGQMessage::~MSGQMessage() {
  this->close();
 }

+
 int MSGQSubSocket::connect(Context *context, std::string endpoint, std::string address, bool conflate){
  assert(context);
  assert(address == "127.0.0.1");

  q = new msgq_queue_t;
-  int r = msgq_new_queue(q, endpoint.c_str(), get_size(endpoint));
+  int r = msgq_new_queue(q, endpoint.c_str(), DEFAULT_SEGMENT_SIZE);
  if (r != 0){
    return r;
  }
@@ -154,11 +143,7 @@ int MSGQPubSocket::connect(Context *context, std::string endpoint){
  assert(context);

  q = new msgq_queue_t;
-  int r = msgq_new_queue(q, endpoint.c_str(), get_size(endpoint));
-  if (r != 0){
-    return r;
-  }
-
+  msgq_new_queue(q, endpoint.c_str(), DEFAULT_SEGMENT_SIZE);
  msgq_init_publisher(q);

  return 0;
@@ -2,15 +2,9 @@
 #include "impl_zmq.hpp"
 #include "impl_msgq.hpp"

-#ifdef __APPLE__
-const bool MUST_USE_ZMQ = true;
-#else
-const bool MUST_USE_ZMQ = false;
-#endif
-
 Context * Context::create(){
  Context * c;
-  if (std::getenv("ZMQ") || MUST_USE_ZMQ){
+  if (std::getenv("ZMQ")){
    c = new ZMQContext();
  } else {
    c = new MSGQContext();
@@ -20,7 +14,7 @@ Context * Context::create(){

 SubSocket * SubSocket::create(){
  SubSocket * s;
-  if (std::getenv("ZMQ") || MUST_USE_ZMQ){
+  if (std::getenv("ZMQ")){
    s = new ZMQSubSocket();
  } else {
    s = new MSGQSubSocket();
@@ -66,7 +60,7 @@ SubSocket * SubSocket::create(Context * context, std::string endpoint, std::stri

 PubSocket * PubSocket::create(){
  PubSocket * s;
-  if (std::getenv("ZMQ") || MUST_USE_ZMQ){
+  if (std::getenv("ZMQ")){
    s = new ZMQPubSocket();
  } else {
    s = new MSGQPubSocket();
@@ -88,7 +82,7 @@ PubSocket * PubSocket::create(Context * context, std::string endpoint){

 Poller * Poller::create(){
  Poller * p;
-  if (std::getenv("ZMQ") || MUST_USE_ZMQ){
+  if (std::getenv("ZMQ")){
    p = new ZMQPoller();
  } else {
    p = new MSGQPoller();
@@ -1,14 +1,7 @@
 #pragma once
 #include <cstddef>
-#include <map>
-#include <string>
 #include <vector>
-#include <capnp/serialize.h>
-#include "../gen/cpp/log.capnp.h"
-
-#ifdef __APPLE__
-#define CLOCK_BOOTTIME CLOCK_MONOTONIC
-#endif
+#include <string>

 #define MSG_MULTIPLE_PUBLISHERS 100

@@ -61,61 +54,3 @@ public:
  static Poller * create(std::vector<SubSocket*> sockets);
  virtual ~Poller(){};
 };
-
-class SubMaster {
-public:
-  SubMaster(const std::initializer_list<const char *> &service_list,
-            const char *address = nullptr, const std::initializer_list<const char *> &ignore_alive = {});
-  int update(int timeout = 1000);
-  inline bool allAlive(const std::initializer_list<const char *> &service_list = {}) { return all_(service_list, false, true); }
-  inline bool allValid(const std::initializer_list<const char *> &service_list = {}) { return all_(service_list, true, false); }
-  inline bool allAliveAndValid(const std::initializer_list<const char *> &service_list = {}) { return all_(service_list, true, true); }
-  void drain();
-  ~SubMaster();
-
-  uint64_t frame = 0;
-  bool updated(const char *name) const;
-  uint64_t rcv_frame(const char *name) const;
-  cereal::Event::Reader &operator[](const char *name);
-
-private:
-  bool all_(const std::initializer_list<const char *> &service_list, bool valid, bool alive);
-  Poller *poller_ = nullptr;
-  struct SubMessage;
-  std::map<SubSocket *, SubMessage *> messages_;
-  std::map<std::string, SubMessage *> services_;
-};
-
-class MessageBuilder : public capnp::MallocMessageBuilder {
-public:
-  MessageBuilder() = default;
-
-  cereal::Event::Builder initEvent(bool valid = true) {
-    cereal::Event::Builder event = initRoot<cereal::Event>();
-    struct timespec t;
-    clock_gettime(CLOCK_BOOTTIME, &t);
-    uint64_t current_time = t.tv_sec * 1000000000ULL + t.tv_nsec;
-    event.setLogMonoTime(current_time);
-    event.setValid(valid);
-    return event;
-  }
-
-  kj::ArrayPtr<capnp::byte> toBytes() {
-    heapArray_ = capnp::messageToFlatArray(*this);
-    return heapArray_.asBytes();
-  }
-
-private:
-  kj::Array<capnp::word> heapArray_;
-};
-
-class PubMaster {
-public:
-  PubMaster(const std::initializer_list<const char *> &service_list);
-  inline int send(const char *name, capnp::byte *data, size_t size) { return sockets_.at(name)->send((char *)data, size); }
-  int send(const char *name, MessageBuilder &msg);
-  ~PubMaster();
-
-private:
-  std::map<std::string, PubSocket *> sockets_;
-};
@@ -140,9 +140,9 @@ cdef class PubSocket:
      else:
        raise MessagingError

-  def send(self, bytes data):
+  def send(self, string data):
    length = len(data)
-    r = self.socket.send(<char*>data, length)
+    r = self.socket.send(<char*>data.c_str(), length)

    if r != length:
      if errno.errno == errno.EADDRINUSE:
@@ -6,7 +6,6 @@ from distutils.core import Extension, setup  # pylint: disable=import-error,no-n
 from Cython.Build import cythonize
 from Cython.Distutils import build_ext

-TICI = os.path.isfile('/TICI')

 def get_ext_filename_without_platform_suffix(filename):
  name, ext = os.path.splitext(filename)
@@ -31,16 +30,16 @@ class BuildExtWithoutPlatformSuffix(build_ext):


 sourcefiles = ['messaging_pyx.pyx']
-extra_compile_args = ["-std=c++1z", "-Wno-nullability-completeness"]
+extra_compile_args = ["-std=c++11"]
 libraries = ['zmq']
 ARCH = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()  # pylint: disable=unexpected-keyword-arg

-if ARCH == "aarch64" and not TICI:
+if ARCH == "aarch64" and os.path.isdir("/system"):
  # android
  extra_compile_args += ["-Wno-deprecated-register"]
  libraries += ['gnustl_shared']

-setup(name='messaging',
+setup(name='CAN parser',
      cmdclass={'build_ext': BuildExtWithoutPlatformSuffix},
      ext_modules=cythonize(
        Extension(
@@ -52,7 +51,7 @@ setup(name='messaging',
          extra_objects=[
            os.path.join(os.path.dirname(os.path.realpath(__file__)), '../', 'libmessaging.a'),
          ]
-        ),
-        nthreads=4,
+        )
      ),
+      nthreads=4,
 )
@@ -21,8 +21,6 @@

 #include <stdio.h>

-#include "services.h"
-
 #include "msgq.hpp"

 void sigusr2_handler(int signal) {
@@ -33,13 +31,7 @@ uint64_t msgq_get_uid(void){
  std::random_device rd("/dev/urandom");
  std::uniform_int_distribution<uint64_t> distribution(0,std::numeric_limits<uint32_t>::max());

-  #ifdef __APPLE__
-    // TODO: this doesn't work
-    uint64_t uid = distribution(rd) << 32 | getpid();
-  #else
-    uint64_t uid = distribution(rd) << 32 | syscall(SYS_gettid);
-  #endif
-
+  uint64_t uid = distribution(rd) << 32 | syscall(SYS_gettid);
  return uid;
 }

@@ -83,20 +75,11 @@ void msgq_wait_for_subscriber(msgq_queue_t *q){
  return;
 }

-bool service_exists(std::string path){
-  for (const auto& it : services) {
-    if (it.name == path) {
-      return true;
-    }
-  }
-  return false;
-}
+

 int msgq_new_queue(msgq_queue_t * q, const char * path, size_t size){
  assert(size < 0xFFFFFFFF); // Buffer must be smaller than 2^32 bytes
-  if (!service_exists(std::string(path))){
-    std::cout << "Warning, " << std::string(path) << " is not in service list." << std::endl;
-  }
+
  std::signal(SIGUSR2, sigusr2_handler);

  const char * prefix = "/dev/shm/";
@@ -105,24 +88,21 @@ int msgq_new_queue(msgq_queue_t * q, const char * path, size_t size){
  strcat(full_path, path);

  auto fd = open(full_path, O_RDWR | O_CREAT, 0777);
-  if (fd < 0) {
-    std::cout << "Warning, could not open: " << full_path << std::endl;
-    delete[] full_path;
-    return -1;
-  }
  delete[] full_path;

-  int rc = ftruncate(fd, size + sizeof(msgq_header_t));
-  if (rc < 0){
-    close(fd);
+  if (fd < 0)
    return -1;
-  }
+
+  int rc = ftruncate(fd, size + sizeof(msgq_header_t));
+  if (rc < 0)
+    return -1;
+
  char * mem = (char*)mmap(NULL, size + sizeof(msgq_header_t), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
  close(fd);

-  if (mem == NULL){
+  if (mem == NULL)
    return -1;
-  }
+
  q->mmap_p = mem;

  msgq_header_t *header = (msgq_header_t *)mem;
@@ -430,6 +410,8 @@ int msgq_msg_recv(msgq_msg_t * msg, msgq_queue_t * q){


 int msgq_poll(msgq_pollitem_t * items, size_t nitems, int timeout){
+  assert(timeout >= 0);
+
  int num = 0;

  // Check if messages ready
@@ -0,0 +1,56 @@
+# MSGQ: A lock free single producer multi consumer message queue
+
+[![Build Status](https://dev.azure.com/commaai/default/_apis/build/status/commaai.msgq?branchName=master)](https://dev.azure.com/commaai/default/_build/latest?definitionId=21&branchName=master)
+
+## What is MSGQ?
+MSGQ is a system to pass messages from a single producer to multiple consumers. All the consumers need to be able to receive all the messages. It is designed to be a high performance replacement for ZMQ-like SUB/PUB patterns. It uses a ring buffer in shared memory to efficiently read and write data. Each read requires a copy. Writing can be done without a copy, as long as the size of the data is known in advance.
+
+## Storage
+The storage for the queue consists of an area of metadata, and the actual buffer. The metadata contains:
+
+1. A counter to the number of readers that are active
+2. A pointer to the head of the queue for writing. From now on referred to as *write pointer*
+3. A cycle counter for the writer. This counter is incremented when the writer wraps around
+4. N pointers, pointing to the current read position for all the readers. From now on referred to as *read pointer*
+5. N counters,  counting the number of cycles for all the readers
+6. N booleans, indicating validity for all the readers. From now on referred to as *validity flag*
+
+The counter and the pointer are both 32 bit values, packed into 64 bit so they can be read and written atomically.
+
+The data buffer is a ring buffer. All messages are prefixed by an 8 byte size field, followed by the data. A size of -1 indicates a wrap-around, and means the next message is stored at the beginning of the buffer.
+
+
+## Writing
+Writing involves the following steps:
+
+1. Check if the area that is to be written overlaps with any of the read pointers, mark those readers as invalid by clearing the validity flag.
+2. Write the message
+3. Increase the write pointer by the size of the message
+
+In case there is not enough space at the end of the buffer, a special empty message with a prefix of -1 is written. The cycle counter is incremented by one. In this case step 1 will check there are no read pointers pointing to the remainder of the buffer. Then another write cycle will start with the actual message.
+
+There always needs to be 8 bytes of empty space at the end of the buffer. By doing this there is always space to write the -1.
+
+## Reset reader
+When the reader is lagging too much behind the read pointer becomes invalid and no longer points to the beginning of a valid message. To reset a reader to the current write pointer, the following steps are performed:
+
+1. Set valid flag
+2. Set read cycle counter to that of the writer
+3. Set read pointer to write pointer
+
+## Reading
+Reading involves the following steps:
+
+1. Read the size field at the current read pointer
+2. Read the validity flag
+3. Copy the data out of the buffer
+4. Increase the read pointer by the size of the message
+5. Check the validity flag again
+
+Before starting the copy, the valid flag is checked. This is to prevent a race condition where the size prefix was invalid, and the read could read outside of the buffer. Make sure that step 1 and 2 are not reordered by your compiler or CPU.
+
+If a writer overwrites the data while it's being copied out, the data will be invalid. Therefore the validity flag is also checked after reading it. The order of step 4 and 5 does not matter.
+
+If at steps 2 or 5 the validity flag is not set, the reader is reset. Any data that was already read is discarded. After the reader is reset, the reading starts from the beginning.
+
+If a message with size -1 is encountered, step 3 and 4 are replaced by increasing the cycle counter and setting the read pointer to the beginning of the buffer. After that another read is performed.
@@ -0,0 +1,395 @@
+#include "catch2/catch.hpp"
+#include "msgq.hpp"
+
+TEST_CASE("ALIGN"){
+  REQUIRE(ALIGN(0) == 0);
+  REQUIRE(ALIGN(1) == 8);
+  REQUIRE(ALIGN(7) == 8);
+  REQUIRE(ALIGN(8) == 8);
+  REQUIRE(ALIGN(99999) == 100000);
+}
+
+TEST_CASE("msgq_msg_init_size"){
+  const size_t msg_size = 30;
+  msgq_msg_t msg;
+
+  msgq_msg_init_size(&msg, msg_size);
+  REQUIRE(msg.size == msg_size);
+
+  msgq_msg_close(&msg);
+}
+
+TEST_CASE("msgq_msg_init_data"){
+  const size_t msg_size = 30;
+  char * data = new char[msg_size];
+
+  for (size_t i = 0; i < msg_size; i++){
+    data[i] = i;
+  }
+
+  msgq_msg_t msg;
+  msgq_msg_init_data(&msg, data, msg_size);
+
+  REQUIRE(msg.size == msg_size);
+  REQUIRE(memcmp(msg.data, data, msg_size) == 0);
+
+  delete[] data;
+  msgq_msg_close(&msg);
+}
+
+
+TEST_CASE("msgq_init_subscriber"){
+  remove("/dev/shm/test_queue");
+  msgq_queue_t q;
+  msgq_new_queue(&q, "test_queue", 1024);
+  REQUIRE(*q.num_readers == 0);
+
+  q.reader_id = 1;
+  *q.read_valids[0] = false;
+  *q.read_pointers[0] = ((uint64_t)1 << 32);
+
+  *q.write_pointer = 255;
+
+  msgq_init_subscriber(&q);
+  REQUIRE(q.read_conflate == false);
+  REQUIRE(*q.read_valids[0] == true);
+  REQUIRE((*q.read_pointers[0] >> 32) == 0);
+  REQUIRE((*q.read_pointers[0]  & 0xFFFFFFFF) == 255);
+}
+
+TEST_CASE("msgq_msg_send first message"){
+  remove("/dev/shm/test_queue");
+  msgq_queue_t q;
+  msgq_new_queue(&q, "test_queue", 1024);
+  msgq_init_publisher(&q);
+
+  REQUIRE(*q.write_pointer == 0);
+
+  size_t msg_size = 128;
+
+  SECTION("Aligned message size"){
+  }
+  SECTION("Unaligned message size"){
+    msg_size--;
+  }
+
+  char * data = new char[msg_size];
+
+  for (size_t i = 0; i < msg_size; i++){
+    data[i] = i;
+  }
+
+  msgq_msg_t msg;
+  msgq_msg_init_data(&msg, data, msg_size);
+
+
+  msgq_msg_send(&msg, &q);
+  REQUIRE(*(int64_t*)q.data == msg_size); // Check size tag
+  REQUIRE(*q.write_pointer == 128 + sizeof(int64_t));
+  REQUIRE(memcmp(q.data + sizeof(int64_t), data, msg_size) == 0);
+
+  delete[] data;
+  msgq_msg_close(&msg);
+}
+
+TEST_CASE("msgq_msg_send test wraparound"){
+  remove("/dev/shm/test_queue");
+  msgq_queue_t q;
+  msgq_new_queue(&q, "test_queue", 1024);
+  msgq_init_publisher(&q);
+
+  REQUIRE((*q.write_pointer & 0xFFFFFFFF) == 0);
+  REQUIRE((*q.write_pointer >> 32) == 0);
+
+  const size_t msg_size = 120;
+  msgq_msg_t msg;
+  msgq_msg_init_size(&msg, msg_size);
+
+  for (int i = 0; i < 8; i++) {
+    msgq_msg_send(&msg, &q);
+  }
+  // Check 8th message was written at the beginning
+  REQUIRE((*q.write_pointer & 0xFFFFFFFF) == msg_size + sizeof(int64_t));
+
+  // Check cycle count
+  REQUIRE((*q.write_pointer >> 32) == 1);
+
+  // Check wraparound tag
+  char * tag_location = q.data;
+  tag_location += 7 * (msg_size + sizeof(int64_t));
+  REQUIRE(*(int64_t*)tag_location == -1);
+
+  msgq_msg_close(&msg);
+}
+
+TEST_CASE("msgq_msg_recv test wraparound"){
+  remove("/dev/shm/test_queue");
+  msgq_queue_t q_pub, q_sub;
+  msgq_new_queue(&q_pub, "test_queue", 1024);
+  msgq_new_queue(&q_sub, "test_queue", 1024);
+
+  msgq_init_publisher(&q_pub);
+  msgq_init_subscriber(&q_sub);
+
+  REQUIRE((*q_pub.write_pointer >> 32) == 0);
+  REQUIRE((*q_sub.read_pointers[0] >> 32) == 0);
+
+  const size_t msg_size = 120;
+  msgq_msg_t msg1;
+  msgq_msg_init_size(&msg1, msg_size);
+
+
+  SECTION("Check cycle counter after reset") {
+    for (int i = 0; i < 8; i++) {
+      msgq_msg_send(&msg1, &q_pub);
+    }
+
+    msgq_msg_t msg2;
+    msgq_msg_recv(&msg2, &q_sub);
+    REQUIRE(msg2.size == 0); // Reader had to reset
+    msgq_msg_close(&msg2);
+
+  }
+  SECTION("Check cycle counter while keeping up with writer") {
+    for (int i = 0; i < 8; i++) {
+      msgq_msg_send(&msg1, &q_pub);
+
+      msgq_msg_t msg2;
+      msgq_msg_recv(&msg2, &q_sub);
+      REQUIRE(msg2.size > 0);
+      msgq_msg_close(&msg2);
+    }
+  }
+
+  REQUIRE((*q_sub.read_pointers[0] >> 32) == 1);
+  msgq_msg_close(&msg1);
+}
+
+TEST_CASE("msgq_msg_send test invalidation"){
+  remove("/dev/shm/test_queue");
+  msgq_queue_t q_pub, q_sub;
+  msgq_new_queue(&q_pub, "test_queue", 1024);
+  msgq_new_queue(&q_sub, "test_queue", 1024);
+
+  msgq_init_publisher(&q_pub);
+  msgq_init_subscriber(&q_sub);
+  *q_sub.write_pointer = (uint64_t)1 << 32;
+
+  REQUIRE(*q_sub.read_valids[0] == true);
+
+  SECTION("read pointer in tag"){
+    *q_sub.read_pointers[0] = 0;
+  }
+  SECTION("read pointer in data section"){
+    *q_sub.read_pointers[0] = 64;
+  }
+  SECTION("read pointer in wraparound section"){
+    *q_pub.write_pointer = ((uint64_t)1 << 32) | 1000; // Writer is one cycle ahead
+    *q_sub.read_pointers[0] = 1020;
+  }
+
+  msgq_msg_t msg;
+  msgq_msg_init_size(&msg, 128);
+  msgq_msg_send(&msg, &q_pub);
+
+  REQUIRE(*q_sub.read_valids[0] == false);
+
+  msgq_msg_close(&msg);
+}
+
+TEST_CASE("msgq_init_subscriber init 2 subscribers"){
+  remove("/dev/shm/test_queue");
+  msgq_queue_t q1, q2;
+  msgq_new_queue(&q1, "test_queue", 1024);
+  msgq_new_queue(&q2, "test_queue", 1024);
+
+  *q1.num_readers = 0;
+
+  REQUIRE(*q1.num_readers == 0);
+  REQUIRE(*q2.num_readers == 0);
+
+  msgq_init_subscriber(&q1);
+  REQUIRE(*q1.num_readers == 1);
+  REQUIRE(*q2.num_readers == 1);
+  REQUIRE(q1.reader_id == 0);
+
+  msgq_init_subscriber(&q2);
+  REQUIRE(*q1.num_readers == 2);
+  REQUIRE(*q2.num_readers == 2);
+  REQUIRE(q2.reader_id == 1);
+}
+
+
+TEST_CASE("Write 1 msg, read 1 msg", "[integration]"){
+  remove("/dev/shm/test_queue");
+  const size_t msg_size = 128;
+  msgq_queue_t writer, reader;
+
+  msgq_new_queue(&writer, "test_queue", 1024);
+  msgq_new_queue(&reader, "test_queue", 1024);
+
+  msgq_init_publisher(&writer);
+  msgq_init_subscriber(&reader);
+
+  // Build 128 byte message
+  msgq_msg_t outgoing_msg;
+  msgq_msg_init_size(&outgoing_msg, msg_size);
+
+  for (size_t i = 0; i < msg_size; i++){
+    outgoing_msg.data[i] = i;
+  }
+
+  REQUIRE(msgq_msg_send(&outgoing_msg, &writer) == msg_size);
+
+  msgq_msg_t incoming_msg1;
+  REQUIRE(msgq_msg_recv(&incoming_msg1, &reader) == msg_size);
+  REQUIRE(memcmp(incoming_msg1.data, outgoing_msg.data, msg_size) == 0);
+
+  // Verify that there are no more messages
+  msgq_msg_t incoming_msg2;
+  REQUIRE(msgq_msg_recv(&incoming_msg2, &reader) == 0);
+
+  msgq_msg_close(&outgoing_msg);
+  msgq_msg_close(&incoming_msg1);
+  msgq_msg_close(&incoming_msg2);
+}
+
+TEST_CASE("Write 2 msg, read 2 msg - conflate = false", "[integration]"){
+  remove("/dev/shm/test_queue");
+  const size_t msg_size = 128;
+  msgq_queue_t writer, reader;
+
+  msgq_new_queue(&writer, "test_queue", 1024);
+  msgq_new_queue(&reader, "test_queue", 1024);
+
+  msgq_init_publisher(&writer);
+  msgq_init_subscriber(&reader);
+
+  // Build 128 byte message
+  msgq_msg_t outgoing_msg;
+  msgq_msg_init_size(&outgoing_msg, msg_size);
+
+  for (size_t i = 0; i < msg_size; i++){
+    outgoing_msg.data[i] = i;
+  }
+
+  REQUIRE(msgq_msg_send(&outgoing_msg, &writer) == msg_size);
+  REQUIRE(msgq_msg_send(&outgoing_msg, &writer) == msg_size);
+
+  msgq_msg_t incoming_msg1;
+  REQUIRE(msgq_msg_recv(&incoming_msg1, &reader) == msg_size);
+  REQUIRE(memcmp(incoming_msg1.data, outgoing_msg.data, msg_size) == 0);
+
+  msgq_msg_t incoming_msg2;
+  REQUIRE(msgq_msg_recv(&incoming_msg2, &reader) == msg_size);
+  REQUIRE(memcmp(incoming_msg2.data, outgoing_msg.data, msg_size) == 0);
+
+  msgq_msg_close(&outgoing_msg);
+  msgq_msg_close(&incoming_msg1);
+  msgq_msg_close(&incoming_msg2);
+}
+
+TEST_CASE("Write 2 msg, read 2 msg - conflate = true", "[integration]"){
+  remove("/dev/shm/test_queue");
+  const size_t msg_size = 128;
+  msgq_queue_t writer, reader;
+
+  msgq_new_queue(&writer, "test_queue", 1024);
+  msgq_new_queue(&reader, "test_queue", 1024);
+
+  msgq_init_publisher(&writer);
+  msgq_init_subscriber(&reader);
+  reader.read_conflate = true;
+
+  // Build 128 byte message
+  msgq_msg_t outgoing_msg;
+  msgq_msg_init_size(&outgoing_msg, msg_size);
+
+  for (size_t i = 0; i < msg_size; i++){
+    outgoing_msg.data[i] = i;
+  }
+
+  REQUIRE(msgq_msg_send(&outgoing_msg, &writer) == msg_size);
+  REQUIRE(msgq_msg_send(&outgoing_msg, &writer) == msg_size);
+
+  msgq_msg_t incoming_msg1;
+  REQUIRE(msgq_msg_recv(&incoming_msg1, &reader) == msg_size);
+  REQUIRE(memcmp(incoming_msg1.data, outgoing_msg.data, msg_size) == 0);
+
+  // Verify that there are no more messages
+  msgq_msg_t incoming_msg2;
+  REQUIRE(msgq_msg_recv(&incoming_msg2, &reader) == 0);
+
+  msgq_msg_close(&outgoing_msg);
+  msgq_msg_close(&incoming_msg1);
+  msgq_msg_close(&incoming_msg2);
+}
+
+TEST_CASE("1 publisher, 1 slow subscriber", "[integration]"){
+  remove("/dev/shm/test_queue");
+  msgq_queue_t writer, reader;
+
+  msgq_new_queue(&writer, "test_queue", 1024);
+  msgq_new_queue(&reader, "test_queue", 1024);
+
+  msgq_init_publisher(&writer);
+  msgq_init_subscriber(&reader);
+
+  int n_received = 0;
+  int n_skipped = 0;
+
+  for (uint64_t i = 0; i < 1e5; i++) {
+    msgq_msg_t outgoing_msg;
+    msgq_msg_init_data(&outgoing_msg, (char*)&i, sizeof(uint64_t));
+    msgq_msg_send(&outgoing_msg, &writer);
+    msgq_msg_close(&outgoing_msg);
+
+    if (i % 10 == 0){
+      msgq_msg_t msg1;
+      msgq_msg_recv(&msg1, &reader);
+
+      if (msg1.size == 0){
+        n_skipped++;
+      } else {
+        n_received++;
+      }
+      msgq_msg_close(&msg1);
+    }
+  }
+
+  // TODO: verify these numbers by hand
+  REQUIRE(n_received == 8572);
+  REQUIRE(n_skipped == 1428);
+}
+
+TEST_CASE("1 publisher, 2 subscribers", "[integration]"){
+  remove("/dev/shm/test_queue");
+  msgq_queue_t writer, reader1, reader2;
+
+  msgq_new_queue(&writer, "test_queue", 1024);
+  msgq_new_queue(&reader1, "test_queue", 1024);
+  msgq_new_queue(&reader2, "test_queue", 1024);
+
+  msgq_init_publisher(&writer);
+  msgq_init_subscriber(&reader1);
+  msgq_init_subscriber(&reader2);
+
+  for (uint64_t i = 0; i < 1024 * 3; i++) {
+    msgq_msg_t outgoing_msg;
+    msgq_msg_init_data(&outgoing_msg, (char*)&i, sizeof(uint64_t));
+    msgq_msg_send(&outgoing_msg, &writer);
+
+    msgq_msg_t msg1, msg2;
+    msgq_msg_recv(&msg1, &reader1);
+    msgq_msg_recv(&msg2, &reader2);
+
+    REQUIRE(msg1.size == sizeof(uint64_t));
+    REQUIRE(msg2.size == sizeof(uint64_t));
+    REQUIRE(*(uint64_t*)msg1.data == i);
+    REQUIRE(*(uint64_t*)msg2.data == i);
+
+    msgq_msg_close(&outgoing_msg);
+    msgq_msg_close(&msg1);
+    msgq_msg_close(&msg2);
+  }
+}
@@ -1,170 +0,0 @@
-#include <assert.h>
-#include <time.h>
-#include "messaging.hpp"
-#include "services.h"
-
-static inline uint64_t nanos_since_boot() {
-  struct timespec t;
-  clock_gettime(CLOCK_BOOTTIME, &t);
-  return t.tv_sec * 1000000000ULL + t.tv_nsec;
-}
-
-static const service *get_service(const char *name) {
-  for (const auto &it : services) {
-    if (strcmp(it.name, name) == 0) return &it;
-  }
-  return nullptr;
-}
-
-static inline bool inList(const std::initializer_list<const char *> &list, const char *value) {
-  for (auto &v : list) {
-    if (strcmp(value, v) == 0) return true;
-  }
-  return false;
-}
-
-class MessageContext {
-public:
-  MessageContext() { ctx_ = Context::create(); }
-  ~MessageContext() { delete ctx_; }
-  Context *ctx_;
-};
-MessageContext ctx;
-
-struct SubMaster::SubMessage {
-  std::string name;
-  SubSocket *socket = nullptr;
-  int freq = 0;
-  bool updated = false, alive = false, valid = false, ignore_alive;
-  uint64_t rcv_time = 0, rcv_frame = 0;
-  void *allocated_msg_reader = nullptr;
-  capnp::FlatArrayMessageReader *msg_reader = nullptr;
-  kj::Array<capnp::word> buf;
-  cereal::Event::Reader event;
-};
-
-SubMaster::SubMaster(const std::initializer_list<const char *> &service_list, const char *address,
-                     const std::initializer_list<const char *> &ignore_alive) {
-  poller_ = Poller::create();
-  for (auto name : service_list) {
-    const service *serv = get_service(name);
-    assert(serv != nullptr);
-    SubSocket *socket = SubSocket::create(ctx.ctx_, name, address ? address : "127.0.0.1", true);
-    assert(socket != 0);
-    poller_->registerSocket(socket);
-    SubMessage *m = new SubMessage{
-      .socket = socket,
-      .freq = serv->frequency,
-      .ignore_alive = inList(ignore_alive, name),
-      .allocated_msg_reader = malloc(sizeof(capnp::FlatArrayMessageReader)),
-      .buf = kj::heapArray<capnp::word>(1024)};
-    messages_[socket] = m;
-    services_[name] = m;
-  }
-}
-
-int SubMaster::update(int timeout) {
-  if (++frame == UINT64_MAX) frame = 1;
-  for (auto &kv : messages_) kv.second->updated = false;
-
-  int updated = 0;
-  auto sockets = poller_->poll(timeout);
-  uint64_t current_time = nanos_since_boot();
-  for (auto s : sockets) {
-    Message *msg = s->receive(true);
-    if (msg == nullptr) continue;
-
-    SubMessage *m = messages_.at(s);
-    const size_t size = (msg->getSize() / sizeof(capnp::word)) + 1;
-    if (m->buf.size() < size) {
-      m->buf = kj::heapArray<capnp::word>(size);
-    }
-    memcpy(m->buf.begin(), msg->getData(), msg->getSize());
-    delete msg;
-
-    if (m->msg_reader) {
-      m->msg_reader->~FlatArrayMessageReader();
-    }
-    m->msg_reader = new (m->allocated_msg_reader) capnp::FlatArrayMessageReader(kj::ArrayPtr<capnp::word>(m->buf.begin(), size));
-    m->event = m->msg_reader->getRoot<cereal::Event>();
-    m->updated = true;
-    m->rcv_time = current_time;
-    m->rcv_frame = frame;
-    m->valid = m->event.getValid();
-
-    ++updated;
-  }
-
-  for (auto &kv : messages_) {
-    SubMessage *m = kv.second;
-    m->alive = (m->freq <= (1e-5) || ((current_time - m->rcv_time) * (1e-9)) < (10.0 / m->freq));
-  }
-  return updated;
-}
-
-bool SubMaster::all_(const std::initializer_list<const char *> &service_list, bool valid, bool alive) {
-  int found = 0;
-  for (auto &kv : messages_) {
-    SubMessage *m = kv.second;
-    if (service_list.size() == 0 || inList(service_list, m->name.c_str())) {
-      found += (!valid || m->valid) && (!alive || (m->alive && !m->ignore_alive));
-    }
-  }
-  return service_list.size() == 0 ? found == messages_.size() : found == service_list.size();
-}
-
-void SubMaster::drain() {
-  while (true) {
-    auto polls = poller_->poll(0);
-    if (polls.size() == 0)
-      break;
-
-    for (auto sock : polls) {
-      Message *msg = sock->receive(true);
-      delete msg;
-    }
-  }
-}
-
-bool SubMaster::updated(const char *name) const {
-  return services_.at(name)->updated;
-}
-
-uint64_t SubMaster::rcv_frame(const char *name) const {
-  return services_.at(name)->rcv_frame;
-}
-
-cereal::Event::Reader &SubMaster::operator[](const char *name) {
-  return services_.at(name)->event;
-};
-
-SubMaster::~SubMaster() {
-  delete poller_;
-  for (auto &kv : messages_) {
-    SubMessage *m = kv.second;
-    if (m->msg_reader) {
-      m->msg_reader->~FlatArrayMessageReader();
-    }
-    free(m->allocated_msg_reader);
-    delete m->socket;
-    delete m;
-  }
-}
-
-PubMaster::PubMaster(const std::initializer_list<const char *> &service_list) {
-  for (auto name : service_list) {
-    assert(get_service(name) != nullptr);
-    PubSocket *socket = PubSocket::create(ctx.ctx_, name);
-    assert(socket);
-    sockets_[name] = socket;
-  }
-}
-
-int PubMaster::send(const char *name, MessageBuilder &msg) {
-  auto bytes = msg.toBytes();
-  return send(name, bytes.begin(), bytes.size());
-}
-
-PubMaster::~PubMaster() {
-  for (auto s : sockets_) delete s.second;
-}
@@ -0,0 +1,14 @@
+from messaging_pyx import Context, SubSocket, PubSocket  # pylint: disable=no-name-in-module, import-error
+
+if __name__ == "__main__":
+  c = Context()
+  pub_sock = PubSocket()
+  pub_sock.connect(c, "controlsState")
+
+  for i in range(int(1e10)):
+    print(i)
+    sub_sock = SubSocket()
+    sub_sock.connect(c, "controlsState")
+
+    pub_sock.send(b'a')
+    print(sub_sock.receive())
@@ -0,0 +1,2 @@
+#define CATCH_CONFIG_MAIN
+#include "catch2/catch.hpp"
@@ -0,0 +1,142 @@
+import unittest
+import time
+import cereal.messaging as messaging
+
+import concurrent.futures
+
+
+def poller():
+  context = messaging.Context()
+
+  p = messaging.Poller()
+
+  sub = messaging.SubSocket()
+  sub.connect(context, 'controlsState')
+  p.registerSocket(sub)
+
+  socks = p.poll(10000)
+  r = [s.receive(non_blocking=True) for s in socks]
+
+  return r
+
+
+class TestPoller(unittest.TestCase):
+  def test_poll_once(self):
+    context = messaging.Context()
+
+    pub = messaging.PubSocket()
+    pub.connect(context, 'controlsState')
+
+    with concurrent.futures.ThreadPoolExecutor() as e:
+      poll = e.submit(poller)
+
+      time.sleep(0.1)  # Slow joiner syndrome
+
+      # Send message
+      pub.send("a")
+
+      # Wait for poll result
+      result = poll.result()
+
+    del pub
+    context.term()
+
+    self.assertEqual(result, [b"a"])
+
+  def test_poll_and_create_many_subscribers(self):
+    context = messaging.Context()
+
+    pub = messaging.PubSocket()
+    pub.connect(context, 'controlsState')
+
+    with concurrent.futures.ThreadPoolExecutor() as e:
+      poll = e.submit(poller)
+
+      time.sleep(0.1)  # Slow joiner syndrome
+      c = messaging.Context()
+      for _ in range(10):
+        messaging.SubSocket().connect(c, 'controlsState')
+
+      time.sleep(0.1)
+
+      # Send message
+      pub.send("a")
+
+      # Wait for poll result
+      result = poll.result()
+
+    del pub
+    context.term()
+
+    self.assertEqual(result, [b"a"])
+
+  def test_multiple_publishers_exception(self):
+    context = messaging.Context()
+
+    with self.assertRaises(messaging.MultiplePublishersError):
+      pub1 = messaging.PubSocket()
+      pub1.connect(context, 'controlsState')
+
+      pub2 = messaging.PubSocket()
+      pub2.connect(context, 'controlsState')
+
+      pub1.send("a")
+
+    del pub1
+    del pub2
+    context.term()
+
+  def test_multiple_messages(self):
+    context = messaging.Context()
+
+    pub = messaging.PubSocket()
+    pub.connect(context, 'controlsState')
+
+    sub = messaging.SubSocket()
+    sub.connect(context, 'controlsState')
+
+    time.sleep(0.1)  # Slow joiner
+
+    for i in range(100):
+      pub.send(str(i))
+
+    msg_seen = False
+    i = 0
+    while True:
+      r = sub.receive(non_blocking=True)
+
+      if r is not None:
+        self.assertEqual(str(i), r.decode('utf8'))
+
+        msg_seen = True
+        i += 1
+
+      if r is None and msg_seen:  # ZMQ sometimes receives nothing on the first receive
+        break
+
+    del pub
+    del sub
+    context.term()
+
+  def test_conflate(self):
+    context = messaging.Context()
+
+    pub = messaging.PubSocket()
+    pub.connect(context, 'controlsState')
+
+    sub = messaging.SubSocket()
+    sub.connect(context, 'controlsState', conflate=True)
+
+    time.sleep(0.1)  # Slow joiner
+    pub.send('a')
+    pub.send('b')
+
+    self.assertEqual(b'b', sub.receive())
+
+    del pub
+    del sub
+    context.term()
+
+
+if __name__ == "__main__":
+  unittest.main()
@@ -30,7 +30,7 @@ androidLog: [8020, true, 0.]
 carState: [8021, true, 100., 10]
 # 8022 is reserved for sshd
 carControl: [8023, true, 100., 10]
-plan: [8024, true, 20., 2]
+plan: [8024, true, 20.]
 liveLocation: [8025, true, 0., 1]
 gpsLocation: [8026, true, 1., 1]
 ethernetData: [8027, true, 0.]
@@ -57,19 +57,19 @@ orbslamCorrection: [8050, true, 0.]
 liveLocationCorrected: [8051, true, 0.]
 orbObservation: [8052, true, 0.]
 applanixLocation: [8053, true, 0.]
-liveLocationKalman: [8054, true, 20., 2]
+liveLocationKalman: [8054, true, 0., 1]
 uiNavigationEvent: [8055, true, 0.]
 orbOdometry: [8057, true, 0.]
 orbFeatures: [8058, false, 0.]
 orbKeyFrame: [8059, true, 0.]
 uiLayoutState: [8060, true, 0.]
-frontEncodeIdx: [8061, true, 5.] # should be 20fps on tici
+frontEncodeIdx: [8061, true, 5.]
 orbFeaturesSummary: [8062, true, 0.]
 driverState: [8063, true, 5., 1]
-liveParameters: [8064, true, 20., 2]
+liveParameters: [8064, true, 10.]
 liveMapData: [8065, true, 0.]
 cameraOdometry: [8066, true, 20., 5]
-pathPlan: [8067, true, 20., 2]
+pathPlan: [8067, true, 20.]
 kalmanOdometry: [8068, true, 0.]
 thumbnail: [8069, true, 0.2, 1]
 carEvents: [8070, true, 1., 1]
@@ -77,11 +77,6 @@ carParams: [8071, true, 0.02, 1]
 frontFrame: [8072, true, 10.]
 dMonitoringState: [8073, true, 5., 1]
 offroadLayout: [8074, false, 0.]
-wideEncodeIdx: [8075, true, 20.]
-wideFrame: [8076, true, 20.]
-modelV2: [8077, true, 20., 20]
-
-dragonConf: [8088, false, 2.]

 testModel: [8040, false, 0.]
 testLiveLocation: [8045, false, 0.]
@@ -118,7 +113,7 @@ testJoystick: [8056, false, 0.]

 # **** stateful data transformers ****

-# modeld -- runs & publishes the model
+# modeld -- runs & publishes the model 
 #   publishes: model, cameraOdometry
 #   subscribes: liveCalibration, pathPlan

@@ -2,7 +2,6 @@
 import os
 import yaml

-
 class Service():
  def __init__(self, port, should_log, frequency, decimation=None):
    self.port = port
@@ -10,7 +9,6 @@ class Service():
    self.frequency = frequency
    self.decimation = decimation

-
 service_list_path = os.path.join(os.path.dirname(__file__), "service_list.yaml")

 service_list = {}
@@ -26,9 +24,10 @@ if __name__ == "__main__":
  print("/* THIS IS AN AUTOGENERATED FILE, PLEASE EDIT service_list.yaml */")
  print("#ifndef __SERVICES_H")
  print("#define __SERVICES_H")
-  print("struct service { char name[0x100]; int port; bool should_log; int frequency; int decimation; };")
+  print("struct service { int port; bool should_log; int frequency; int decimation; char name[0x100]; };")
  print("static struct service services[] = {")
  for k, v in service_list.items():
    print('  { .name = "%s", .port = %d, .should_log = %s, .frequency = %d, .decimation = %d },' % (k, v.port, "true" if v.should_log else "false", v.frequency, -1 if v.decimation is None else v.decimation))
  print("};")
  print("#endif")
+
@@ -0,0 +1,7 @@
+coverage:
+  status:
+    project:
+      default:
+        informational: true
+    patch: off
+
@@ -1,14 +1,6 @@
-Import('env', 'cython_dependencies')
+Import('env')

-# Build cython clock module
-env.Command(['common_pyx.so', 'clock.cpp'],
-            cython_dependencies + ['common_pyx_setup.py', 'clock.pyx'],
-            "cd common && python3 common_pyx_setup.py build_ext --inplace")
-
-# Build cython params module
-env.Command(['params_pyx.so', 'params_pyx.cpp'],
-            cython_dependencies + [
-                'params_pyx_setup.py', 'params_pyx.pyx', 'params_pxd.pxd',
-                '#selfdrive/common/params.cc', '#selfdrive/common/params.h',
-                '#selfdrive/common/util.c', '#selfdrive/common/util.h'],
-            "cd common && python3 params_pyx_setup.py build_ext --inplace")
+# parser
+env.Command(['common_pyx.so'],
+  ['common_pyx_setup.py', 'clock.pyx'],
+  "cd common && python3 common_pyx_setup.py build_ext --inplace")
@@ -0,0 +1,281 @@
+import os
+import binascii
+import itertools
+import re
+import struct
+import subprocess
+import random
+from cereal import log
+
+NetworkType = log.ThermalData.NetworkType
+NetworkStrength = log.ThermalData.NetworkStrength
+
+ANDROID = os.path.isfile('/EON')
+
+def getprop(key):
+  if not ANDROID:
+    return ""
+  return subprocess.check_output(["getprop", key], encoding='utf8').strip()
+
+def get_imei(slot):
+  slot = str(slot)
+  if slot not in ("0", "1"):
+    raise ValueError("SIM slot must be 0 or 1")
+
+  ret = parse_service_call_string(service_call(["iphonesubinfo", "3" ,"i32", str(slot)]))
+  if not ret:
+    # allow non android to be identified differently
+    ret = "%015d" % random.randint(0, 1<<32)
+  return ret
+
+def get_serial():
+  ret = getprop("ro.serialno")
+  if ret == "":
+    ret = "cccccccc"
+  return ret
+
+def get_subscriber_info():
+  ret = parse_service_call_string(service_call(["iphonesubinfo", "7"]))
+  if ret is None or len(ret) < 8:
+    return ""
+  return ret
+
+def reboot(reason=None):
+  if reason is None:
+    reason_args = ["null"]
+  else:
+    reason_args = ["s16", reason]
+
+  subprocess.check_output([
+    "service", "call", "power", "16", # IPowerManager.reboot
+    "i32", "0", # no confirmation,
+    *reason_args,
+    "i32", "1" # wait
+  ])
+
+def service_call(call):
+  if not ANDROID:
+    return None
+
+  ret = subprocess.check_output(["service", "call", *call], encoding='utf8').strip()
+  if 'Parcel' not in ret:
+    return None
+
+  return parse_service_call_bytes(ret)
+
+def parse_service_call_unpack(r, fmt):
+  try:
+    return struct.unpack(fmt, r)[0]
+  except Exception:
+    return None
+
+def parse_service_call_string(r):
+  try:
+    r = r[8:] # Cut off length field
+    r = r.decode('utf_16_be')
+
+    # All pairs of two characters seem to be swapped. Not sure why
+    result = ""
+    for a, b, in itertools.zip_longest(r[::2], r[1::2], fillvalue='\x00'):
+        result += b + a
+
+    result = result.replace('\x00', '')
+
+    return result
+  except Exception:
+    return None
+
+def parse_service_call_bytes(ret):
+  try:
+    r = b""
+    for hex_part in re.findall(r'[ (]([0-9a-f]{8})', ret):
+      r += binascii.unhexlify(hex_part)
+    return r
+  except Exception:
+    return None
+
+def get_network_type():
+  if not ANDROID:
+    return NetworkType.none
+
+  wifi_check = parse_service_call_string(service_call(["connectivity", "2"]))
+  if wifi_check is None:
+    return NetworkType.none
+  elif 'WIFI' in wifi_check:
+    return NetworkType.wifi
+  else:
+    cell_check = parse_service_call_unpack(service_call(['phone', '59']), ">q")
+    # from TelephonyManager.java
+    cell_networks = {
+      0: NetworkType.none,
+      1: NetworkType.cell2G,
+      2: NetworkType.cell2G,
+      3: NetworkType.cell3G,
+      4: NetworkType.cell2G,
+      5: NetworkType.cell3G,
+      6: NetworkType.cell3G,
+      7: NetworkType.cell3G,
+      8: NetworkType.cell3G,
+      9: NetworkType.cell3G,
+      10: NetworkType.cell3G,
+      11: NetworkType.cell2G,
+      12: NetworkType.cell3G,
+      13: NetworkType.cell4G,
+      14: NetworkType.cell4G,
+      15: NetworkType.cell3G,
+      16: NetworkType.cell2G,
+      17: NetworkType.cell3G,
+      18: NetworkType.cell4G,
+      19: NetworkType.cell4G
+    }
+    return cell_networks.get(cell_check, NetworkType.none)
+
+def get_network_strength(network_type):
+  network_strength = NetworkStrength.unknown
+  # from SignalStrength.java
+  def get_lte_level(rsrp, rssnr):
+    INT_MAX = 2147483647
+    if rsrp == INT_MAX:
+      lvl_rsrp = NetworkStrength.unknown
+    elif rsrp >= -95:
+      lvl_rsrp = NetworkStrength.great
+    elif rsrp >= -105:
+      lvl_rsrp = NetworkStrength.good
+    elif rsrp >= -115:
+      lvl_rsrp = NetworkStrength.moderate
+    else:
+      lvl_rsrp = NetworkStrength.poor
+    if rssnr == INT_MAX:
+      lvl_rssnr = NetworkStrength.unknown
+    elif rssnr >= 45:
+      lvl_rssnr = NetworkStrength.great
+    elif rssnr >= 10:
+      lvl_rssnr = NetworkStrength.good
+    elif rssnr >= -30:
+      lvl_rssnr = NetworkStrength.moderate
+    else:
+      lvl_rssnr = NetworkStrength.poor
+    return max(lvl_rsrp, lvl_rssnr)
+
+  def get_tdscdma_level(tdscmadbm):
+    lvl = NetworkStrength.unknown
+    if tdscmadbm > -25:
+      lvl = NetworkStrength.unknown
+    elif tdscmadbm >= -49:
+      lvl = NetworkStrength.great
+    elif tdscmadbm >= -73:
+      lvl = NetworkStrength.good
+    elif tdscmadbm >= -97:
+      lvl = NetworkStrength.moderate
+    elif tdscmadbm >= -110:
+      lvl = NetworkStrength.poor
+    return lvl
+
+  def get_gsm_level(asu):
+    if asu <= 2 or asu == 99:
+      lvl = NetworkStrength.unknown
+    elif asu >= 12:
+      lvl = NetworkStrength.great
+    elif asu >= 8:
+      lvl = NetworkStrength.good
+    elif asu >= 5:
+      lvl = NetworkStrength.moderate
+    else:
+      lvl = NetworkStrength.poor
+    return lvl
+
+  def get_evdo_level(evdodbm, evdosnr):
+    lvl_evdodbm = NetworkStrength.unknown
+    lvl_evdosnr = NetworkStrength.unknown
+    if evdodbm >= -65:
+      lvl_evdodbm = NetworkStrength.great
+    elif evdodbm >= -75:
+      lvl_evdodbm = NetworkStrength.good
+    elif evdodbm >= -90:
+      lvl_evdodbm = NetworkStrength.moderate
+    elif evdodbm >= -105:
+      lvl_evdodbm = NetworkStrength.poor
+    if evdosnr >= 7:
+      lvl_evdosnr = NetworkStrength.great
+    elif evdosnr >= 5:
+      lvl_evdosnr = NetworkStrength.good
+    elif evdosnr >= 3:
+      lvl_evdosnr = NetworkStrength.moderate
+    elif evdosnr >= 1:
+      lvl_evdosnr = NetworkStrength.poor
+    return max(lvl_evdodbm, lvl_evdosnr)
+
+  def get_cdma_level(cdmadbm, cdmaecio):
+    lvl_cdmadbm = NetworkStrength.unknown
+    lvl_cdmaecio = NetworkStrength.unknown
+    if cdmadbm >= -75:
+      lvl_cdmadbm = NetworkStrength.great
+    elif cdmadbm >= -85:
+      lvl_cdmadbm = NetworkStrength.good
+    elif cdmadbm >= -95:
+      lvl_cdmadbm = NetworkStrength.moderate
+    elif cdmadbm >= -100:
+      lvl_cdmadbm = NetworkStrength.poor
+    if cdmaecio >= -90:
+      lvl_cdmaecio = NetworkStrength.great
+    elif cdmaecio >= -110:
+      lvl_cdmaecio = NetworkStrength.good
+    elif cdmaecio >= -130:
+      lvl_cdmaecio = NetworkStrength.moderate
+    elif cdmaecio >= -150:
+      lvl_cdmaecio = NetworkStrength.poor
+    return max(lvl_cdmadbm, lvl_cdmaecio)
+
+  if network_type == NetworkType.none:
+    return network_strength
+  if network_type == NetworkType.wifi:
+    out = subprocess.check_output('dumpsys connectivity', shell=True).decode('utf-8')
+    network_strength = NetworkStrength.unknown
+    for line in out.split('\n'):
+      signal_str = "SignalStrength: "
+      if signal_str in line:
+        lvl_idx_start = line.find(signal_str) + len(signal_str)
+        lvl_idx_end = line.find(']', lvl_idx_start)
+        lvl = int(line[lvl_idx_start : lvl_idx_end])
+        if lvl >= -50:
+          network_strength = NetworkStrength.great
+        elif lvl >= -60:
+          network_strength = NetworkStrength.good
+        elif lvl >= -70:
+          network_strength = NetworkStrength.moderate
+        else:
+          network_strength = NetworkStrength.poor
+    return network_strength
+  else:
+    # check cell strength
+    out = subprocess.check_output('dumpsys telephony.registry', shell=True).decode('utf-8')
+    for line in out.split('\n'):
+      if "mSignalStrength" in line:
+        arr = line.split(' ')
+        ns = 0
+        if ("gsm" in arr[14]):
+          rsrp = int(arr[9])
+          rssnr = int(arr[11])
+          ns = get_lte_level(rsrp, rssnr)
+          if ns == NetworkStrength.unknown:
+            tdscmadbm = int(arr[13])
+            ns = get_tdscdma_level(tdscmadbm)
+            if ns == NetworkStrength.unknown:
+              asu = int(arr[1])
+              ns = get_gsm_level(asu)
+        else:
+          cdmadbm = int(arr[3])
+          cdmaecio = int(arr[4])
+          evdodbm = int(arr[5])
+          evdosnr = int(arr[7])
+          lvl_cdma = get_cdma_level(cdmadbm, cdmaecio)
+          lvl_edmo = get_evdo_level(evdodbm, evdosnr)
+          if lvl_edmo == NetworkStrength.unknown:
+            ns = lvl_cdma
+          elif lvl_cdma == NetworkStrength.unknown:
+            ns = lvl_edmo
+          else:
+            ns = min(lvl_cdma, lvl_edmo)
+        network_strength = max(network_strength, ns)
+
+    return network_strength
@@ -38,4 +38,5 @@ def api_get(endpoint, method='GET', timeout=None, access_token=None, **params):

  headers['User-Agent'] = "openpilot-" + version

-  return requests.request(method, backend+endpoint, timeout=timeout, headers=headers, params=params)
+  return requests.request(method, backend+endpoint, timeout=timeout, headers = headers, params=params)
+
@@ -13,7 +13,7 @@ def get_installed_apks():
  ret = {}
  for x in dat:
    if x.startswith("package:"):
-      v, k = x.split("package:")[1].split("=")
+      v,k = x.split("package:")[1].split("=")
      ret[k] = v
  return ret

@@ -31,17 +31,9 @@ def start_offroad():
  system("am start -n ai.comma.plus.offroad/.MainActivity")

 def set_package_permissions():
-  try:
-    output = subprocess.check_output(['dumpsys', 'package', 'ai.comma.plus.offroad'], encoding="utf-8")
-    given_permissions = output.split("runtime permissions")[1]
-  except Exception:
-    given_permissions = ""
-
-  wanted_permissions = ["ACCESS_FINE_LOCATION", "READ_PHONE_STATE", "READ_EXTERNAL_STORAGE"]
-  for permission in wanted_permissions:
-    if permission not in given_permissions:
-      pm_grant("ai.comma.plus.offroad", "android.permission."+permission)
-
+  pm_grant("ai.comma.plus.offroad", "android.permission.ACCESS_FINE_LOCATION")
+  pm_grant("ai.comma.plus.offroad", "android.permission.READ_PHONE_STATE")
+  pm_grant("ai.comma.plus.offroad", "android.permission.READ_EXTERNAL_STORAGE")
  appops_set("ai.comma.plus.offroad", "SU", "allow")
  appops_set("ai.comma.plus.offroad", "WIFI_SCAN", "allow")

@@ -1,8 +1,11 @@
 import os
 BASEDIR = os.path.abspath(os.path.join(os.path.dirname(os.path.realpath(__file__)), "../"))

-from common.hardware import PC
-if PC:
-  PERSIST = os.path.join(BASEDIR, "persist")
-else:
+from common.android import ANDROID
+if ANDROID:
  PERSIST = "/persist"
+  PARAMS = "/data/params"
+else:
+  PERSIST = os.path.join(BASEDIR, "persist")
+  PARAMS = os.path.join(BASEDIR, "persist", "params")
+
@@ -1,4 +1,3 @@
-# cython: language_level = 3
 from posix.time cimport clock_gettime, timespec, CLOCK_MONOTONIC_RAW, clockid_t

 IF UNAME_SYSNAME == "Darwin":
@@ -4,9 +4,9 @@ from Cython.Build import cythonize
 from common.cython_hacks import BuildExtWithoutPlatformSuffix

 sourcefiles = ['clock.pyx']
-extra_compile_args = ["-std=c++1z"]
+extra_compile_args = ["-std=c++11"]

-setup(name='common',
+setup(name='Common',
      cmdclass={'build_ext': BuildExtWithoutPlatformSuffix},
      ext_modules=cythonize(
        Extension(
@@ -14,7 +14,7 @@ setup(name='common',
          language="c++",
          sources=sourcefiles,
          extra_compile_args=extra_compile_args,
-        ),
-        nthreads=4,
+        )
      ),
+      nthreads=4,
 )
@@ -0,0 +1,33 @@
+#!/usr/bin/env python3.7
+import subprocess
+from common.params import Params
+params = Params()
+
+def get_last_modified():
+  return Params().get('DragonLastModified', encoding='utf-8')
+
+def is_online():
+  return not subprocess.call(["ping", "-W", "4", "-c", "1", "117.28.245.92"])
+
+def common_controller_update():
+  dragon_lat_ctrl = False if params.get("DragonLatCtrl", encoding='utf8') == "0" else True
+  if dragon_lat_ctrl:
+    dragon_enable_steering_on_signal = True if (params.get("DragonEnableSteeringOnSignal", encoding='utf8') == "1" and params.get("LaneChangeEnabled", encoding='utf8') == "0") else False
+    if dragon_enable_steering_on_signal:
+      try:
+        dragon_blinker_off_timer = float(params.get("DragonBlinkerOffTimer", encoding='utf8')) * 100
+      except (TypeError, ValueError):
+        dragon_blinker_off_timer = 0
+    else:
+      dragon_blinker_off_timer = 0
+  else:
+    dragon_enable_steering_on_signal = False
+    dragon_blinker_off_timer = 0
+  return dragon_lat_ctrl, dragon_enable_steering_on_signal, dragon_blinker_off_timer
+
+def common_controller_ctrl(enabled, dragon_lat_ctrl, dragon_enable_steering_on_signal, blinker_on, steer_req):
+  if enabled:
+    if (dragon_enable_steering_on_signal and blinker_on) or not dragon_lat_ctrl:
+      steer_req = 0 if isinstance(steer_req, int) else False
+
+  return steer_req
@@ -1,80 +0,0 @@
-#!/usr/bin/env python3.7
-import subprocess
-from cereal import car
-from common.params import Params
-from common.realtime import sec_since_boot
-import os
-params = Params()
-PARAM_PATH = "/data/params/d/"
-LAST_MODIFIED = PARAM_PATH + "dp_last_modified"
-
-def is_online():
-  try:
-    return not subprocess.call(["ping", "-W", "4", "-c", "1", "117.28.245.92"])
-  except ProcessLookupError:
-    return False
-
-def common_controller_ctrl(enabled, dragonconf, blinker_on, steer_req, v_ego):
-  if enabled:
-    if (dragonconf.dpSteeringOnSignal and blinker_on) or not dragonconf.dpLatCtrl:
-      steer_req = 0 if isinstance(steer_req, int) else False
-  return steer_req
-
-def common_interface_atl(ret, atl):
-  # dp
-  enable_acc = ret.cruiseState.enabled
-  if atl and ret.cruiseState.available:
-    enable_acc = True
-    if ret.gearShifter in [car.CarState.GearShifter.reverse, car.CarState.GearShifter.park]:
-      enable_acc = False
-    if ret.seatbeltUnlatched or ret.doorOpen:
-      enable_acc = False
-  return enable_acc
-
-def common_interface_get_params_lqr(ret):
-  if params.get('dp_lqr') == b'1':
-    ret.lateralTuning.init('lqr')
-    ret.lateralTuning.lqr.scale = 1500.0
-    ret.lateralTuning.lqr.ki = 0.05
-
-    ret.lateralTuning.lqr.a = [0., 1., -0.22619643, 1.21822268]
-    ret.lateralTuning.lqr.b = [-1.92006585e-04, 3.95603032e-05]
-    ret.lateralTuning.lqr.c = [1., 0.]
-    ret.lateralTuning.lqr.k = [-110.73572306, 451.22718255]
-    ret.lateralTuning.lqr.l = [0.3233671, 0.3185757]
-    ret.lateralTuning.lqr.dcGain = 0.002237852961363602
-  return ret
-
-
-def get_last_modified(delay, old_check, old_modified):
-  new_check = sec_since_boot()
-  if old_check is None or new_check - old_check >= delay:
-    return new_check, os.stat(LAST_MODIFIED).st_mtime
-  else:
-    return old_check, old_modified
-
-def param_get_if_updated(param, type, old_val, old_modified):
-  try:
-    modified = os.stat(PARAM_PATH + param).st_mtime
-  except OSError:
-    return old_val, old_modified
-  if old_modified != modified:
-    new_val = param_get(param, type, old_val)
-    new_modified = modified
-  else:
-    new_val = old_val
-    new_modified = old_modified
-  return new_val, new_modified
-
-def param_get(param_name, type, default):
-  try:
-    val = params.get(param_name, encoding='utf8').rstrip('\x00')
-    if type == 'bool':
-      val = val == '1'
-    elif type == 'int':
-      val = int(val)
-    elif type == 'float':
-      val = float(val)
-  except (TypeError, ValueError):
-    val = default
-  return val
@@ -1,248 +0,0 @@
-#!/usr/bin/env python3.7
-import os
-import json
-import time
-from math import floor
-
-'''
-* type: Bool, Int8, UInt8, UInt16, Float32
-* conf_type: param, struct
-* dependencies needs to use struct and loaded prior so we don't have to read the param multiple times.
-* update_once: True, False (the param will only load up once.)
-'''
-confs = [
-  # thermald data
-  {'name': 'dp_thermal_started', 'default': False, 'type': 'Bool', 'conf_type': ['struct']},
-  {'name': 'dp_thermal_overheat', 'default': False, 'type': 'Bool', 'conf_type': ['struct']},
-
-  # car specific
-  {'name': 'dp_vw', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct'], 'update_once': True},
-
-  {'name': 'dp_atl', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct'], 'update_once': True},
-  # full screen apps
-  {'name': 'dp_app_waze', 'default':  False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_waze_manual', 'default': 0, 'type': 'Int8', 'min': -1, 'max': 1, 'depends': [{'name': 'dp_app_waze', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_hr', 'default':  False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_hr_manual', 'default': 0, 'type': 'Int8', 'min': -1, 'max': 1, 'depends': [{'name': 'dp_app_hr', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  # dashcam related
-  {'name': 'dp_dashcam', 'default': 0, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_dashcam_hours_stored', 'default': 24, 'type': 'UInt8', 'min': 1, 'max': 255, 'depends': [{'name': 'dp_dashcam', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  # auto shutdown
-  {'name': 'dp_auto_shutdown', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_auto_shutdown_in', 'default': 90, 'type': 'UInt16', 'min': 1, 'max': 65535, 'depends': [{'name': 'dp_auto_shutdown', 'vals': [True]}], 'conf_type': ['param']},
-  # service
-  {'name': 'dp_logger', 'default': True, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_athenad', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_uploader', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_upload_on_mobile', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_uploader', 'vals': [True]}], 'conf_type': ['param']},
-  {'name': 'dp_upload_on_hotspot', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_uploader', 'vals': [True]}], 'conf_type': ['param']},
-  {'name': 'dp_updated', 'default': True, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_gpxd', 'default': False, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_hotspot_on_boot', 'default': False, 'type': 'Bool', 'conf_type': ['param']},
-  # lat ctrl
-  {'name': 'dp_lat_ctrl', 'default': True, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_steering_limit_alert', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_lat_ctrl', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_steering_on_signal', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_lat_ctrl', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_signal_off_delay', 'default': 0, 'type': 'UInt8', 'min': 0, 'max': 10, 'conf_type': ['param', 'struct']},
-  # assist/auto lane change
-  {'name': 'dp_assisted_lc_min_mph', 'default': 45, 'type': 'Float32', 'min': 0, 'max': 255., 'conf_type': ['param', 'struct']},
-  {'name': 'dp_auto_lc', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_auto_lc_cont', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_auto_lc', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_auto_lc_min_mph', 'default': 60, 'type': 'Float32', 'min': 0, 'max': 255., 'depends': [{'name': 'dp_auto_lc', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_auto_lc_delay', 'default': 3, 'type': 'Float32', 'min': 0, 'max': 10., 'depends': [{'name': 'dp_auto_lc', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  # long ctrl
-  {'name': 'dp_slow_on_curve', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_allow_gas', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_max_ctrl_speed', 'default': 92., 'type': 'Float32', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_lead_car_alert', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_dynamic_follow', 'default': 0, 'type': 'UInt8', 'min': 0, 'max': 4, 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_dynamic_follow_multiplier', 'default': 1., 'type': 'Float32', 'min': 0.85, 'max': 1.2, 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param']},
-  {'name': 'dp_dynamic_follow_min_tr', 'default': 0.9, 'type': 'Float32', 'min': 0.85, 'max': 1.6, 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param']},
-  {'name': 'dp_dynamic_gas', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param']},
-  {'name': 'dp_accel_profile', 'default': 0, 'type': 'UInt8', 'min': 0, 'max': 3, 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  # safety
-  {'name': 'dp_driver_monitor', 'default': True, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_steering_monitor', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_driver_monitor', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_steering_monitor_timer', 'default': 70, 'type': 'UInt8', 'min': 70, 'max': 360, 'depends': [{'name': 'dp_driver_monitor', 'vals': [False]}, {'name': 'dp_steering_monitor', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_gear_check', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_temp_monitor', 'default': True, 'type': 'Bool', 'conf_type': ['param']},
-  # UIs
-  {'name': 'dp_driving_ui', 'default': True, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_screen_off_reversing', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_screen_off_driving', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_app_waze', 'vals': [False]}, {'name': 'dp_app_hr', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_speed', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_driving_ui', 'vals': [True]}, {'name': 'dp_ui_screen_off_driving', 'vals': [False]}, {'name': 'dp_app_waze', 'vals': [False]}, {'name': 'dp_app_hr', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_event', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_driving_ui', 'vals': [True]}, {'name': 'dp_ui_screen_off_driving', 'vals': [False]}, {'name': 'dp_app_waze', 'vals': [False]}, {'name': 'dp_app_hr', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_max_speed', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_driving_ui', 'vals': [True]}, {'name': 'dp_ui_screen_off_driving', 'vals': [False]}, {'name': 'dp_app_waze', 'vals': [False]}, {'name': 'dp_app_hr', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_face', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_driving_ui', 'vals': [True]}, {'name': 'dp_driver_monitor', 'vals': [True]}, {'name': 'dp_ui_screen_off_driving', 'vals': [False]}, {'name': 'dp_app_waze', 'vals': [False]}, {'name': 'dp_app_hr', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_lane', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_driving_ui', 'vals': [True]}, {'name': 'dp_ui_screen_off_driving', 'vals': [False]}, {'name': 'dp_app_waze', 'vals': [False]}, {'name': 'dp_app_hr', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_path', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_driving_ui', 'vals': [True]}, {'name': 'dp_ui_screen_off_driving', 'vals': [False]}, {'name': 'dp_app_waze', 'vals': [False]}, {'name': 'dp_app_hr', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_lead', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_driving_ui', 'vals': [True]}, {'name': 'dp_ui_screen_off_driving', 'vals': [False]}, {'name': 'dp_app_waze', 'vals': [False]}, {'name': 'dp_app_hr', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_dev', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_dev_mini', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_blinker', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_driving_ui', 'vals': [True]}, {'name': 'dp_ui_screen_off_driving', 'vals': [False]}, {'name': 'dp_app_waze', 'vals': [False]}, {'name': 'dp_app_hr', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_brightness', 'default': 0, 'type': 'UInt8', 'min': 0, 'max': 100, 'conf_type': ['param', 'struct']},
-  {'name': 'dp_ui_volume_boost', 'default': 0, 'type': 'Int8', 'min': -100, 'max': 100, 'conf_type': ['param', 'struct']},
-  # Apps
-  {'name': 'dp_app_auto_update', 'default': True, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_ext_gps', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_tomtom', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_tomtom_auto', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_app_tomtom', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_tomtom_manual', 'default': 0, 'type': 'Int8', 'min': -1, 'max': 1, 'depends': [{'name': 'dp_app_tomtom', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_autonavi', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_autonavi_auto', 'default':  False, 'type': 'Bool', 'depends': [{'name': 'dp_app_autonavi', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_autonavi_manual', 'default': 0, 'type': 'Int8', 'min': -1, 'max': 1, 'depends': [{'name': 'dp_app_autonavi', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_aegis', 'default':  False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_aegis_auto', 'default':  False, 'type': 'Bool', 'depends': [{'name': 'dp_app_aegis', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_aegis_manual', 'default': 0, 'type': 'Int8', 'min': -1, 'max': 1, 'depends': [{'name': 'dp_app_aegis', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_mixplorer', 'default':  False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_app_mixplorer_manual', 'default': 0, 'type': 'Int8', 'min': -1, 'max': 1, 'depends': [{'name': 'dp_app_mixplorer', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  # custom car
-  {'name': 'dp_car_selected', 'default': '', 'type': 'Text', 'conf_type': ['param']},
-  {'name': 'dp_car_list', 'default': '', 'type': 'Text', 'conf_type': ['param']},
-  {'name': 'dp_car_detected', 'default': '', 'type': 'Text', 'conf_type': ['param', 'struct']},
-  # toyota
-  {'name': 'dp_toyota_ldw', 'default': True, 'type': 'Bool', 'depends': [{'name': 'dp_car_detected', 'vals': ['toyota']}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_toyota_sng', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_car_detected', 'vals': ['toyota']}, {'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_toyota_zss', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_car_detected', 'vals': ['toyota']}], 'conf_type': ['param']},
-  {'name': 'dp_toyota_lowest_cruise_override', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_car_detected', 'vals': ['toyota']}, {'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_toyota_lowest_cruise_override_vego', 'default': False, 'type': 'Bool', 'depends': [{'name': 'dp_car_detected', 'vals': ['toyota']}, {'name': 'dp_atl', 'vals': [False]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_toyota_lowest_cruise_override_at', 'default': 44, 'type': 'Float32', 'depends': [{'name': 'dp_car_detected', 'vals': ['toyota']}, {'name': 'dp_toyota_lowest_cruise_override', 'vals': [True]}], 'min': 0, 'max': 255., 'conf_type': ['param', 'struct']},
-  {'name': 'dp_toyota_lowest_cruise_override_speed', 'default': 32, 'type': 'Float32', 'depends': [{'name': 'dp_car_detected', 'vals': ['toyota']}, {'name': 'dp_toyota_lowest_cruise_override_speed', 'vals': [True]}], 'min': 0, 'max': 255., 'conf_type': ['param', 'struct']},
-  # hyundai
-  {'name': 'dp_hkg_smart_mdps', 'default': False, 'type': 'Bool', 'conf_type': ['param']},
-  # honda
-  {'name': 'dp_honda_eps_mod', 'default': False, 'type': 'Bool', 'conf_type': ['param']},
-  #misc
-  {'name': 'dp_ip_addr', 'default': '', 'type': 'Text', 'conf_type': ['struct']},
-  {'name': 'dp_full_speed_fan', 'default': False, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_uno_fan_mode', 'default': False, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_last_modified', 'default': str(floor(time.time())), 'type': 'Text', 'conf_type': ['param']},
-  {'name': 'dp_camera_offset', 'default': 6, 'type': 'Int8', 'min': -255, 'max': 255, 'conf_type': ['param', 'struct']},
-
-  {'name': 'dp_locale', 'default': 'en-US', 'type': 'Text', 'conf_type': ['param', 'struct'], 'update_once': True},
-  {'name': 'dp_disable_relay', 'default': False, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_charging_ctrl', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-  {'name': 'dp_charging_at', 'default': 60, 'type': 'UInt8', 'min': 0, 'max': 100, 'depends': [{'name': 'dp_charging_ctrl', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_discharging_at', 'default': 70, 'type': 'UInt8', 'min': 0, 'max': 100, 'depends': [{'name': 'dp_charging_ctrl', 'vals': [True]}], 'conf_type': ['param', 'struct']},
-  {'name': 'dp_reg', 'default': True, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_is_updating', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-
-  {'name': 'dp_sr_learner', 'default': True, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_lqr', 'default': False, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_reset_live_param_on_start', 'default': False, 'type': 'Bool', 'conf_type': ['param']},
-  {'name': 'dp_timebomb_assist', 'default': False, 'type': 'Bool', 'conf_type': ['param', 'struct']},
-]
-
-def get_definition(name):
-  for conf in confs:
-    if conf['name'] == name:
-      return conf
-  return None
-
-def to_param_val(name, val):
-  conf = get_definition(name)
-  if conf is not None:
-    type = conf['type'].lower()
-    try:
-      if 'bool' in type:
-        val = '1' if val else '0'
-      elif 'int' in type:
-        val = int(val)
-      elif 'float' in type:
-        val = float(val)
-      return str(val)
-    except (ValueError, TypeError):
-      return ''
-  return ''
-
-def to_struct_val(name, val):
-  conf = get_definition(name)
-  if conf is not None:
-    try:
-      type = conf['type'].lower()
-      if 'bool' in type:
-        val = True if val == '1' else False
-      elif 'int' in type:
-        val = int(val)
-      elif 'float' in type:
-        val = float(val)
-      return val
-    except (ValueError, TypeError):
-      return None
-  return None
-
-'''
-function to convert param name into struct name.
-'''
-def get_struct_name(snake_str):
-  components = snake_str.split('_')
-  # We capitalize the first letter of each component except the first one
-  # with the 'title' method and join them together.
-  return components[0] + ''.join(x.title() for x in components[1:])
-
-'''
-function to generate struct for log.capnp
-'''
-def gen_log_struct():
-  count = 0
-  str = "# dp\n"
-  str += "struct DragonConf {\n"
-  for conf in confs:
-    name = get_struct_name(conf['name'])
-    if 'struct' in conf['conf_type']:
-      str += f"  {name} @{count} :{conf['type']};\n"
-      count += 1
-  str += "}"
-  print(str)
-
-'''
-function to append new keys to params.py
-'''
-def init_params_keys(keys, type):
-  for conf in confs:
-    if 'param' in conf['conf_type']:
-      keys[conf['name'].encode('utf-8')] = type
-  return keys
-
-'''
-function to generate support car list
-'''
-def get_support_car_list():
-  attrs = ['FINGERPRINTS', 'FW_VERSIONS']
-  cars = dict()
-  for car_folder in [x[0] for x in os.walk('/data/openpilot/selfdrive/car')]:
-    try:
-      car_name = car_folder.split('/')[-1]
-      if car_name != "mock":
-        names = []
-        for attr in attrs:
-          values = __import__('selfdrive.car.%s.values' % car_name, fromlist=[attr])
-          if hasattr(values, attr):
-            attr_values = getattr(values, attr)
-          else:
-            continue
-
-          if isinstance(attr_values, dict):
-            for f, v in attr_values.items():
-              if f not in names:
-                names.append(f)
-          names.sort()
-        cars[car_name] = names
-    except (ImportError, IOError, ValueError):
-      pass
-  return json.dumps(cars)
-
-'''
-function to init param value.
-should add this into manager.py
-'''
-def init_params_vals(params):
-  for conf in confs:
-    if 'param' in conf['conf_type']:
-      if conf['name'] == 'dp_car_list':
-        params.put(conf['name'], get_support_car_list())
-      elif params.get(conf['name']) is None:
-        params.put(conf['name'], to_param_val(conf['name'], conf['default']))
-
-if __name__ == "__main__":
-  gen_log_struct()
@@ -1,8 +0,0 @@
-#!/usr/bin/env python3.7
-
-# delay of reading last modified
-LAST_MODIFIED_DYNAMIC_FOLLOW = 3.
-LAST_MODIFIED_THERMALD = 10.
-LAST_MODIFIED_SYSTEMD = 1.
-LAST_MODIFIED_LANE_PLANNER = 3.
-LAST_MODIFIED_UPLOADER = 10.
@@ -44,7 +44,7 @@ def compile_code(name, c_code, c_header, directory, cflags="", libraries=None):
  ffibuilder = FFI()
  ffibuilder.set_source(name, c_code, source_extension='.cpp', libraries=libraries)
  ffibuilder.cdef(c_header)
-  os.environ['OPT'] = "-fwrapv -O2 -DNDEBUG -std=c++1z"
+  os.environ['OPT'] = "-fwrapv -O2 -DNDEBUG -std=c++11"
  os.environ['CFLAGS'] = cflags
  ffibuilder.compile(verbose=True, debug=False, tmpdir=directory)

@@ -3,17 +3,13 @@ import shutil
 import tempfile
 from atomicwrites import AtomicWriter

-
 def mkdirs_exists_ok(path):
-  if path.startswith('http://') or path.startswith('https://'):
-    raise ValueError('URL path')
  try:
    os.makedirs(path)
  except OSError:
    if not os.path.isdir(path):
      raise

-
 def rm_not_exists_ok(path):
  try:
    os.remove(path)
@@ -21,14 +17,12 @@ def rm_not_exists_ok(path):
    if os.path.exists(path):
      raise

-
 def rm_tree_or_link(path):
  if os.path.islink(path):
    os.unlink(path)
  elif os.path.isdir(path):
    shutil.rmtree(path)

-
 def get_tmpdir_on_same_filesystem(path):
  normpath = os.path.normpath(path)
  parts = normpath.split("/")
@@ -38,7 +32,6 @@ def get_tmpdir_on_same_filesystem(path):
    return "/{}/runner/tmp".format(parts[1])
  return "/tmp"

-
 class AutoMoveTempdir():
  def __init__(self, target_path, temp_dir=None):
    self._target_path = target_path
@@ -51,16 +44,14 @@ class AutoMoveTempdir():
  def close(self):
    os.rename(self._path, self._target_path)

-  def __enter__(self):
-    return self
+  def __enter__(self): return self

-  def __exit__(self, exc_type, exc_value, traceback):
-    if exc_type is None:
+  def __exit__(self, type, value, traceback):
+    if type is None:
      self.close()
    else:
      shutil.rmtree(self._path)

-
 class NamedTemporaryDir():
  def __init__(self, temp_dir=None):
    self._path = tempfile.mkdtemp(dir=temp_dir)
@@ -72,13 +63,11 @@ class NamedTemporaryDir():
  def close(self):
    shutil.rmtree(self._path)

-  def __enter__(self):
-    return self
+  def __enter__(self): return self

-  def __exit__(self, exc_type, exc_value, traceback):
+  def __exit__(self, type, value, traceback):
    self.close()

-
 def _get_fileobject_func(writer, temp_dir):
  def _get_fileobject():
    file_obj = writer.get_fileobject(dir=temp_dir)
@@ -86,7 +75,6 @@ def _get_fileobject_func(writer, temp_dir):
    return file_obj
  return _get_fileobject

-
 def atomic_write_on_fs_tmp(path, **kwargs):
  """Creates an atomic writer using a temporary file in a temporary directory
     on the same filesystem as path.
@@ -104,7 +92,6 @@ def atomic_write_in_dir(path, **kwargs):
  writer = AtomicWriter(path, **kwargs)
  return writer._open(_get_fileobject_func(writer, os.path.dirname(path)))

-
 def atomic_write_in_dir_neos(path, contents, mode=None):
  """
  Atomically writes contents to path using a temporary file in the same directory
@@ -1,22 +0,0 @@
-GPIO_HUB_RST_N = 30
-GPIO_UBLOX_RST_N = 32
-GPIO_UBLOX_SAFEBOOT_N = 33
-GPIO_UBLOX_PWR_EN = 34
-GPIO_STM_RST_N = 124
-GPIO_STM_BOOT0 = 134
-
-
-def gpio_init(pin, output):
-  try:
-    with open(f"/sys/class/gpio/gpio{pin}/direction", 'wb') as f:
-      f.write(b"out" if output else b"in")
-  except Exception as e:
-    print(f"Failed to set gpio {pin} direction: {e}")
-
-
-def gpio_set(pin, high):
-  try:
-    with open(f"/sys/class/gpio/gpio{pin}/value", 'wb') as f:
-      f.write(b"1" if high else b"0")
-  except Exception as e:
-    print(f"Failed to set gpio {pin} value: {e}")
@@ -1,57 +0,0 @@
-import os
-import random
-from typing import cast
-
-from cereal import log
-from common.hardware_android import Android
-from common.hardware_tici import Tici
-from common.hardware_base import HardwareBase
-
-EON = os.path.isfile('/EON')
-TICI = os.path.isfile('/TICI')
-PC = not (EON or TICI)
-ANDROID = EON
-
-
-NetworkType = log.ThermalData.NetworkType
-NetworkStrength = log.ThermalData.NetworkStrength
-
-
-class Pc(HardwareBase):
-  def get_sound_card_online(self):
-    return True
-
-  def get_imei(self, slot):
-    return "%015d" % random.randint(0, 1 << 32)
-
-  def get_serial(self):
-    return "cccccccc"
-
-  def get_subscriber_info(self):
-    return ""
-
-  def reboot(self, reason=None):
-    print("REBOOT!")
-
-  def get_network_type(self):
-    return NetworkType.wifi
-
-  def get_sim_info(self):
-    return {
-      'sim_id': '',
-      'mcc_mnc': None,
-      'network_type': ["Unknown"],
-      'sim_state': ["ABSENT"],
-      'data_connected': False
-    }
-
-  def get_network_strength(self, network_type):
-    return NetworkStrength.unknown
-
-
-if EON:
-  HARDWARE = cast(HardwareBase, Android())
-elif TICI:
-  HARDWARE = cast(HardwareBase, Tici())
-else:
-  HARDWARE = cast(HardwareBase, Pc())
@@ -1,302 +0,0 @@
-import binascii
-import itertools
-import os
-import re
-import struct
-import subprocess
-
-from cereal import log
-from common.hardware_base import HardwareBase
-
-NetworkType = log.ThermalData.NetworkType
-NetworkStrength = log.ThermalData.NetworkStrength
-
-
-def service_call(call):
-  try:
-    ret = subprocess.check_output(["service", "call", *call], encoding='utf8').strip()
-    if 'Parcel' not in ret:
-      return None
-    return parse_service_call_bytes(ret)
-  except subprocess.CalledProcessError:
-    return None
-
-
-def parse_service_call_unpack(r, fmt):
-  try:
-    return struct.unpack(fmt, r)[0]
-  except Exception:
-    return None
-
-
-def parse_service_call_string(r):
-  try:
-    r = r[8:]  # Cut off length field
-    r = r.decode('utf_16_be')
-
-    # All pairs of two characters seem to be swapped. Not sure why
-    result = ""
-    for a, b, in itertools.zip_longest(r[::2], r[1::2], fillvalue='\x00'):
-        result += b + a
-
-    result = result.replace('\x00', '')
-
-    return result
-  except Exception:
-    return None
-
-
-def parse_service_call_bytes(ret):
-  try:
-    r = b""
-    for hex_part in re.findall(r'[ (]([0-9a-f]{8})', ret):
-      r += binascii.unhexlify(hex_part)
-    return r
-  except Exception:
-    return None
-
-
-def getprop(key):
-  return subprocess.check_output(["getprop", key], encoding='utf8').strip()
-
-
-class Android(HardwareBase):
-  def get_sound_card_online(self):
-    return (os.path.isfile('/proc/asound/card0/state') and
-            open('/proc/asound/card0/state').read().strip() == 'ONLINE')
-
-  def get_imei(self, slot):
-    slot = str(slot)
-    if slot not in ("0", "1"):
-      raise ValueError("SIM slot must be 0 or 1")
-
-    return parse_service_call_string(service_call(["iphonesubinfo", "3", "i32", str(slot)]))
-
-  def get_serial(self):
-    ret = getprop("ro.serialno")
-    if ret == "":
-      ret = "cccccccc"
-    return ret
-
-  def get_subscriber_info(self):
-    ret = parse_service_call_string(service_call(["iphonesubinfo", "7"]))
-    if ret is None or len(ret) < 8:
-      return ""
-    return ret
-
-  def reboot(self, reason=None):
-    # e.g. reason="recovery" to go into recover mode
-    if reason is None:
-      reason_args = ["null"]
-    else:
-      reason_args = ["s16", reason]
-
-    subprocess.check_output([
-      "service", "call", "power", "16",  # IPowerManager.reboot
-      "i32", "0",  # no confirmation,
-      *reason_args,
-      "i32", "1"  # wait
-    ])
-
-  def get_sim_info(self):
-    # Used for athena
-    # TODO: build using methods from this class
-    sim_state = getprop("gsm.sim.state").split(",")
-    network_type = getprop("gsm.network.type").split(',')
-    mcc_mnc = getprop("gsm.sim.operator.numeric") or None
-
-    sim_id = parse_service_call_string(service_call(['iphonesubinfo', '11']))
-    cell_data_state = parse_service_call_unpack(service_call(['phone', '46']), ">q")
-    cell_data_connected = (cell_data_state == 2)
-
-    return {
-      'sim_id': sim_id,
-      'mcc_mnc': mcc_mnc,
-      'network_type': network_type,
-      'sim_state': sim_state,
-      'data_connected': cell_data_connected
-    }
-
-  def get_network_type(self):
-    wifi_check = parse_service_call_string(service_call(["connectivity", "2"]))
-    if wifi_check is None:
-      return NetworkType.none
-    elif 'WIFI' in wifi_check:
-      return NetworkType.wifi
-    else:
-      cell_check = parse_service_call_unpack(service_call(['phone', '59']), ">q")
-      # from TelephonyManager.java
-      cell_networks = {
-        0: NetworkType.none,
-        1: NetworkType.cell2G,
-        2: NetworkType.cell2G,
-        3: NetworkType.cell3G,
-        4: NetworkType.cell2G,
-        5: NetworkType.cell3G,
-        6: NetworkType.cell3G,
-        7: NetworkType.cell3G,
-        8: NetworkType.cell3G,
-        9: NetworkType.cell3G,
-        10: NetworkType.cell3G,
-        11: NetworkType.cell2G,
-        12: NetworkType.cell3G,
-        13: NetworkType.cell4G,
-        14: NetworkType.cell4G,
-        15: NetworkType.cell3G,
-        16: NetworkType.cell2G,
-        17: NetworkType.cell3G,
-        18: NetworkType.cell4G,
-        19: NetworkType.cell4G
-      }
-      return cell_networks.get(cell_check, NetworkType.none)
-
-  def get_network_strength(self, network_type):
-    network_strength = NetworkStrength.unknown
-
-    # from SignalStrength.java
-    def get_lte_level(rsrp, rssnr):
-      INT_MAX = 2147483647
-      if rsrp == INT_MAX:
-        lvl_rsrp = NetworkStrength.unknown
-      elif rsrp >= -95:
-        lvl_rsrp = NetworkStrength.great
-      elif rsrp >= -105:
-        lvl_rsrp = NetworkStrength.good
-      elif rsrp >= -115:
-        lvl_rsrp = NetworkStrength.moderate
-      else:
-        lvl_rsrp = NetworkStrength.poor
-      if rssnr == INT_MAX:
-        lvl_rssnr = NetworkStrength.unknown
-      elif rssnr >= 45:
-        lvl_rssnr = NetworkStrength.great
-      elif rssnr >= 10:
-        lvl_rssnr = NetworkStrength.good
-      elif rssnr >= -30:
-        lvl_rssnr = NetworkStrength.moderate
-      else:
-        lvl_rssnr = NetworkStrength.poor
-      return max(lvl_rsrp, lvl_rssnr)
-
-    def get_tdscdma_level(tdscmadbm):
-      lvl = NetworkStrength.unknown
-      if tdscmadbm > -25:
-        lvl = NetworkStrength.unknown
-      elif tdscmadbm >= -49:
-        lvl = NetworkStrength.great
-      elif tdscmadbm >= -73:
-        lvl = NetworkStrength.good
-      elif tdscmadbm >= -97:
-        lvl = NetworkStrength.moderate
-      elif tdscmadbm >= -110:
-        lvl = NetworkStrength.poor
-      return lvl
-
-    def get_gsm_level(asu):
-      if asu <= 2 or asu == 99:
-        lvl = NetworkStrength.unknown
-      elif asu >= 12:
-        lvl = NetworkStrength.great
-      elif asu >= 8:
-        lvl = NetworkStrength.good
-      elif asu >= 5:
-        lvl = NetworkStrength.moderate
-      else:
-        lvl = NetworkStrength.poor
-      return lvl
-
-    def get_evdo_level(evdodbm, evdosnr):
-      lvl_evdodbm = NetworkStrength.unknown
-      lvl_evdosnr = NetworkStrength.unknown
-      if evdodbm >= -65:
-        lvl_evdodbm = NetworkStrength.great
-      elif evdodbm >= -75:
-        lvl_evdodbm = NetworkStrength.good
-      elif evdodbm >= -90:
-        lvl_evdodbm = NetworkStrength.moderate
-      elif evdodbm >= -105:
-        lvl_evdodbm = NetworkStrength.poor
-      if evdosnr >= 7:
-        lvl_evdosnr = NetworkStrength.great
-      elif evdosnr >= 5:
-        lvl_evdosnr = NetworkStrength.good
-      elif evdosnr >= 3:
-        lvl_evdosnr = NetworkStrength.moderate
-      elif evdosnr >= 1:
-        lvl_evdosnr = NetworkStrength.poor
-      return max(lvl_evdodbm, lvl_evdosnr)
-
-    def get_cdma_level(cdmadbm, cdmaecio):
-      lvl_cdmadbm = NetworkStrength.unknown
-      lvl_cdmaecio = NetworkStrength.unknown
-      if cdmadbm >= -75:
-        lvl_cdmadbm = NetworkStrength.great
-      elif cdmadbm >= -85:
-        lvl_cdmadbm = NetworkStrength.good
-      elif cdmadbm >= -95:
-        lvl_cdmadbm = NetworkStrength.moderate
-      elif cdmadbm >= -100:
-        lvl_cdmadbm = NetworkStrength.poor
-      if cdmaecio >= -90:
-        lvl_cdmaecio = NetworkStrength.great
-      elif cdmaecio >= -110:
-        lvl_cdmaecio = NetworkStrength.good
-      elif cdmaecio >= -130:
-        lvl_cdmaecio = NetworkStrength.moderate
-      elif cdmaecio >= -150:
-        lvl_cdmaecio = NetworkStrength.poor
-      return max(lvl_cdmadbm, lvl_cdmaecio)
-
-    if network_type == NetworkType.none:
-      return network_strength
-    if network_type == NetworkType.wifi:
-      out = subprocess.check_output('dumpsys connectivity', shell=True).decode('utf-8')
-      network_strength = NetworkStrength.unknown
-      for line in out.split('\n'):
-        signal_str = "SignalStrength: "
-        if signal_str in line:
-          lvl_idx_start = line.find(signal_str) + len(signal_str)
-          lvl_idx_end = line.find(']', lvl_idx_start)
-          lvl = int(line[lvl_idx_start : lvl_idx_end])
-          if lvl >= -50:
-            network_strength = NetworkStrength.great
-          elif lvl >= -60:
-            network_strength = NetworkStrength.good
-          elif lvl >= -70:
-            network_strength = NetworkStrength.moderate
-          else:
-            network_strength = NetworkStrength.poor
-      return network_strength
-    else:
-      # check cell strength
-      out = subprocess.check_output('dumpsys telephony.registry', shell=True).decode('utf-8')
-      for line in out.split('\n'):
-        if "mSignalStrength" in line:
-          arr = line.split(' ')
-          ns = 0
-          if ("gsm" in arr[14]):
-            rsrp = int(arr[9])
-            rssnr = int(arr[11])
-            ns = get_lte_level(rsrp, rssnr)
-            if ns == NetworkStrength.unknown:
-              tdscmadbm = int(arr[13])
-              ns = get_tdscdma_level(tdscmadbm)
-              if ns == NetworkStrength.unknown:
-                asu = int(arr[1])
-                ns = get_gsm_level(asu)
-          else:
-            cdmadbm = int(arr[3])
-            cdmaecio = int(arr[4])
-            evdodbm = int(arr[5])
-            evdosnr = int(arr[7])
-            lvl_cdma = get_cdma_level(cdmadbm, cdmaecio)
-            lvl_edmo = get_evdo_level(evdodbm, evdosnr)
-            if lvl_edmo == NetworkStrength.unknown:
-              ns = lvl_cdma
-            elif lvl_cdma == NetworkStrength.unknown:
-              ns = lvl_edmo
-            else:
-              ns = min(lvl_cdma, lvl_edmo)
-          network_strength = max(network_strength, ns)
-
-      return network_strength
@@ -1,41 +0,0 @@
-from abc import abstractmethod
-
-
-class HardwareBase:
-  @staticmethod
-  def get_cmdline():
-    with open('/proc/cmdline') as f:
-      cmdline = f.read()
-    return {kv[0]: kv[1] for kv in [s.split('=') for s in cmdline.split(' ')] if len(kv) == 2}
-
-  @abstractmethod
-  def get_sound_card_online(self):
-    pass
-
-  @abstractmethod
-  def get_imei(self, slot):
-    pass
-
-  @abstractmethod
-  def get_serial(self):
-    pass
-
-  @abstractmethod
-  def get_subscriber_info(self):
-    pass
-
-  @abstractmethod
-  def reboot(self, reason=None):
-    pass
-
-  @abstractmethod
-  def get_network_type(self):
-    pass
-
-  @abstractmethod
-  def get_sim_info(self):
-    pass
-
-  @abstractmethod
-  def get_network_strength(self, network_type):
-    pass
@@ -1,59 +0,0 @@
-import serial
-
-from common.hardware_base import HardwareBase
-from cereal import log
-import subprocess
-
-
-NetworkType = log.ThermalData.NetworkType
-NetworkStrength = log.ThermalData.NetworkStrength
-
-
-def run_at_command(cmd, timeout=0.1):
-  with serial.Serial("/dev/ttyUSB2", timeout=timeout) as ser:
-    ser.write(cmd + b"\r\n")
-    ser.readline()  # Modem echos request
-    return ser.readline().decode().rstrip()
-
-
-class Tici(HardwareBase):
-  def get_sound_card_online(self):
-    return True
-
-  def get_imei(self, slot):
-    if slot != 0:
-      return ""
-
-    for _ in range(10):
-      try:
-        imei = run_at_command(b"AT+CGSN")
-        if len(imei) == 15:
-          return imei
-      except serial.SerialException:
-        pass
-
-    raise RuntimeError("Error getting IMEI")
-
-  def get_serial(self):
-    return self.get_cmdline()['androidboot.serialno']
-
-  def get_subscriber_info(self):
-    return ""
-
-  def reboot(self, reason=None):
-    subprocess.check_output(["sudo", "reboot"])
-
-  def get_network_type(self):
-    return NetworkType.wifi
-
-  def get_sim_info(self):
-    return {
-      'sim_id': '',
-      'mcc_mnc': None,
-      'network_type': ["Unknown"],
-      'sim_state': ["ABSENT"],
-      'data_connected': False
-    }
-
-  def get_network_strength(self, network_type):
-    return NetworkStrength.unknown
@@ -1,14 +0,0 @@
-import gettext
-from common.hardware import EON
-from common.hardware_android import getprop
-
-locale_dir = '/data/openpilot/selfdrive/assets/locales'
-supported_language = ['en-US', 'zh-TW', 'zh-CN', 'ja-JP', 'ko-KR']
-
-def get_locale():
-  return getprop("persist.sys.locale") if EON else 'en-US'
-
-def events():
-  i18n = gettext.translation('events', localedir=locale_dir, fallback=True, languages=[get_locale()])
-  i18n.install()
-  return i18n.gettext
@@ -1,6 +1,6 @@
-Import('env', 'cython_dependencies')
+Import('env')

 env.Command(['simple_kalman_impl.so'],
-            cython_dependencies + ['simple_kalman_impl.pyx', 'simple_kalman_impl.pxd', 'simple_kalman_setup.py'],
-            "cd common/kalman && python3 simple_kalman_setup.py build_ext --inplace")
+  ['simple_kalman_impl.pyx', 'simple_kalman_impl.pxd', 'simple_kalman_setup.py'],
+  "cd common/kalman && python3 simple_kalman_setup.py build_ext --inplace")

@@ -1,5 +1,3 @@
-# cython: language_level = 3
-
 cdef class KF1D:
  cdef public:
    double x0_0
@@ -15,4 +13,4 @@ cdef class KF1D:
    double A_K_0
    double A_K_1
    double A_K_2
-    double A_K_3
+    double A_K_3
@@ -8,7 +8,7 @@ class KF1D:
  def __init__(self, x0, A, C, K):
    self.x = x0
    self.A = A
-    self.C = np.atleast_2d(C)
+    self.C = C
    self.K = K

    self.A_K = self.A - np.dot(self.K, self.C)
@@ -6,5 +6,4 @@ from common.cython_hacks import BuildExtWithoutPlatformSuffix

 setup(name='Simple Kalman Implementation',
      cmdclass={'build_ext': BuildExtWithoutPlatformSuffix},
-      ext_modules=cythonize(Extension("simple_kalman_impl",
-                                      ["simple_kalman_impl.pyx"])))
+      ext_modules=cythonize(Extension("simple_kalman_impl", ["simple_kalman_impl.pyx"])))
@@ -21,10 +21,10 @@ class TestSimpleKalman(unittest.TestCase):
    K0_0 = 0.12287673
    K1_0 = 0.29666309

-    self.kf_old = KF1D_old(x0=np.array([[x0_0], [x1_0]]),
-                           A=np.array([[A0_0, A0_1], [A1_0, A1_1]]),
-                           C=np.array([C0_0, C0_1]),
-                           K=np.array([[K0_0], [K1_0]]))
+    self.kf_old = KF1D_old(x0=np.matrix([[x0_0], [x1_0]]),
+                           A=np.matrix([[A0_0, A0_1], [A1_0, A1_1]]),
+                           C=np.matrix([C0_0, C0_1]),
+                           K=np.matrix([[K0_0], [K1_0]]))

    self.kf = KF1D(x0=[[x0_0], [x1_0]],
                   A=[[A0_0, A0_1], [A1_0, A1_1]],
@@ -47,8 +47,9 @@ class TestSimpleKalman(unittest.TestCase):
      x = self.kf.update(v_wheel)

      # Compare the output x, verify that the error is less than 1e-4
-      np.testing.assert_almost_equal(x_old[0], x[0])
-      np.testing.assert_almost_equal(x_old[1], x[1])
+      self.assertAlmostEqual(x_old[0], x[0])
+      self.assertAlmostEqual(x_old[1], x[1])
+

  def test_new_is_faster(self):
    setup = """
@@ -69,10 +70,10 @@ C0_1 = 0.0
 K0_0 = 0.12287673
 K1_0 = 0.29666309

-kf_old = KF1D_old(x0=np.array([[x0_0], [x1_0]]),
-                  A=np.array([[A0_0, A0_1], [A1_0, A1_1]]),
-                  C=np.array([C0_0, C0_1]),
-                  K=np.array([[K0_0], [K1_0]]))
+kf_old = KF1D_old(x0=np.matrix([[x0_0], [x1_0]]),
+                  A=np.matrix([[A0_0, A0_1], [A1_0, A1_1]]),
+                  C=np.matrix([C0_0, C0_1]),
+                  K=np.matrix([[K0_0], [K1_0]]))

 kf = KF1D(x0=[[x0_0], [x1_0]],
          A=[[A0_0, A0_1], [A1_0, A1_1]],
@@ -68,11 +68,8 @@ class SwagErrorFilter(logging.Filter):
  def filter(self, record):
    return record.levelno < logging.ERROR

-def _tmpfunc():
-  return 0
-
-def _srcfile():
-  return os.path.normcase(_tmpfunc.__code__.co_filename)
+_tmpfunc = lambda: 0
+_srcfile = os.path.normcase(_tmpfunc.__code__.co_filename)

 class SwagLogger(logging.Logger):
  def __init__(self):
@@ -115,6 +112,9 @@ class SwagLogger(logging.Logger):
    if args:
      evt['args'] = args
    evt.update(kwargs)
+    ctx = self.get_ctx()
+    if ctx:
+      evt['ctx'] = self.get_ctx()
    if 'error' in kwargs:
      self.error(evt)
    else:
@@ -140,9 +140,7 @@ class SwagLogger(logging.Logger):
    while hasattr(f, "f_code"):
        co = f.f_code
        filename = os.path.normcase(co.co_filename)
-
-        # TODO: is this pylint exception correct?
-        if filename == _srcfile:  # pylint: disable=comparison-with-callable
+        if filename == _srcfile:
            f = f.f_back
            continue
        sinfo = None
@@ -0,0 +1,50 @@
+def cputime_total(ct):
+  return ct.cpuUser + ct.cpuSystem + ct.cpuChildrenUser + ct.cpuChildrenSystem
+
+
+def print_cpu_usage(first_proc, last_proc):
+  r = 0
+  procs = [
+      ("selfdrive.controls.controlsd", 59.46),
+      ("./_modeld", 48.94),
+      ("./loggerd", 28.49),
+      ("selfdrive.controls.plannerd", 19.77),
+      ("selfdrive.controls.radard", 9.54),
+      ("./_ui", 9.54),
+      ("./camerad", 7.07),
+      ("selfdrive.locationd.locationd", 7.13),
+      ("./_sensord", 6.17),
+      ("selfdrive.controls.dmonitoringd", 5.48),
+      ("./boardd", 3.63),
+      ("./_dmonitoringmodeld", 2.67),
+      ("selfdrive.logmessaged", 2.71),
+      ("selfdrive.thermald", 2.41),
+      ("./paramsd", 2.18),
+      ("selfdrive.locationd.calibrationd", 1.76),
+      ("./proclogd", 1.54),
+      ("./_gpsd", 0.09),
+      ("./clocksd", 0.02),
+      ("./ubloxd", 0.02),
+      ("selfdrive.tombstoned", 0),
+      ("./logcatd", 0),
+      ("selfdrive.updated", 0),
+  ]
+
+  dt = (last_proc.logMonoTime - first_proc.logMonoTime) / 1e9
+  print("------------------------------------------------")
+  for proc_name, normal_cpu_usage in procs:
+    try:
+      first = [p for p in first_proc.procLog.procs if proc_name in p.cmdline][0]
+      last = [p for p in last_proc.procLog.procs if proc_name in p.cmdline][0]
+      cpu_time = cputime_total(last) - cputime_total(first)
+      cpu_usage = cpu_time / dt * 100.
+      if cpu_usage > max(normal_cpu_usage * 1.1, normal_cpu_usage + 5.0):
+        print(f"Warning {proc_name} using more CPU than normal")
+        r = 1
+
+      print(f"{proc_name.ljust(35)}  {cpu_usage:.2f}%")
+    except IndexError:
+      print(f"{proc_name.ljust(35)}  NO METRICS FOUND")
+  print("------------------------------------------------")
+
+  return r
@@ -6,7 +6,6 @@ def clip(x, lo, hi):

 def interp(x, xp, fp):
  N = len(xp)
-
  def get_interp(xv):
    hi = 0
    while hi < N and xv > xp[hi]:
@@ -15,8 +14,8 @@ def interp(x, xp, fp):
    return fp[-1] if hi == N and xv > xp[low] else (
      fp[0] if hi == 0 else
      (xv - xp[low]) * (fp[hi] - fp[low]) / (xp[hi] - xp[low]) + fp[low])
-
-  return [get_interp(v) for v in x] if hasattr(x, '__iter__') else get_interp(x)
+  return [get_interp(v) for v in x] if hasattr(
+    x, '__iter__') else get_interp(x)

 def mean(x):
  return sum(x) / len(x)
@@ -1,4 +1,513 @@
-from common.params_pyx import Params, UnknownKeyName, put_nonblocking # pylint: disable=no-name-in-module, import-error
-assert Params
-assert UnknownKeyName
-assert put_nonblocking
+#!/usr/bin/env python3
+"""ROS has a parameter server, we have files.
+
+The parameter store is a persistent key value store, implemented as a directory with a writer lock.
+On Android, we store params under params_dir = /data/params. The writer lock is a file
+"<params_dir>/.lock" taken using flock(), and data is stored in a directory symlinked to by
+"<params_dir>/d".
+
+Each key, value pair is stored as a file with named <key> with contents <value>, located in
+  <params_dir>/d/<key>
+
+Readers of a single key can just open("<params_dir>/d/<key>") and read the file contents.
+Readers who want a consistent snapshot of multiple keys should take the lock.
+
+Writers should take the lock before modifying anything. Writers should also leave the DB in a
+consistent state after a crash. The implementation below does this by copying all params to a temp
+directory <params_dir>/<tmp>, then atomically symlinking <params_dir>/<d> to <params_dir>/<tmp>
+before deleting the old <params_dir>/<d> directory.
+
+Writers that only modify a single key can simply take the lock, then swap the corresponding value
+file in place without messing with <params_dir>/d.
+"""
+import time
+import os
+import string
+import binascii
+import errno
+import sys
+import shutil
+import fcntl
+import tempfile
+import threading
+from enum import Enum
+from common.basedir import PARAMS
+
+def mkdirs_exists_ok(path):
+  try:
+    os.makedirs(path)
+  except OSError:
+    if not os.path.isdir(path):
+      raise
+
+
+class TxType(Enum):
+  PERSISTENT = 1
+  CLEAR_ON_MANAGER_START = 2
+  CLEAR_ON_PANDA_DISCONNECT = 3
+
+
+class UnknownKeyName(Exception):
+  pass
+
+
+keys = {
+  "AccessToken": [TxType.CLEAR_ON_MANAGER_START],
+  "AthenadPid": [TxType.PERSISTENT],
+  "CalibrationParams": [TxType.PERSISTENT],
+  "CarParams": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
+  "CarParamsCache": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
+  "CarVin": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
+  "CommunityFeaturesToggle": [TxType.PERSISTENT],
+  "CompletedTrainingVersion": [TxType.PERSISTENT],
+  "ControlsParams": [TxType.PERSISTENT],
+  "DisablePowerDown": [TxType.PERSISTENT],
+  "DoUninstall": [TxType.CLEAR_ON_MANAGER_START],
+  "DongleId": [TxType.PERSISTENT],
+  "GitBranch": [TxType.PERSISTENT],
+  "GitCommit": [TxType.PERSISTENT],
+  "GitRemote": [TxType.PERSISTENT],
+  "GithubSshKeys": [TxType.PERSISTENT],
+  "HasAcceptedTerms": [TxType.PERSISTENT],
+  "HasCompletedSetup": [TxType.PERSISTENT],
+  "IsDriverViewEnabled": [TxType.CLEAR_ON_MANAGER_START],
+  "IsLdwEnabled": [TxType.PERSISTENT],
+  "IsGeofenceEnabled": [TxType.PERSISTENT],
+  "IsMetric": [TxType.PERSISTENT],
+  "IsOffroad": [TxType.CLEAR_ON_MANAGER_START],
+  "IsRHD": [TxType.PERSISTENT],
+  "IsTakingSnapshot": [TxType.CLEAR_ON_MANAGER_START],
+  "IsUpdateAvailable": [TxType.CLEAR_ON_MANAGER_START],
+  "IsUploadRawEnabled": [TxType.PERSISTENT],
+  "LastAthenaPingTime": [TxType.PERSISTENT],
+  "LastUpdateTime": [TxType.PERSISTENT],
+  "LimitSetSpeed": [TxType.PERSISTENT],
+  "LimitSetSpeedNeural": [TxType.PERSISTENT],
+  "LiveParameters": [TxType.PERSISTENT],
+  "LongitudinalControl": [TxType.PERSISTENT],
+  "OpenpilotEnabledToggle": [TxType.PERSISTENT],
+  "LaneChangeEnabled": [TxType.PERSISTENT],
+  "PandaFirmware": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
+  "PandaFirmwareHex": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
+  "PandaDongleId": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
+  "Passive": [TxType.PERSISTENT],
+  "RecordFront": [TxType.PERSISTENT],
+  "ReleaseNotes": [TxType.PERSISTENT],
+  "ShouldDoUpdate": [TxType.CLEAR_ON_MANAGER_START],
+  "SpeedLimitOffset": [TxType.PERSISTENT],
+  "SubscriberInfo": [TxType.PERSISTENT],
+  "TermsVersion": [TxType.PERSISTENT],
+  "TrainingVersion": [TxType.PERSISTENT],
+  "UpdateAvailable": [TxType.CLEAR_ON_MANAGER_START],
+  "UpdateFailedCount": [TxType.CLEAR_ON_MANAGER_START],
+  "Version": [TxType.PERSISTENT],
+  "Offroad_ChargeDisabled": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
+  "Offroad_ConnectivityNeeded": [TxType.CLEAR_ON_MANAGER_START],
+  "Offroad_ConnectivityNeededPrompt": [TxType.CLEAR_ON_MANAGER_START],
+  "Offroad_TemperatureTooHigh": [TxType.CLEAR_ON_MANAGER_START],
+  "Offroad_PandaFirmwareMismatch": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
+  "Offroad_InvalidTime": [TxType.CLEAR_ON_MANAGER_START],
+  "Offroad_IsTakingSnapshot": [TxType.CLEAR_ON_MANAGER_START],
+  #dragonpilot config
+  "DragonEnableDashcam": [TxType.PERSISTENT],
+  "DragonEnableAutoShutdown": [TxType.PERSISTENT],
+  "DragonAutoShutdownAt": [TxType.PERSISTENT],
+  "DragonEnableSteeringOnSignal": [TxType.PERSISTENT],
+  "DragonEnableLogger": [TxType.PERSISTENT],
+  "DragonEnableUploader": [TxType.PERSISTENT],
+  "DragonNoctuaMode": [TxType.PERSISTENT],
+  "DragonCacheCar": [TxType.PERSISTENT],
+  "DragonCachedModel": [TxType.CLEAR_ON_MANAGER_START], # deprecated
+  "DragonCachedFP": [TxType.CLEAR_ON_MANAGER_START], # deprecated
+  "DragonCachedVIN": [TxType.CLEAR_ON_MANAGER_START], # deprecated
+  "DragonCachedCarFW": [TxType.CLEAR_ON_MANAGER_START], # deprecated
+  "DragonCachedSource": [TxType.CLEAR_ON_MANAGER_START], # deprecated
+  "DragonAllowGas": [TxType.PERSISTENT],
+  "DragonToyotaStockDSU": [TxType.PERSISTENT],
+  "DragonLatCtrl": [TxType.PERSISTENT],
+  "DragonUISpeed": [TxType.PERSISTENT],
+  "DragonUIEvent": [TxType.PERSISTENT],
+  "DragonUIMaxSpeed": [TxType.PERSISTENT],
+  "DragonUIFace": [TxType.PERSISTENT],
+  "DragonUIDev": [TxType.PERSISTENT],
+  "DragonUIDevMini": [TxType.PERSISTENT],
+  "DragonEnableTomTom": [TxType.PERSISTENT],
+  "DragonBootTomTom": [TxType.PERSISTENT],
+  "DragonRunTomTom": [TxType.PERSISTENT],
+  "DragonEnableAutonavi": [TxType.PERSISTENT],
+  "DragonBootAutonavi": [TxType.PERSISTENT],
+  "DragonRunAutonavi": [TxType.PERSISTENT],
+  "DragonEnableAegis": [TxType.PERSISTENT],
+  "DragonBootAegis": [TxType.PERSISTENT],
+  "DragonRunAegis": [TxType.PERSISTENT],
+  "DragonEnableMixplorer": [TxType.PERSISTENT],
+  "DragonRunMixplorer": [TxType.PERSISTENT],
+  "DragonSteeringMonitorTimer": [TxType.PERSISTENT],
+  "DragonCameraOffset": [TxType.PERSISTENT],
+  "DragonUIVolumeBoost": [TxType.PERSISTENT],
+  "DragonGreyPandaMode": [TxType.PERSISTENT],
+  "DragonDrivingUI": [TxType.PERSISTENT],
+  "DragonDisplaySteeringLimitAlert": [TxType.PERSISTENT],
+  "DragonChargingCtrl": [TxType.PERSISTENT],
+  "DragonCharging": [TxType.PERSISTENT],
+  "DragonDisCharging": [TxType.PERSISTENT],
+  "DragonToyotaLaneDepartureWarning": [TxType.PERSISTENT],
+  "DragonUILane": [TxType.PERSISTENT],
+  "DragonUILead": [TxType.PERSISTENT],
+  "DragonUIPath": [TxType.PERSISTENT],
+  "DragonUIBlinker": [TxType.PERSISTENT],
+  "DragonUIDMView": [TxType.PERSISTENT],
+  "DragonEnableDriverMonitoring": [TxType.PERSISTENT],
+  "DragonCarModel": [TxType.CLEAR_ON_MANAGER_START],
+  "DragonEnableSlowOnCurve": [TxType.PERSISTENT],
+  "DragonEnableLeadCarMovingAlert": [TxType.PERSISTENT],
+  "DragonToyotaSnGMod": [TxType.PERSISTENT],
+  "DragonWazeMode": [TxType.PERSISTENT],
+  "DragonRunWaze": [TxType.PERSISTENT],
+  "DragonEnableAutoLC": [TxType.PERSISTENT],
+  "DragonAssistedLCMinMPH": [TxType.PERSISTENT],
+  "DragonAutoLCMinMPH": [TxType.PERSISTENT],
+  "DragonAutoLCDelay": [TxType.PERSISTENT],
+  "DragonBTG": [TxType.PERSISTENT],
+  "DragonBootHotspot": [TxType.PERSISTENT],
+  "DragonAccelProfile": [TxType.PERSISTENT],
+  "DragonLastModified": [TxType.CLEAR_ON_MANAGER_START],
+  "DragonEnableRegistration": [TxType.PERSISTENT],
+  "DragonDynamicFollow": [TxType.PERSISTENT],
+  "DragonToyotaSngResponse": [TxType.PERSISTENT],
+  "DragonEnableGearCheck": [TxType.PERSISTENT],
+  "DragonEnableTempMonitor": [TxType.PERSISTENT],
+  "DragonAppAutoUpdate": [TxType.PERSISTENT],
+  "DragonUpdating": [TxType.CLEAR_ON_MANAGER_START],
+  "DragonCustomModel": [TxType.PERSISTENT],
+  "DragonSupportedCars": [TxType.PERSISTENT],
+  "DragonLocale": [TxType.PERSISTENT],
+  "DragonUIScreenOffReversing": [TxType.PERSISTENT],
+  "DragonEnableSRLearner": [TxType.PERSISTENT],
+  "DragonEnableSteerBoost": [TxType.PERSISTENT],
+  "DragonSteerBoostMin": [TxType.PERSISTENT],
+  "DragonSteerBoostMax": [TxType.PERSISTENT],
+  "DragonSteerBoostMinAt": [TxType.PERSISTENT],
+  "DragonSteerBoostMaxAt": [TxType.PERSISTENT],
+  "DragonDashcamHours": [TxType.PERSISTENT],
+  "DragonUIScreenOffDriving": [TxType.PERSISTENT],
+  "DragonEnableAutoUpdate": [TxType.PERSISTENT],
+  "DragonUIBrightness": [TxType.PERSISTENT],
+  "DragonEnableUploadOnMobile": [TxType.PERSISTENT],
+  "DragonEnableUploadOnHotspot": [TxType.PERSISTENT],
+  "DragonMaxSpeedLimit": [TxType.PERSISTENT],
+  "DragonEnableRating": [TxType.PERSISTENT],
+  "DragonEnableContALC": [TxType.PERSISTENT],
+  "DragonBlinkerOffTimer": [TxType.PERSISTENT],
+}
+
+
+def fsync_dir(path):
+  fd = os.open(path, os.O_RDONLY)
+  try:
+    os.fsync(fd)
+  finally:
+    os.close(fd)
+
+
+class FileLock():
+  def __init__(self, path, create):
+    self._path = path
+    self._create = create
+    self._fd = None
+
+  def acquire(self):
+    self._fd = os.open(self._path, os.O_CREAT if self._create else 0)
+    fcntl.flock(self._fd, fcntl.LOCK_EX)
+
+  def release(self):
+    if self._fd is not None:
+      os.close(self._fd)
+      self._fd = None
+
+
+class DBAccessor():
+  def __init__(self, path):
+    self._path = path
+    self._vals = None
+
+  def keys(self):
+    self._check_entered()
+    return self._vals.keys()
+
+  def get(self, key):
+    self._check_entered()
+    try:
+      return self._vals[key]
+    except KeyError:
+      return None
+
+  def _get_lock(self, create):
+    lock = FileLock(os.path.join(self._path, ".lock"), create)
+    lock.acquire()
+    return lock
+
+  def _read_values_locked(self):
+    """Callers should hold a lock while calling this method."""
+    vals = {}
+    try:
+      data_path = self._data_path()
+      keys = os.listdir(data_path)
+      for key in keys:
+        with open(os.path.join(data_path, key), "rb") as f:
+          vals[key] = f.read()
+    except (OSError, IOError) as e:
+      # Either the DB hasn't been created yet, or somebody wrote a bug and left the DB in an
+      # inconsistent state. Either way, return empty.
+      if e.errno == errno.ENOENT:
+        return {}
+
+    return vals
+
+  def _data_path(self):
+    return os.path.join(self._path, "d")
+
+  def _check_entered(self):
+    if self._vals is None:
+      raise Exception("Must call __enter__ before using DB")
+
+
+class DBReader(DBAccessor):
+  def __enter__(self):
+    try:
+      lock = self._get_lock(False)
+    except OSError as e:
+      # Do not create lock if it does not exist.
+      if e.errno == errno.ENOENT:
+        self._vals = {}
+        return self
+
+    try:
+      # Read everything.
+      self._vals = self._read_values_locked()
+      return self
+    finally:
+      lock.release()
+
+  def __exit__(self, type, value, traceback): pass
+
+
+class DBWriter(DBAccessor):
+  def __init__(self, path):
+    super(DBWriter, self).__init__(path)
+    self._lock = None
+    self._prev_umask = None
+
+  def put(self, key, value):
+    self._vals[key] = value
+
+  def delete(self, key):
+    self._vals.pop(key, None)
+
+  def __enter__(self):
+    mkdirs_exists_ok(self._path)
+
+    # Make sure we can write and that permissions are correct.
+    self._prev_umask = os.umask(0)
+
+    try:
+      os.chmod(self._path, 0o777)
+      self._lock = self._get_lock(True)
+      self._vals = self._read_values_locked()
+    except:
+      os.umask(self._prev_umask)
+      self._prev_umask = None
+      raise
+
+    return self
+
+  def __exit__(self, type, value, traceback):
+    self._check_entered()
+
+    try:
+      # data_path refers to the externally used path to the params. It is a symlink.
+      # old_data_path is the path currently pointed to by data_path.
+      # tempdir_path is a path where the new params will go, which the new data path will point to.
+      # new_data_path is a temporary symlink that will atomically overwrite data_path.
+      #
+      # The current situation is:
+      #   data_path -> old_data_path
+      # We're going to write params data to tempdir_path
+      #   tempdir_path -> params data
+      # Then point new_data_path to tempdir_path
+      #   new_data_path -> tempdir_path
+      # Then atomically overwrite data_path with new_data_path
+      #   data_path -> tempdir_path
+      old_data_path = None
+      new_data_path = None
+      tempdir_path = tempfile.mkdtemp(prefix=".tmp", dir=self._path)
+
+      try:
+        # Write back all keys.
+        os.chmod(tempdir_path, 0o777)
+        for k, v in self._vals.items():
+          with open(os.path.join(tempdir_path, k), "wb") as f:
+            f.write(v)
+            f.flush()
+            os.fsync(f.fileno())
+        fsync_dir(tempdir_path)
+
+        data_path = self._data_path()
+        try:
+          old_data_path = os.path.join(self._path, os.readlink(data_path))
+        except (OSError, IOError):
+          # NOTE(mgraczyk): If other DB implementations have bugs, this could cause
+          #                 copies to be left behind, but we still want to overwrite.
+          pass
+
+        new_data_path = "{}.link".format(tempdir_path)
+        os.symlink(os.path.basename(tempdir_path), new_data_path)
+        os.rename(new_data_path, data_path)
+        fsync_dir(self._path)
+      finally:
+        # If the rename worked, we can delete the old data. Otherwise delete the new one.
+        success = new_data_path is not None and os.path.exists(data_path) and (
+          os.readlink(data_path) == os.path.basename(tempdir_path))
+
+        if success:
+          if old_data_path is not None:
+            shutil.rmtree(old_data_path)
+        else:
+          shutil.rmtree(tempdir_path)
+
+        # Regardless of what happened above, there should be no link at new_data_path.
+        if new_data_path is not None and os.path.islink(new_data_path):
+          os.remove(new_data_path)
+    finally:
+      os.umask(self._prev_umask)
+      self._prev_umask = None
+
+      # Always release the lock.
+      self._lock.release()
+      self._lock = None
+
+
+def read_db(params_path, key):
+  path = "%s/d/%s" % (params_path, key)
+  try:
+    with open(path, "rb") as f:
+      return f.read()
+  except IOError:
+    return None
+
+def write_db(params_path, key, value):
+  if isinstance(value, str):
+    value = value.encode('utf8')
+
+  prev_umask = os.umask(0)
+  lock = FileLock(params_path+"/.lock", True)
+  lock.acquire()
+
+  try:
+    tmp_path = tempfile.mktemp(prefix=".tmp", dir=params_path)
+    with open(tmp_path, "wb") as f:
+      f.write(value)
+      f.flush()
+      os.fsync(f.fileno())
+
+    path = "%s/d/%s" % (params_path, key)
+    os.rename(tmp_path, path)
+    fsync_dir(os.path.dirname(path))
+  finally:
+    os.umask(prev_umask)
+    lock.release()
+
+class Params():
+  def __init__(self, db=PARAMS):
+    self.db = db
+
+    # create the database if it doesn't exist...
+    if not os.path.exists(self.db+"/d"):
+      with self.transaction(write=True):
+        pass
+
+  def clear_all(self):
+    shutil.rmtree(self.db, ignore_errors=True)
+    with self.transaction(write=True):
+      pass
+
+  def transaction(self, write=False):
+    if write:
+      return DBWriter(self.db)
+    else:
+      return DBReader(self.db)
+
+  def _clear_keys_with_type(self, tx_type):
+    with self.transaction(write=True) as txn:
+      for key in keys:
+        if tx_type in keys[key]:
+          txn.delete(key)
+
+  def manager_start(self):
+    self._clear_keys_with_type(TxType.CLEAR_ON_MANAGER_START)
+
+  def panda_disconnect(self):
+    self._clear_keys_with_type(TxType.CLEAR_ON_PANDA_DISCONNECT)
+
+  def delete(self, key):
+    with self.transaction(write=True) as txn:
+      txn.delete(key)
+
+  def get(self, key, block=False, encoding=None):
+    if key not in keys:
+      raise UnknownKeyName(key)
+
+    while 1:
+      ret = read_db(self.db, key)
+      if not block or ret is not None:
+        break
+      # is polling really the best we can do?
+      time.sleep(0.05)
+
+    if ret is not None and encoding is not None:
+      ret = ret.decode(encoding)
+
+    return ret
+
+  def put(self, key, dat):
+    """
+    Warning: This function blocks until the param is written to disk!
+    In very rare cases this can take over a second, and your code will hang.
+
+    Use the put_nonblocking helper function in time sensitive code, but
+    in general try to avoid writing params as much as possible.
+    """
+
+    if key not in keys:
+      raise UnknownKeyName(key)
+
+    write_db(self.db, key, dat)
+
+
+def put_nonblocking(key, val):
+  def f(key, val):
+    params = Params()
+    params.put(key, val)
+
+  t = threading.Thread(target=f, args=(key, val))
+  t.start()
+  return t
+
+
+if __name__ == "__main__":
+  params = Params()
+  if len(sys.argv) > 2:
+    params.put(sys.argv[1], sys.argv[2])
+  else:
+    for k in keys:
+      pp = params.get(k)
+      if pp is None:
+        print("%s is None" % k)
+      elif all(chr(c) in string.printable for c in pp):
+        print("%s = %s" % (k, pp))
+      else:
+        print("%s = %s" % (k, binascii.hexlify(pp)))
+
+  # Test multiprocess:
+  # seq 0 100000 | xargs -P20 -I{} python common/params.py DongleId {} && sleep 0.05
+  # while python common/params.py DongleId; do sleep 0.05; done
@@ -1,16 +0,0 @@
-from libcpp.string cimport string
-from libcpp cimport bool
-
-cdef extern from "selfdrive/common/params.cc":
-  pass
-
-cdef extern from "selfdrive/common/util.c":
-  pass
-
-cdef extern from "selfdrive/common/params.h":
-  cdef cppclass Params:
-    Params(bool)
-    Params(string)
-    string get(string, bool) nogil
-    int delete_db_value(string)
-    int write_db_value(string, string)
@@ -1,163 +0,0 @@
-# distutils: language = c++
-# cython: language_level = 3
-from libcpp cimport bool
-from libcpp.string cimport string
-from params_pxd cimport Params as c_Params
-from common.dp_conf import init_params_keys
-
-import os
-import threading
-from common.basedir import BASEDIR
-
-cdef enum TxType:
-  PERSISTENT = 1
-  CLEAR_ON_MANAGER_START = 2
-  CLEAR_ON_PANDA_DISCONNECT = 3
-
-keys = {
-  b"AccessToken": [TxType.CLEAR_ON_MANAGER_START],
-  b"AthenadPid": [TxType.PERSISTENT],
-  b"CalibrationParams": [TxType.PERSISTENT],
-  b"CarBatteryCapacity": [TxType.PERSISTENT],
-  b"CarParams": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
-  b"CarParamsCache": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
-  b"CarVin": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
-  b"CommunityFeaturesToggle": [TxType.PERSISTENT],
-  b"CompletedTrainingVersion": [TxType.PERSISTENT],
-  b"DisablePowerDown": [TxType.PERSISTENT],
-  b"DisableUpdates": [TxType.PERSISTENT],
-  b"DoUninstall": [TxType.CLEAR_ON_MANAGER_START],
-  b"DongleId": [TxType.PERSISTENT],
-  b"GitBranch": [TxType.PERSISTENT],
-  b"GitCommit": [TxType.PERSISTENT],
-  b"GitRemote": [TxType.PERSISTENT],
-  b"GithubSshKeys": [TxType.PERSISTENT],
-  b"HasAcceptedTerms": [TxType.PERSISTENT],
-  b"HasCompletedSetup": [TxType.PERSISTENT],
-  b"IsDriverViewEnabled": [TxType.CLEAR_ON_MANAGER_START],
-  b"IsLdwEnabled": [TxType.PERSISTENT],
-  b"IsMetric": [TxType.PERSISTENT],
-  b"IsOffroad": [TxType.CLEAR_ON_MANAGER_START],
-  b"IsRHD": [TxType.PERSISTENT],
-  b"IsTakingSnapshot": [TxType.CLEAR_ON_MANAGER_START],
-  b"IsUpdateAvailable": [TxType.CLEAR_ON_MANAGER_START],
-  b"IsUploadRawEnabled": [TxType.PERSISTENT],
-  b"LastAthenaPingTime": [TxType.PERSISTENT],
-  b"LastUpdateTime": [TxType.PERSISTENT],
-  b"LastUpdateException": [TxType.PERSISTENT],
-  b"LiveParameters": [TxType.PERSISTENT],
-  b"OpenpilotEnabledToggle": [TxType.PERSISTENT],
-  b"LaneChangeEnabled": [TxType.PERSISTENT],
-  b"PandaFirmware": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
-  b"PandaFirmwareHex": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
-  b"PandaDongleId": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
-  b"Passive": [TxType.PERSISTENT],
-  b"RecordFront": [TxType.PERSISTENT],
-  b"ReleaseNotes": [TxType.PERSISTENT],
-  b"ShouldDoUpdate": [TxType.CLEAR_ON_MANAGER_START],
-  b"SubscriberInfo": [TxType.PERSISTENT],
-  b"TermsVersion": [TxType.PERSISTENT],
-  b"TrainingVersion": [TxType.PERSISTENT],
-  b"UpdateAvailable": [TxType.CLEAR_ON_MANAGER_START],
-  b"UpdateFailedCount": [TxType.CLEAR_ON_MANAGER_START],
-  b"Version": [TxType.PERSISTENT],
-  b"Offroad_ChargeDisabled": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
-  b"Offroad_ConnectivityNeeded": [TxType.CLEAR_ON_MANAGER_START],
-  b"Offroad_ConnectivityNeededPrompt": [TxType.CLEAR_ON_MANAGER_START],
-  b"Offroad_TemperatureTooHigh": [TxType.CLEAR_ON_MANAGER_START],
-  b"Offroad_PandaFirmwareMismatch": [TxType.CLEAR_ON_MANAGER_START, TxType.CLEAR_ON_PANDA_DISCONNECT],
-  b"Offroad_InvalidTime": [TxType.CLEAR_ON_MANAGER_START],
-  b"Offroad_IsTakingSnapshot": [TxType.CLEAR_ON_MANAGER_START],
-  b"Offroad_NeosUpdate": [TxType.CLEAR_ON_MANAGER_START],
-  b"Offroad_UpdateFailed": [TxType.CLEAR_ON_MANAGER_START],
-}
-
-keys = init_params_keys(keys, [TxType.PERSISTENT])
-
-def ensure_bytes(v):
-  if isinstance(v, str):
-    return v.encode()
-  else:
-    return v
-
-
-class UnknownKeyName(Exception):
-  pass
-
-cdef class Params:
-  cdef c_Params* p
-
-  def __cinit__(self, d=None, bool persistent_params=False):
-    if d is None:
-      self.p = new c_Params(persistent_params)
-    else:
-      self.p = new c_Params(<string>d.encode())
-
-  def __dealloc__(self):
-    del self.p
-
-  def clear_all(self, tx_type=None):
-    for key in keys:
-      if tx_type is None or tx_type in keys[key]:
-        self.delete(key)
-
-  def manager_start(self):
-    self.clear_all(TxType.CLEAR_ON_MANAGER_START)
-
-  def panda_disconnect(self):
-    self.clear_all(TxType.CLEAR_ON_PANDA_DISCONNECT)
-
-  def get(self, key, block=False, encoding=None):
-    key = ensure_bytes(key)
-
-    if key not in keys:
-      raise UnknownKeyName(key)
-
-    cdef string k = key
-    cdef bool b = block
-
-    cdef string val
-    with nogil:
-      val = self.p.get(k, b)
-
-    if val == b"":
-      if block:
-        # If we got no value while running in blocked mode
-        # it means we got an interrupt while waiting
-        raise KeyboardInterrupt
-      else:
-        return None
-
-    if encoding is not None:
-      return val.decode(encoding)
-    else:
-      return val
-
-  def put(self, key, dat):
-    """
-    Warning: This function blocks until the param is written to disk!
-    In very rare cases this can take over a second, and your code will hang.
-    Use the put_nonblocking helper function in time sensitive code, but
-    in general try to avoid writing params as much as possible.
-    """
-    key = ensure_bytes(key)
-    dat = ensure_bytes(dat)
-
-    if key not in keys:
-      raise UnknownKeyName(key)
-
-    self.p.write_db_value(key, dat)
-
-  def delete(self, key):
-    key = ensure_bytes(key)
-    self.p.delete_db_value(key)
-
-
-def put_nonblocking(key, val, d=None):
-  def f(key, val):
-    params = Params(d)
-    params.put(key, val)
-
-  t = threading.Thread(target=f, args=(key, val))
-  t.start()
-  return t
@@ -1,33 +0,0 @@
-import os
-import subprocess
-from distutils.core import Extension, setup
-from Cython.Build import cythonize
-
-from common.cython_hacks import BuildExtWithoutPlatformSuffix
-from common.basedir import BASEDIR
-from common.hardware import TICI
-
-ARCH = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()  # pylint: disable=unexpected-keyword-arg
-
-sourcefiles = ['params_pyx.pyx']
-extra_compile_args = ["-std=c++11"]
-
-if ARCH == "aarch64":
-  if TICI:
-    extra_compile_args += ["-DQCOM2"]
-  else:
-    extra_compile_args += ["-DQCOM"]
-
-
-setup(name='common',
-      cmdclass={'build_ext': BuildExtWithoutPlatformSuffix},
-      ext_modules=cythonize(
-        Extension(
-          "params_pyx",
-          language="c++",
-          sources=sourcefiles,
-          include_dirs=[BASEDIR, os.path.join(BASEDIR, 'selfdrive')],
-          extra_compile_args=extra_compile_args
-        )
-      )
-)
@@ -36,10 +36,11 @@ class Profiler():
    if not self.enabled:
      return
    self.iter += 1
-    print("******* Profiling %d *******" % self.iter)
+    print("******* Profiling *******")
    for n, ms in sorted(self.cp.items(), key=lambda x: -x[1]):
      if n in self.cp_ignored:
-        print("%30s: %9.2f  avg: %7.2f  percent: %3.0f   IGNORED" % (n, ms*1000.0, ms*1000.0/self.iter, ms/self.tot*100))
+        print("%30s: %9.2f   percent: %3.0f   IGNORED" % (n, ms*1000.0, ms/self.tot*100))
      else:
-        print("%30s: %9.2f  avg: %7.2f  percent: %3.0f" % (n, ms*1000.0, ms*1000.0/self.iter, ms/self.tot*100))
+        print("%30s: %9.2f   percent: %3.0f" % (n, ms*1000.0, ms/self.tot*100))
    print("Iter clock: %2.6f   TOTAL: %2.2f" % (self.tot/self.iter, self.tot))
+
@@ -1,10 +1,11 @@
 """Utilities for reading real time clocks and keeping soft real time constraints."""
-import gc
 import os
 import time
+import platform
+import subprocess
 import multiprocessing
+from cffi import FFI

-from common.hardware import PC
 from common.common_pyx import sec_since_boot  # pylint: disable=no-name-in-module, import-error


@@ -15,26 +16,24 @@ DT_DMON = 0.1  # driver monitoring
 DT_TRML = 0.5  # thermald and manager


-class Priority:
-  MIN_REALTIME = 52 # highest android process priority is 51
-  CTRL_LOW = MIN_REALTIME
-  CTRL_HIGH = MIN_REALTIME + 1
+ffi = FFI()
+ffi.cdef("long syscall(long number, ...);")
+libc = ffi.dlopen(None)


 def set_realtime_priority(level):
-  if not PC:
-    os.sched_setscheduler(0, os.SCHED_FIFO, os.sched_param(level))
+  if os.getuid() != 0:
+    print("not setting priority, not root")
+    return
+  if platform.machine() == "x86_64":
+    NR_gettid = 186
+  elif platform.machine() == "aarch64":
+    NR_gettid = 178
+  else:
+    raise NotImplementedError

-
-def set_core_affinity(core):
-  if not PC:
-    os.sched_setaffinity(0, [core,])
-
-
-def config_realtime_process(core, priority):
-  gc.disable()
-  set_realtime_priority(priority)
-  set_core_affinity(core)
+  tid = libc.syscall(NR_gettid)
+  return subprocess.call(['chrt', '-f', '-p', str(level), str(tid)])


 class Ratekeeper():
@@ -4,17 +4,14 @@ from common.basedir import BASEDIR


 class Spinner():
-  def __init__(self, noop=False):
-    # spinner is currently only implemented for android
-    self.spinner_proc = None
-    if not noop:
-      try:
-        self.spinner_proc = subprocess.Popen(["./spinner"],
-                                             stdin=subprocess.PIPE,
-                                             cwd=os.path.join(BASEDIR, "selfdrive", "ui", "spinner"),
-                                             close_fds=True)
-      except OSError:
-        self.spinner_proc = None
+  def __init__(self):
+    try:
+      self.spinner_proc = subprocess.Popen(["./spinner"],
+                                           stdin=subprocess.PIPE,
+                                           cwd=os.path.join(BASEDIR, "selfdrive", "ui", "spinner"),
+                                           close_fds=True)
+    except OSError:
+      self.spinner_proc = None

  def __enter__(self):
    return self
@@ -39,10 +36,27 @@ class Spinner():
  def __del__(self):
    self.close()

-  def __exit__(self, exc_type, exc_value, traceback):
+  def __exit__(self, type, value, traceback):
    self.close()


+class FakeSpinner():
+  def __init__(self):
+    pass
+
+  def __enter__(self):
+    return self
+
+  def update(self, _):
+    pass
+
+  def close(self):
+    pass
+
+  def __exit__(self, type, value, traceback):
+    pass
+
+
 if __name__ == "__main__":
  import time
  with Spinner() as s:
@@ -32,7 +32,7 @@ class RunningStat():
      self.S_last = 0.
    else:
      self.M = self.M_last + (new_data - self.M_last) / self.n
-      self.S = self.S_last + (new_data - self.M_last) * (new_data - self.M)
+      self.S = self.S_last + (new_data - self.M_last) * (new_data - self.M);
      self.M_last = self.M
      self.S_last = self.S

@@ -64,7 +64,7 @@ class RunningStatFilter():
    _std_last = self.raw_stat.std()
    self.raw_stat.push_data(new_data)
    _delta_std = self.raw_stat.std() - _std_last
-    if _delta_std <= 0:
+    if _delta_std<=0:
      self.filtered_stat.push_data(new_data)
    else:
      pass
@@ -0,0 +1,9 @@
+import os
+from nose.tools import nottest
+
+def phone_only(x):
+  if os.path.isfile("/init.qcom.rc"):
+    return x
+  else:
+    return nottest(x)
+
@@ -5,23 +5,21 @@ import subprocess
 from common.basedir import BASEDIR


-class TextWindow:
-  def __init__(self, s, noop=False):
-    # text window is only implemented for android currently
-    self.text_proc = None
-    if not noop:
-      try:
-        self.text_proc = subprocess.Popen(["./text", s],
-                                          stdin=subprocess.PIPE,
-                                          cwd=os.path.join(BASEDIR, "selfdrive", "ui", "text"),
-                                          close_fds=True)
-      except OSError:
-        self.text_proc = None
+class TextWindow():
+  def __init__(self, s):
+    try:
+      self.text_proc = subprocess.Popen(["./text", s],
+                                        stdin=subprocess.PIPE,
+                                        cwd=os.path.join(BASEDIR, "selfdrive", "ui", "text"),
+                                        close_fds=True)
+    except OSError:
+      self.text_proc = None

  def get_status(self):
    if self.text_proc is not None:
      self.text_proc.poll()
      return self.text_proc.returncode
+
    return None

  def __enter__(self):
@@ -33,20 +31,38 @@ class TextWindow:
      self.text_proc = None

  def wait_for_exit(self):
-    if self.text_proc is not None:
-      while True:
-        if self.get_status() == 1:
-          os.system('/data/openpilot/scripts/reset_update.sh')
-          return
-        time.sleep(0.1)
+    while True:
+      if self.get_status() == 1:
+        return
+      time.sleep(0.1)

  def __del__(self):
    self.close()

-  def __exit__(self, exc_type, exc_value, traceback):
+  def __exit__(self, type, value, traceback):
    self.close()


+class FakeTextWindow():
+  def __init__(self, s):
+    pass
+
+  def get_status(self):
+    return 1
+
+  def wait_for_exit(self):
+    return
+
+  def __enter__(self):
+    return self
+
+  def update(self, _):
+    pass
+
+  def __exit__(self, type, value, traceback):
+    pass
+
+
 if __name__ == "__main__":
  text = """Traceback (most recent call last):
  File "./controlsd.py", line 608, in <module>
@@ -25,3 +25,4 @@ class Timeout:

  def __exit__(self, exc_type, exc_val, exc_tb):
    signal.alarm(0)
+
@@ -1,8 +0,0 @@
-Import('env', 'cython_dependencies')
-
-d = Dir('.')
-
-env.Command(['transformations.so'],
-            cython_dependencies + ['transformations.pxd', 'transformations.pyx',
-             'coordinates.cc', 'orientation.cc', 'coordinates.hpp', 'orientation.hpp'],
-            'cd ' + d.path + ' && python3 setup.py build_ext --inplace')
@@ -1,67 +1,29 @@
 import numpy as np
-
 import common.transformations.orientation as orient
-from common.hardware import TICI

-## -- hardcoded hardware params --
-eon_f_focal_length = 910.0
-eon_d_focal_length = 860.0
-leon_d_focal_length = 650.0
-tici_f_focal_length = 2648.0
-tici_e_focal_length = tici_d_focal_length = 567.0 # probably wrong? magnification is not consistent across frame
-
-eon_f_frame_size = (1164, 874)
-eon_d_frame_size = (1152, 864)
-leon_d_frame_size = (816, 612)
-tici_f_frame_size = tici_e_frame_size = tici_d_frame_size = (1928, 1208)
+FULL_FRAME_SIZE = (1164, 874)
+W, H = FULL_FRAME_SIZE[0], FULL_FRAME_SIZE[1]
+eon_focal_length = FOCAL = 910.0

 # aka 'K' aka camera_frame_from_view_frame
-eon_fcam_intrinsics = np.array([
-  [eon_f_focal_length,  0.0,  float(eon_f_frame_size[0])/2],
-  [0.0,  eon_f_focal_length,  float(eon_f_frame_size[1])/2],
-  [0.0,  0.0,                                          1.0]])
-eon_intrinsics = eon_fcam_intrinsics # xx
+eon_intrinsics = np.array([
+  [FOCAL,   0.,   W/2.],
+  [  0.,  FOCAL,  H/2.],
+  [  0.,    0.,     1.]])
+

 leon_dcam_intrinsics = np.array([
-  [leon_d_focal_length,  0.0,  float(leon_d_frame_size[0])/2],
-  [0.0,  leon_d_focal_length,  float(leon_d_frame_size[1])/2],
-  [0.0,  0.0,                                            1.0]])
+  [650,   0,   816//2],
+  [  0,  650,  612//2],
+  [  0,    0,     1]])

 eon_dcam_intrinsics = np.array([
-  [eon_d_focal_length,  0.0,  float(eon_d_frame_size[0])/2],
-  [0.0,  eon_d_focal_length,  float(eon_d_frame_size[1])/2],
-  [0.0,  0.0,                                          1.0]])
-
-tici_fcam_intrinsics = np.array([
-  [tici_f_focal_length,  0.0,  float(tici_f_frame_size[0])/2],
-  [0.0,  tici_f_focal_length,  float(tici_f_frame_size[1])/2],
-  [0.0,  0.0,                                            1.0]])
-
-tici_dcam_intrinsics = np.array([
-  [tici_d_focal_length,  0.0,  float(tici_d_frame_size[0])/2],
-  [0.0,  tici_d_focal_length,  float(tici_d_frame_size[1])/2],
-  [0.0,  0.0,                                            1.0]])
-
-tici_ecam_intrinsics = tici_dcam_intrinsics
+  [860,   0,   1152//2],
+  [  0,  860,  864//2],
+  [  0,    0,     1]])

 # aka 'K_inv' aka view_frame_from_camera_frame
-eon_fcam_intrinsics_inv = np.linalg.inv(eon_fcam_intrinsics)
-eon_intrinsics_inv = eon_fcam_intrinsics_inv # xx
-
-tici_fcam_intrinsics_inv = np.linalg.inv(tici_fcam_intrinsics)
-tici_ecam_intrinsics_inv = np.linalg.inv(tici_ecam_intrinsics)
-
-
-if not TICI:
-  FULL_FRAME_SIZE = eon_f_frame_size
-  FOCAL = eon_f_focal_length
-  fcam_intrinsics = eon_fcam_intrinsics
-else:
-  FULL_FRAME_SIZE = tici_f_frame_size
-  FOCAL = tici_f_focal_length
-  fcam_intrinsics = tici_fcam_intrinsics
-
-W, H = FULL_FRAME_SIZE[0], FULL_FRAME_SIZE[1]
+eon_intrinsics_inv = np.linalg.inv(eon_intrinsics)


 # device/mesh : x->forward, y-> right, z->down
@@ -90,13 +52,6 @@ def get_view_frame_from_road_frame(roll, pitch, yaw, height):
  return np.hstack((view_from_road, [[0], [height], [0]]))


-# aka 'extrinsic_matrix'
-def get_view_frame_from_calib_frame(roll, pitch, yaw, height):
-  device_from_calib= orient.rot_from_euler([roll, pitch, yaw])
-  view_from_calib = view_frame_from_device_frame.dot(device_from_calib)
-  return np.hstack((view_from_calib, [[0], [height], [0]]))
-
-
 def vp_from_ke(m):
  """
  Computes the vanishing point from the product of the intrinsic and extrinsic
@@ -104,12 +59,12 @@ def vp_from_ke(m):

  The vanishing point is defined as lim x->infinity C (x, 0, 0, 1).T
  """
-  return (m[0, 0]/m[2, 0], m[1, 0]/m[2, 0])
+  return (m[0, 0]/m[2,0], m[1,0]/m[2,0])


-def vp_from_rpy(rpy, intrinsics=fcam_intrinsics):
+def vp_from_rpy(rpy):
  e = get_view_frame_from_road_frame(rpy[0], rpy[1], rpy[2], 1.22)
-  ke = np.dot(intrinsics, e)
+  ke = np.dot(eon_intrinsics, e)
  return vp_from_ke(ke)


@@ -119,32 +74,32 @@ def roll_from_ke(m):
                    -(m[0, 0] - m[0, 1] * m[2, 0] / m[2, 1]))


-def normalize(img_pts, intrinsics=fcam_intrinsics):
+def normalize(img_pts, intrinsics=eon_intrinsics):
  # normalizes image coordinates
  # accepts single pt or array of pts
  intrinsics_inv = np.linalg.inv(intrinsics)
  img_pts = np.array(img_pts)
  input_shape = img_pts.shape
  img_pts = np.atleast_2d(img_pts)
-  img_pts = np.hstack((img_pts, np.ones((img_pts.shape[0], 1))))
+  img_pts = np.hstack((img_pts, np.ones((img_pts.shape[0],1))))
  img_pts_normalized = img_pts.dot(intrinsics_inv.T)
  img_pts_normalized[(img_pts < 0).any(axis=1)] = np.nan
-  return img_pts_normalized[:, :2].reshape(input_shape)
+  return img_pts_normalized[:,:2].reshape(input_shape)


-def denormalize(img_pts, intrinsics=fcam_intrinsics, width=W, height=H):
+def denormalize(img_pts, intrinsics=eon_intrinsics):
  # denormalizes image coordinates
  # accepts single pt or array of pts
  img_pts = np.array(img_pts)
  input_shape = img_pts.shape
  img_pts = np.atleast_2d(img_pts)
-  img_pts = np.hstack((img_pts, np.ones((img_pts.shape[0], 1))))
+  img_pts = np.hstack((img_pts, np.ones((img_pts.shape[0],1))))
  img_pts_denormalized = img_pts.dot(intrinsics.T)
-  img_pts_denormalized[img_pts_denormalized[:, 0] > width] = np.nan
-  img_pts_denormalized[img_pts_denormalized[:, 0] < 0] = np.nan
-  img_pts_denormalized[img_pts_denormalized[:, 1] > height] = np.nan
-  img_pts_denormalized[img_pts_denormalized[:, 1] < 0] = np.nan
-  return img_pts_denormalized[:, :2].reshape(input_shape)
+  img_pts_denormalized[img_pts_denormalized[:,0] > W] = np.nan
+  img_pts_denormalized[img_pts_denormalized[:,0] < 0] = np.nan
+  img_pts_denormalized[img_pts_denormalized[:,1] > H] = np.nan
+  img_pts_denormalized[img_pts_denormalized[:,1] < 0] = np.nan
+  return img_pts_denormalized[:,:2].reshape(input_shape)


 def device_from_ecef(pos_ecef, orientation_ecef, pt_ecef):
@@ -169,16 +124,25 @@ def img_from_device(pt_device):
  pt_view = np.einsum('jk,ik->ij', view_frame_from_device_frame, pt_device)

  # This function should never return negative depths
-  pt_view[pt_view[:, 2] < 0] = np.nan
+  pt_view[pt_view[:,2] < 0] = np.nan

-  pt_img = pt_view/pt_view[:, 2:3]
-  return pt_img.reshape(input_shape)[:, :2]
+  pt_img = pt_view/pt_view[:,2:3]
+  return pt_img.reshape(input_shape)[:,:2]


-def get_camera_frame_from_calib_frame(camera_frame_from_road_frame, intrinsics=fcam_intrinsics):
+def get_camera_frame_from_calib_frame(camera_frame_from_road_frame):
  camera_frame_from_ground = camera_frame_from_road_frame[:, (0, 1, 3)]
-  calib_frame_from_ground = np.dot(intrinsics,
-                                   get_view_frame_from_road_frame(0, 0, 0, 1.22))[:, (0, 1, 3)]
+  calib_frame_from_ground = np.dot(eon_intrinsics,
+                                     get_view_frame_from_road_frame(0, 0, 0, 1.22))[:, (0, 1, 3)]
  ground_from_calib_frame = np.linalg.inv(calib_frame_from_ground)
  camera_frame_from_calib_frame = np.dot(camera_frame_from_ground, ground_from_calib_frame)
  return camera_frame_from_calib_frame
+
+
+def pretransform_from_calib(calib):
+  roll, pitch, yaw, height = calib
+  view_frame_from_road_frame = get_view_frame_from_road_frame(roll, pitch, yaw, height)
+  camera_frame_from_road_frame = np.dot(eon_intrinsics, view_frame_from_road_frame)
+  camera_frame_from_calib_frame = get_camera_frame_from_calib_frame(camera_frame_from_road_frame)
+  return np.linalg.inv(camera_frame_from_calib_frame)
+
@@ -1,104 +0,0 @@
-#define _USE_MATH_DEFINES
-
-#include <iostream>
-#include <cmath>
-#include <eigen3/Eigen/Dense>
-
-#include "coordinates.hpp"
-
-#define DEG2RAD(x) ((x) * M_PI / 180.0)
-#define RAD2DEG(x) ((x) * 180.0 / M_PI)
-
-
-double a = 6378137; // lgtm [cpp/short-global-name]
-double b = 6356752.3142; // lgtm [cpp/short-global-name]
-double esq = 6.69437999014 * 0.001; // lgtm [cpp/short-global-name]
-double e1sq = 6.73949674228 * 0.001;
-
-
-static Geodetic to_degrees(Geodetic geodetic){
-  geodetic.lat = RAD2DEG(geodetic.lat);
-  geodetic.lon = RAD2DEG(geodetic.lon);
-  return geodetic;
-}
-
-static Geodetic to_radians(Geodetic geodetic){
-  geodetic.lat = DEG2RAD(geodetic.lat);
-  geodetic.lon = DEG2RAD(geodetic.lon);
-  return geodetic;
-}
-
-
-ECEF geodetic2ecef(Geodetic g){
-  g = to_radians(g);
-  double xi = sqrt(1.0 - esq * pow(sin(g.lat), 2));
-  double x = (a / xi + g.alt) * cos(g.lat) * cos(g.lon);
-  double y = (a / xi + g.alt) * cos(g.lat) * sin(g.lon);
-  double z = (a / xi * (1.0 - esq) + g.alt) * sin(g.lat);
-  return {x, y, z};
-}
-
-Geodetic ecef2geodetic(ECEF e){
-  // Convert from ECEF to geodetic using Ferrari's methods
-  // https://en.wikipedia.org/wiki/Geographic_coordinate_conversion#Ferrari.27s_solution
-  double x = e.x;
-  double y = e.y;
-  double z = e.z;
-
-  double r = sqrt(x * x + y * y);
-  double Esq = a * a - b * b;
-  double F = 54 * b * b * z * z;
-  double G = r * r + (1 - esq) * z * z - esq * Esq;
-  double C = (esq * esq * F * r * r) / (pow(G, 3));
-  double S = cbrt(1 + C + sqrt(C * C + 2 * C));
-  double P = F / (3 * pow((S + 1 / S + 1), 2) * G * G);
-  double Q = sqrt(1 + 2 * esq * esq * P);
-  double r_0 = -(P * esq * r) / (1 + Q) + sqrt(0.5 * a * a*(1 + 1.0 / Q) - P * (1 - esq) * z * z / (Q * (1 + Q)) - 0.5 * P * r * r);
-  double U = sqrt(pow((r - esq * r_0), 2) + z * z);
-  double V = sqrt(pow((r - esq * r_0), 2) + (1 - esq) * z * z);
-  double Z_0 = b * b * z / (a * V);
-  double h = U * (1 - b * b / (a * V));
-
-  double lat = atan((z + e1sq * Z_0) / r);
-  double lon = atan2(y, x);
-
-  return to_degrees({lat, lon, h});
-}
-
-LocalCoord::LocalCoord(Geodetic g, ECEF e){
-  init_ecef <<  e.x, e.y, e.z;
-
-  g = to_radians(g);
-
-  ned2ecef_matrix <<
-    -sin(g.lat)*cos(g.lon), -sin(g.lon), -cos(g.lat)*cos(g.lon),
-    -sin(g.lat)*sin(g.lon), cos(g.lon), -cos(g.lat)*sin(g.lon),
-    cos(g.lat), 0, -sin(g.lat);
-  ecef2ned_matrix = ned2ecef_matrix.transpose();
-}
-
-NED LocalCoord::ecef2ned(ECEF e) {
-  Eigen::Vector3d ecef;
-  ecef << e.x, e.y, e.z;
-
-  Eigen::Vector3d ned = (ecef2ned_matrix * (ecef - init_ecef));
-  return {ned[0], ned[1], ned[2]};
-}
-
-ECEF LocalCoord::ned2ecef(NED n) {
-  Eigen::Vector3d ned;
-  ned << n.n, n.e, n.d;
-
-  Eigen::Vector3d ecef = (ned2ecef_matrix * ned) + init_ecef;
-  return {ecef[0], ecef[1], ecef[2]};
-}
-
-NED LocalCoord::geodetic2ned(Geodetic g) {
-  ECEF e = ::geodetic2ecef(g);
-  return ecef2ned(e);
-}
-
-Geodetic LocalCoord::ned2geodetic(NED n){
-  ECEF e = ned2ecef(n);
-  return ::ecef2geodetic(e);
-}
@@ -1,35 +0,0 @@
-#pragma once
-
-struct ECEF {
-  double x, y, z;
-  Eigen::Vector3d to_vector(){
-    return Eigen::Vector3d(x, y, z);
-  }
-};
-
-struct NED {
-  double n, e, d;
-};
-
-struct Geodetic {
-  double lat, lon, alt;
-  bool radians=false;
-};
-
-ECEF geodetic2ecef(Geodetic g);
-Geodetic ecef2geodetic(ECEF e);
-
-class LocalCoord {
-public:
-  Eigen::Matrix3d ned2ecef_matrix;
-  Eigen::Matrix3d ecef2ned_matrix;
-  Eigen::Vector3d init_ecef;
-  LocalCoord(Geodetic g, ECEF e);
-  LocalCoord(Geodetic g) : LocalCoord(g, ::geodetic2ecef(g)) {}
-  LocalCoord(ECEF e) : LocalCoord(::ecef2geodetic(e), e) {}
-
-  NED ecef2ned(ECEF e);
-  ECEF ned2ecef(NED n);
-  NED geodetic2ned(Geodetic g);
-  Geodetic ned2geodetic(NED n);
-};
@@ -1,19 +1,108 @@
-# pylint: skip-file
-from common.transformations.orientation import numpy_wrap
-from common.transformations.transformations import (ecef2geodetic_single,
-                                                    geodetic2ecef_single)
-from common.transformations.transformations import LocalCoord as LocalCoord_single
+import numpy as np
+"""
+Coordinate transformation module. All methods accept arrays as input
+with each row as a position.
+"""


-class LocalCoord(LocalCoord_single):
-  ecef2ned = numpy_wrap(LocalCoord_single.ecef2ned_single, (3,), (3,))
-  ned2ecef = numpy_wrap(LocalCoord_single.ned2ecef_single, (3,), (3,))
-  geodetic2ned = numpy_wrap(LocalCoord_single.geodetic2ned_single, (3,), (3,))
-  ned2geodetic = numpy_wrap(LocalCoord_single.ned2geodetic_single, (3,), (3,))
+
+a = 6378137
+b = 6356752.3142
+esq = 6.69437999014 * 0.001
+e1sq = 6.73949674228 * 0.001


-geodetic2ecef = numpy_wrap(geodetic2ecef_single, (3,), (3,))
-ecef2geodetic = numpy_wrap(ecef2geodetic_single, (3,), (3,))
+def geodetic2ecef(geodetic, radians=False):
+  geodetic = np.array(geodetic)
+  input_shape = geodetic.shape
+  geodetic = np.atleast_2d(geodetic)

-geodetic_from_ecef = ecef2geodetic
-ecef_from_geodetic = geodetic2ecef
+  ratio = 1.0 if radians else (np.pi / 180.0)
+  lat = ratio*geodetic[:,0]
+  lon = ratio*geodetic[:,1]
+  alt = geodetic[:,2]
+
+  xi = np.sqrt(1 - esq * np.sin(lat)**2)
+  x = (a / xi + alt) * np.cos(lat) * np.cos(lon)
+  y = (a / xi + alt) * np.cos(lat) * np.sin(lon)
+  z = (a / xi * (1 - esq) + alt) * np.sin(lat)
+  ecef = np.array([x, y, z]).T
+  return ecef.reshape(input_shape)
+
+
+def ecef2geodetic(ecef, radians=False):
+  """
+  Convert ECEF coordinates to geodetic using ferrari's method
+  """
+  # Save shape and export column
+  ecef = np.atleast_1d(ecef)
+  input_shape = ecef.shape
+  ecef = np.atleast_2d(ecef)
+  x, y, z = ecef[:, 0], ecef[:, 1], ecef[:, 2]
+
+  ratio = 1.0 if radians else (180.0 / np.pi)
+
+  # Conver from ECEF to geodetic using Ferrari's methods
+  # https://en.wikipedia.org/wiki/Geographic_coordinate_conversion#Ferrari.27s_solution
+  r = np.sqrt(x * x + y * y)
+  Esq = a * a - b * b
+  F = 54 * b * b * z * z
+  G = r * r + (1 - esq) * z * z - esq * Esq
+  C = (esq * esq * F * r * r) / (pow(G, 3))
+  S = np.cbrt(1 + C + np.sqrt(C * C + 2 * C))
+  P = F / (3 * pow((S + 1 / S + 1), 2) * G * G)
+  Q = np.sqrt(1 + 2 * esq * esq * P)
+  r_0 =  -(P * esq * r) / (1 + Q) + np.sqrt(0.5 * a * a*(1 + 1.0 / Q) - \
+        P * (1 - esq) * z * z / (Q * (1 + Q)) - 0.5 * P * r * r)
+  U = np.sqrt(pow((r - esq * r_0), 2) + z * z)
+  V = np.sqrt(pow((r - esq * r_0), 2) + (1 - esq) * z * z)
+  Z_0 = b * b * z / (a * V)
+  h = U * (1 - b * b / (a * V))
+  lat = ratio*np.arctan((z + e1sq * Z_0) / r)
+  lon = ratio*np.arctan2(y, x)
+
+  # stack the new columns and return to the original shape
+  geodetic = np.column_stack((lat, lon, h))
+  return geodetic.reshape(input_shape)
+
+class LocalCoord():
+  """
+   Allows conversions to local frames. In this case NED.
+   That is: North East Down from the start position in
+   meters.
+  """
+  def __init__(self, init_geodetic, init_ecef):
+    self.init_ecef = init_ecef
+    lat, lon, _ = (np.pi/180)*np.array(init_geodetic)
+    self.ned2ecef_matrix = np.array([[-np.sin(lat)*np.cos(lon), -np.sin(lon), -np.cos(lat)*np.cos(lon)],
+                                     [-np.sin(lat)*np.sin(lon), np.cos(lon), -np.cos(lat)*np.sin(lon)],
+                                     [np.cos(lat), 0, -np.sin(lat)]])
+    self.ecef2ned_matrix = self.ned2ecef_matrix.T
+
+  @classmethod
+  def from_geodetic(cls, init_geodetic):
+    init_ecef = geodetic2ecef(init_geodetic)
+    return LocalCoord(init_geodetic, init_ecef)
+
+  @classmethod
+  def from_ecef(cls, init_ecef):
+    init_geodetic = ecef2geodetic(init_ecef)
+    return LocalCoord(init_geodetic, init_ecef)
+
+
+  def ecef2ned(self, ecef):
+    ecef = np.array(ecef)
+    return np.dot(self.ecef2ned_matrix, (ecef - self.init_ecef).T).T
+
+  def ned2ecef(self, ned):
+    ned = np.array(ned)
+    # Transpose so that init_ecef will broadcast correctly for 1d or 2d ned.
+    return (np.dot(self.ned2ecef_matrix, ned.T).T + self.init_ecef)
+
+  def geodetic2ned(self, geodetic):
+    ecef = geodetic2ecef(geodetic)
+    return self.ecef2ned(ecef)
+
+  def ned2geodetic(self, ned):
+    ecef = self.ned2ecef(ned)
+    return ecef2geodetic(ecef)
@@ -1,33 +1,33 @@
 import numpy as np

-from common.transformations.camera import (FULL_FRAME_SIZE,
-                                           FOCAL,
+from common.transformations.camera import (FULL_FRAME_SIZE, eon_focal_length,
                                           get_view_frame_from_road_frame,
-                                           get_view_frame_from_calib_frame,
                                           vp_from_ke)

 # segnet
+
 SEGNET_SIZE = (512, 384)

-def get_segnet_frame_from_camera_frame(segnet_size=SEGNET_SIZE, full_frame_size=FULL_FRAME_SIZE):
-  return np.array([[float(segnet_size[0]) / full_frame_size[0],  0.0],
-                   [0.0,  float(segnet_size[1]) / full_frame_size[1]]])
-segnet_frame_from_camera_frame = get_segnet_frame_from_camera_frame() # xx
+segnet_frame_from_camera_frame = np.array([
+  [float(SEGNET_SIZE[0])/FULL_FRAME_SIZE[0],    0.,          ],
+  [     0.,          float(SEGNET_SIZE[1])/FULL_FRAME_SIZE[1]]])
+

 # model
+
 MODEL_INPUT_SIZE = (320, 160)
 MODEL_YUV_SIZE = (MODEL_INPUT_SIZE[0], MODEL_INPUT_SIZE[1] * 3 // 2)
-MODEL_CX = MODEL_INPUT_SIZE[0] / 2.
+MODEL_CX = MODEL_INPUT_SIZE[0]/2.
 MODEL_CY = 21.

-model_fl = 728.0
+model_zoom = 1.25
 model_height = 1.22

 # canonical model transform
-model_intrinsics = np.array([
-  [model_fl,  0.0,  MODEL_CX],
-  [0.0,  model_fl,  MODEL_CY],
-  [0.0,  0.0,            1.0]])
+model_intrinsics = np.array(
+  [[ eon_focal_length / model_zoom,    0. ,  MODEL_CX],
+   [   0. ,  eon_focal_length / model_zoom,  MODEL_CY],
+   [   0. ,                            0. ,   1.]])


 # MED model
@@ -35,76 +35,50 @@ MEDMODEL_INPUT_SIZE = (512, 256)
 MEDMODEL_YUV_SIZE = (MEDMODEL_INPUT_SIZE[0], MEDMODEL_INPUT_SIZE[1] * 3 // 2)
 MEDMODEL_CY = 47.6

-medmodel_fl = 910.0
-medmodel_intrinsics = np.array([
-  [medmodel_fl,  0.0,  0.5 * MEDMODEL_INPUT_SIZE[0]],
-  [0.0,  medmodel_fl,                   MEDMODEL_CY],
-  [0.0,  0.0,                                   1.0]])
-
-
-# CAL model
-CALMODEL_INPUT_SIZE = (512, 256)
-CALMODEL_YUV_SIZE = (CALMODEL_INPUT_SIZE[0], CALMODEL_INPUT_SIZE[1] * 3 // 2)
-CALMODEL_CY = 47.6
-
-calmodel_fl = 606.7
-calmodel_intrinsics = np.array([
-  [calmodel_fl,  0.0,  0.5 * CALMODEL_INPUT_SIZE[0]],
-  [0.0,  calmodel_fl,                   CALMODEL_CY],
-  [0.0,  0.0,                                   1.0]])
+medmodel_zoom = 1.
+medmodel_intrinsics = np.array(
+  [[ eon_focal_length / medmodel_zoom,    0. ,  0.5 * MEDMODEL_INPUT_SIZE[0]],
+   [   0. ,  eon_focal_length / medmodel_zoom,  MEDMODEL_CY],
+   [   0. ,                            0. ,   1.]])


 # BIG model
+
 BIGMODEL_INPUT_SIZE = (1024, 512)
 BIGMODEL_YUV_SIZE = (BIGMODEL_INPUT_SIZE[0], BIGMODEL_INPUT_SIZE[1] * 3 // 2)

-bigmodel_fl = 910.0
-bigmodel_intrinsics = np.array([
-  [bigmodel_fl,  0.0,  0.5 * BIGMODEL_INPUT_SIZE[0]],
-  [0.0,  bigmodel_fl,             256 + MEDMODEL_CY],
-  [0.0,  0.0,                                   1.0]])
-
-
-# SBIG model (big model with the size of small model)
-SBIGMODEL_INPUT_SIZE = (512, 256)
-SBIGMODEL_YUV_SIZE = (SBIGMODEL_INPUT_SIZE[0], SBIGMODEL_INPUT_SIZE[1] * 3 // 2)
-
-sbigmodel_fl = 455.0
-sbigmodel_intrinsics = np.array([
-  [sbigmodel_fl,  0.0,  0.5 * SBIGMODEL_INPUT_SIZE[0]],
-  [0.0,  sbigmodel_fl,      0.5 * (256 + MEDMODEL_CY)],
-  [0.0,  0.0,                                     1.0]])
+bigmodel_zoom = 1.
+bigmodel_intrinsics = np.array(
+  [[ eon_focal_length / bigmodel_zoom,    0. , 0.5 * BIGMODEL_INPUT_SIZE[0]],
+   [   0. ,  eon_focal_length / bigmodel_zoom,  256+MEDMODEL_CY],
+   [   0. ,                            0. ,   1.]])

 model_frame_from_road_frame = np.dot(model_intrinsics,
-                                     get_view_frame_from_road_frame(0, 0, 0, model_height))
+  get_view_frame_from_road_frame(0, 0, 0, model_height))

 bigmodel_frame_from_road_frame = np.dot(bigmodel_intrinsics,
-                                        get_view_frame_from_road_frame(0, 0, 0, model_height))
+  get_view_frame_from_road_frame(0, 0, 0, model_height))

 medmodel_frame_from_road_frame = np.dot(medmodel_intrinsics,
-                                        get_view_frame_from_road_frame(0, 0, 0, model_height))
-
-medmodel_frame_from_calib_frame = np.dot(medmodel_intrinsics,
-                                         get_view_frame_from_calib_frame(0, 0, 0, 0))
+  get_view_frame_from_road_frame(0, 0, 0, model_height))

 model_frame_from_bigmodel_frame = np.dot(model_intrinsics, np.linalg.inv(bigmodel_intrinsics))
 medmodel_frame_from_bigmodel_frame = np.dot(medmodel_intrinsics, np.linalg.inv(bigmodel_intrinsics))

-
 # 'camera from model camera'
 def get_model_height_transform(camera_frame_from_road_frame, height):
  camera_frame_from_road_ground = np.dot(camera_frame_from_road_frame, np.array([
-    [1, 0, 0],
-    [0, 1, 0],
-    [0, 0, 0],
-    [0, 0, 1],
+      [1, 0, 0],
+      [0, 1, 0],
+      [0, 0, 0],
+      [0, 0, 1],
  ]))

  camera_frame_from_road_high = np.dot(camera_frame_from_road_frame, np.array([
-    [1, 0, 0],
-    [0, 1, 0],
-    [0, 0, height - model_height],
-    [0, 0, 1],
+      [1, 0, 0],
+      [0, 1, 0],
+      [0, 0, height - model_height],
+      [0, 0, 1],
  ]))

  road_high_from_camera_frame = np.linalg.inv(camera_frame_from_road_high)
@@ -115,19 +89,18 @@ def get_model_height_transform(camera_frame_from_road_frame, height):

 # camera_frame_from_model_frame aka 'warp matrix'
 # was: calibration.h/CalibrationTransform
-def get_camera_frame_from_model_frame(camera_frame_from_road_frame, height=model_height, camera_fl=FOCAL):
+def get_camera_frame_from_model_frame(camera_frame_from_road_frame, height=model_height):
  vp = vp_from_ke(camera_frame_from_road_frame)

-  model_zoom = camera_fl / model_fl
  model_camera_from_model_frame = np.array([
-    [model_zoom,  0.0,  vp[0] - MODEL_CX * model_zoom],
-    [0.0,  model_zoom,  vp[1] - MODEL_CY * model_zoom],
-    [0.0,  0.0,                                   1.0],
+    [model_zoom,         0., vp[0] - MODEL_CX * model_zoom],
+    [        0., model_zoom, vp[1] - MODEL_CY * model_zoom],
+    [        0.,         0.,                            1.],
  ])

  # This function is super slow, so skip it if height is very close to canonical
  # TODO: speed it up!
-  if abs(height - model_height) > 0.001:
+  if abs(height - model_height) > 0.001: #
    camera_from_model_camera = get_model_height_transform(camera_frame_from_road_frame, height)
  else:
    camera_from_model_camera = np.eye(3)
@@ -157,9 +130,9 @@ def get_camera_frame_from_bigmodel_frame(camera_frame_from_road_frame):

 def get_model_frame(snu_full, camera_frame_from_model_frame, size):
  idxs = camera_frame_from_model_frame.dot(np.column_stack([np.tile(np.arange(size[0]), size[1]),
-                                                            np.tile(np.arange(size[1]), (size[0], 1)).T.flatten(),
+                                                            np.tile(np.arange(size[1]), (size[0],1)).T.flatten(),
                                                            np.ones(size[0] * size[1])]).T).T.astype(int)
-  calib_flat = snu_full[idxs[:, 1], idxs[:, 0]]
+  calib_flat = snu_full[idxs[:,1], idxs[:,0]]
  if len(snu_full.shape) == 3:
    calib = calib_flat.reshape((size[1], size[0], 3))
  elif len(snu_full.shape) == 2:
@@ -1,147 +0,0 @@
-#define _USE_MATH_DEFINES
-
-#include <iostream>
-#include <cmath>
-#include <eigen3/Eigen/Dense>
-
-#include "orientation.hpp"
-#include "coordinates.hpp"
-
-Eigen::Quaterniond ensure_unique(Eigen::Quaterniond quat){
-  if (quat.w() > 0){
-    return quat;
-  } else {
-    return Eigen::Quaterniond(-quat.w(), -quat.x(), -quat.y(), -quat.z());
-  }
-}
-
-Eigen::Quaterniond euler2quat(Eigen::Vector3d euler){
-  Eigen::Quaterniond q;
-
-  q = Eigen::AngleAxisd(euler(2), Eigen::Vector3d::UnitZ())
-    * Eigen::AngleAxisd(euler(1), Eigen::Vector3d::UnitY())
-    * Eigen::AngleAxisd(euler(0), Eigen::Vector3d::UnitX());
-  return ensure_unique(q);
-}
-
-
-Eigen::Vector3d quat2euler(Eigen::Quaterniond quat){
-  // TODO: switch to eigen implementation if the range of the Euler angles doesn't matter anymore
-  // Eigen::Vector3d euler = quat.toRotationMatrix().eulerAngles(2, 1, 0);
-  // return {euler(2), euler(1), euler(0)};
-  double gamma = atan2(2 * (quat.w() * quat.x() + quat.y() * quat.z()), 1 - 2 * (quat.x()*quat.x() + quat.y()*quat.y()));
-  double theta = asin(2 * (quat.w() * quat.y() - quat.z() * quat.x()));
-  double psi = atan2(2 * (quat.w() * quat.z() + quat.x() * quat.y()), 1 - 2 * (quat.y()*quat.y() + quat.z()*quat.z()));
-  return {gamma, theta, psi};
-}
-
-Eigen::Matrix3d quat2rot(Eigen::Quaterniond quat){
-  return quat.toRotationMatrix();
-}
-
-Eigen::Quaterniond rot2quat(Eigen::Matrix3d rot){
-  return ensure_unique(Eigen::Quaterniond(rot));
-}
-
-Eigen::Matrix3d euler2rot(Eigen::Vector3d euler){
-  return quat2rot(euler2quat(euler));
-}
-
-Eigen::Vector3d rot2euler(Eigen::Matrix3d rot){
-  return quat2euler(rot2quat(rot));
-}
-
-Eigen::Matrix3d rot_matrix(double roll, double pitch, double yaw){
-  return euler2rot({roll, pitch, yaw});
-}
-
-Eigen::Matrix3d rot(Eigen::Vector3d axis, double angle){
-  Eigen::Quaterniond q;
-  q = Eigen::AngleAxisd(angle, axis);
-  return q.toRotationMatrix();
-}
-
-
-Eigen::Vector3d ecef_euler_from_ned(ECEF ecef_init, Eigen::Vector3d ned_pose) {
-  /*
-    Using Rotations to Build Aerospace Coordinate Systems
-    Don Koks
-    https://apps.dtic.mil/dtic/tr/fulltext/u2/a484864.pdf
-  */
-  LocalCoord converter = LocalCoord(ecef_init);
-  Eigen::Vector3d zero = ecef_init.to_vector();
-
-  Eigen::Vector3d x0 = converter.ned2ecef({1, 0, 0}).to_vector() - zero;
-  Eigen::Vector3d y0 = converter.ned2ecef({0, 1, 0}).to_vector() - zero;
-  Eigen::Vector3d z0 = converter.ned2ecef({0, 0, 1}).to_vector() - zero;
-
-  Eigen::Vector3d x1 = rot(z0, ned_pose(2)) * x0;
-  Eigen::Vector3d y1 = rot(z0, ned_pose(2)) * y0;
-  Eigen::Vector3d z1 = rot(z0, ned_pose(2)) * z0;
-
-  Eigen::Vector3d x2 = rot(y1, ned_pose(1)) * x1;
-  Eigen::Vector3d y2 = rot(y1, ned_pose(1)) * y1;
-  Eigen::Vector3d z2 = rot(y1, ned_pose(1)) * z1;
-
-  Eigen::Vector3d x3 = rot(x2, ned_pose(0)) * x2;
-  Eigen::Vector3d y3 = rot(x2, ned_pose(0)) * y2;
-
-
-  x0 = Eigen::Vector3d(1, 0, 0);
-  y0 = Eigen::Vector3d(0, 1, 0);
-  z0 = Eigen::Vector3d(0, 0, 1);
-
-  double psi = atan2(x3.dot(y0), x3.dot(x0));
-  double theta = atan2(-x3.dot(z0), sqrt(pow(x3.dot(x0), 2) + pow(x3.dot(y0), 2)));
-
-  y2 = rot(z0, psi) * y0;
-  z2 = rot(y2, theta) * z0;
-
-  double phi = atan2(y3.dot(z2), y3.dot(y2));
-
-  return {phi, theta, psi};
-}
-
-Eigen::Vector3d ned_euler_from_ecef(ECEF ecef_init, Eigen::Vector3d ecef_pose){
-  /*
-    Using Rotations to Build Aerospace Coordinate Systems
-    Don Koks
-    https://apps.dtic.mil/dtic/tr/fulltext/u2/a484864.pdf
-  */
-  LocalCoord converter = LocalCoord(ecef_init);
-
-  Eigen::Vector3d x0 = Eigen::Vector3d(1, 0, 0);
-  Eigen::Vector3d y0 = Eigen::Vector3d(0, 1, 0);
-  Eigen::Vector3d z0 = Eigen::Vector3d(0, 0, 1);
-
-  Eigen::Vector3d x1 = rot(z0, ecef_pose(2)) * x0;
-  Eigen::Vector3d y1 = rot(z0, ecef_pose(2)) * y0;
-  Eigen::Vector3d z1 = rot(z0, ecef_pose(2)) * z0;
-
-  Eigen::Vector3d x2 = rot(y1, ecef_pose(1)) * x1;
-  Eigen::Vector3d y2 = rot(y1, ecef_pose(1)) * y1;
-  Eigen::Vector3d z2 = rot(y1, ecef_pose(1)) * z1;
-
-  Eigen::Vector3d x3 = rot(x2, ecef_pose(0)) * x2;
-  Eigen::Vector3d y3 = rot(x2, ecef_pose(0)) * y2;
-
-  Eigen::Vector3d zero = ecef_init.to_vector();
-  x0 = converter.ned2ecef({1, 0, 0}).to_vector() - zero;
-  y0 = converter.ned2ecef({0, 1, 0}).to_vector() - zero;
-  z0 = converter.ned2ecef({0, 0, 1}).to_vector() - zero;
-
-  double psi = atan2(x3.dot(y0), x3.dot(x0));
-  double theta = atan2(-x3.dot(z0), sqrt(pow(x3.dot(x0), 2) + pow(x3.dot(y0), 2)));
-
-  y2 = rot(z0, psi) * y0;
-  z2 = rot(y2, theta) * z0;
-
-  double phi = atan2(y3.dot(z2), y3.dot(y2));
-
-  return {phi, theta, psi};
-}
-
-
-
-int main(void){
-}
@@ -1,17 +0,0 @@
-#pragma once
-#include <eigen3/Eigen/Dense>
-#include "coordinates.hpp"
-
-
-Eigen::Quaterniond ensure_unique(Eigen::Quaterniond quat);
-
-Eigen::Quaterniond euler2quat(Eigen::Vector3d euler);
-Eigen::Vector3d quat2euler(Eigen::Quaterniond quat);
-Eigen::Matrix3d quat2rot(Eigen::Quaterniond quat);
-Eigen::Quaterniond rot2quat(Eigen::Matrix3d rot);
-Eigen::Matrix3d euler2rot(Eigen::Vector3d euler);
-Eigen::Vector3d rot2euler(Eigen::Matrix3d rot);
-Eigen::Matrix3d rot_matrix(double roll, double pitch, double yaw);
-Eigen::Matrix3d rot(Eigen::Vector3d axis, double angle);
-Eigen::Vector3d ecef_euler_from_ned(ECEF ecef_init, Eigen::Vector3d ned_pose);
-Eigen::Vector3d ned_euler_from_ecef(ECEF ecef_init, Eigen::Vector3d ecef_pose);
@@ -1,52 +1,295 @@
-# pylint: skip-file
 import numpy as np
-
-from common.transformations.transformations import (ecef_euler_from_ned_single,
-                                                    euler2quat_single,
-                                                    euler2rot_single,
-                                                    ned_euler_from_ecef_single,
-                                                    quat2euler_single,
-                                                    quat2rot_single,
-                                                    rot2euler_single,
-                                                    rot2quat_single)
+from numpy import dot, inner, array, linalg
+from common.transformations.coordinates import LocalCoord


-def numpy_wrap(function, input_shape, output_shape):
-  """Wrap a function to take either an input or list of inputs and return the correct shape"""
-  def f(*inps):
-    *args, inp = inps
-    inp = np.array(inp)
-    shape = inp.shape
+'''
+Vectorized functions that transform between
+rotation matrices, euler angles and quaternions.
+All support lists, array or array of arrays as inputs.
+Supports both x2y and y_from_x format (y_from_x preferred!).
+'''

-    if len(shape) == len(input_shape):
-      out_shape = output_shape
-    else:
-      out_shape = (shape[0],) + output_shape
+def euler2quat(eulers):
+  eulers = array(eulers)
+  if len(eulers.shape) > 1:
+    output_shape = (-1,4)
+  else:
+    output_shape = (4,)
+  eulers = np.atleast_2d(eulers)
+  gamma, theta, psi = eulers[:,0],  eulers[:,1],  eulers[:,2]

-    # Add empty dimension if inputs is not a list
-    if len(shape) == len(input_shape):
-      inp.shape = (1, ) + inp.shape
+  q0 = np.cos(gamma / 2) * np.cos(theta / 2) * np.cos(psi / 2) + \
+       np.sin(gamma / 2) * np.sin(theta / 2) * np.sin(psi / 2)
+  q1 = np.sin(gamma / 2) * np.cos(theta / 2) * np.cos(psi / 2) - \
+       np.cos(gamma / 2) * np.sin(theta / 2) * np.sin(psi / 2)
+  q2 = np.cos(gamma / 2) * np.sin(theta / 2) * np.cos(psi / 2) + \
+       np.sin(gamma / 2) * np.cos(theta / 2) * np.sin(psi / 2)
+  q3 = np.cos(gamma / 2) * np.cos(theta / 2) * np.sin(psi / 2) - \
+       np.sin(gamma / 2) * np.sin(theta / 2) * np.cos(psi / 2)

-    result = np.asarray([function(*args, i) for i in inp])
-    result.shape = out_shape
-    return result
-  return f
+  quats = array([q0, q1, q2, q3]).T
+  for i in range(len(quats)):
+    if quats[i,0] < 0:
+      quats[i] = -quats[i]
+  return quats.reshape(output_shape)


-euler2quat = numpy_wrap(euler2quat_single, (3,), (4,))
-quat2euler = numpy_wrap(quat2euler_single, (4,), (3,))
-quat2rot = numpy_wrap(quat2rot_single, (4,), (3, 3))
-rot2quat = numpy_wrap(rot2quat_single, (3, 3), (4,))
-euler2rot = numpy_wrap(euler2rot_single, (3,), (3, 3))
-rot2euler = numpy_wrap(rot2euler_single, (3, 3), (3,))
-ecef_euler_from_ned = numpy_wrap(ecef_euler_from_ned_single, (3,), (3,))
-ned_euler_from_ecef = numpy_wrap(ned_euler_from_ecef_single, (3,), (3,))
+def quat2euler(quats):
+  quats = array(quats)
+  if len(quats.shape) > 1:
+    output_shape = (-1,3)
+  else:
+    output_shape = (3,)
+  quats = np.atleast_2d(quats)
+  q0, q1, q2, q3 = quats[:,0], quats[:,1], quats[:,2], quats[:,3]
+
+  gamma = np.arctan2(2 * (q0 * q1 + q2 * q3), 1 - 2 * (q1**2 + q2**2))
+  theta = np.arcsin(2 * (q0 * q2 - q3 * q1))
+  psi = np.arctan2(2 * (q0 * q3 + q1 * q2), 1 - 2 * (q2**2 + q3**2))
+
+  eulers = array([gamma, theta, psi]).T
+  return eulers.reshape(output_shape)
+
+
+def quat2rot(quats):
+  quats = array(quats)
+  input_shape = quats.shape
+  quats = np.atleast_2d(quats)
+  Rs = np.zeros((quats.shape[0], 3, 3))
+  q0 = quats[:, 0]
+  q1 = quats[:, 1]
+  q2 = quats[:, 2]
+  q3 = quats[:, 3]
+  Rs[:, 0, 0] = q0 * q0 + q1 * q1 - q2 * q2 - q3 * q3
+  Rs[:, 0, 1] = 2 * (q1 * q2 - q0 * q3)
+  Rs[:, 0, 2] = 2 * (q0 * q2 + q1 * q3)
+  Rs[:, 1, 0] = 2 * (q1 * q2 + q0 * q3)
+  Rs[:, 1, 1] = q0 * q0 - q1 * q1 + q2 * q2 - q3 * q3
+  Rs[:, 1, 2] = 2 * (q2 * q3 - q0 * q1)
+  Rs[:, 2, 0] = 2 * (q1 * q3 - q0 * q2)
+  Rs[:, 2, 1] = 2 * (q0 * q1 + q2 * q3)
+  Rs[:, 2, 2] = q0 * q0 - q1 * q1 - q2 * q2 + q3 * q3
+
+  if len(input_shape) < 2:
+    return Rs[0]
+  else:
+    return Rs
+
+
+def rot2quat(rots):
+  input_shape = rots.shape
+  if len(input_shape) < 3:
+    rots = array([rots])
+  K3 = np.empty((len(rots), 4, 4))
+  K3[:, 0, 0] = (rots[:, 0, 0] - rots[:, 1, 1] - rots[:, 2, 2]) / 3.0
+  K3[:, 0, 1] = (rots[:, 1, 0] + rots[:, 0, 1]) / 3.0
+  K3[:, 0, 2] = (rots[:, 2, 0] + rots[:, 0, 2]) / 3.0
+  K3[:, 0, 3] = (rots[:, 1, 2] - rots[:, 2, 1]) / 3.0
+  K3[:, 1, 0] = K3[:, 0, 1]
+  K3[:, 1, 1] = (rots[:, 1, 1] - rots[:, 0, 0] - rots[:, 2, 2]) / 3.0
+  K3[:, 1, 2] = (rots[:, 2, 1] + rots[:, 1, 2]) / 3.0
+  K3[:, 1, 3] = (rots[:, 2, 0] - rots[:, 0, 2]) / 3.0
+  K3[:, 2, 0] = K3[:, 0, 2]
+  K3[:, 2, 1] = K3[:, 1, 2]
+  K3[:, 2, 2] = (rots[:, 2, 2] - rots[:, 0, 0] - rots[:, 1, 1]) / 3.0
+  K3[:, 2, 3] = (rots[:, 0, 1] - rots[:, 1, 0]) / 3.0
+  K3[:, 3, 0] = K3[:, 0, 3]
+  K3[:, 3, 1] = K3[:, 1, 3]
+  K3[:, 3, 2] = K3[:, 2, 3]
+  K3[:, 3, 3] = (rots[:, 0, 0] + rots[:, 1, 1] + rots[:, 2, 2]) / 3.0
+  q = np.empty((len(rots), 4))
+  for i in range(len(rots)):
+    _, eigvecs = linalg.eigh(K3[i].T)
+    eigvecs = eigvecs[:,3:]
+    q[i, 0] = eigvecs[-1]
+    q[i, 1:] = -eigvecs[:-1].flatten()
+    if q[i, 0] < 0:
+      q[i] = -q[i]
+
+  if len(input_shape) < 3:
+    return q[0]
+  else:
+    return q
+
+
+def euler2rot(eulers):
+  return rotations_from_quats(euler2quat(eulers))
+
+
+def rot2euler(rots):
+  return quat2euler(quats_from_rotations(rots))
+

 quats_from_rotations = rot2quat
 quat_from_rot = rot2quat
 rotations_from_quats = quat2rot
-rot_from_quat = quat2rot
+rot_from_quat= quat2rot
+rot_from_quat= quat2rot
 euler_from_rot = rot2euler
 euler_from_quat = quat2euler
 rot_from_euler = euler2rot
 quat_from_euler = euler2quat
+
+
+
+
+
+
+'''
+Random helpers below
+'''
+
+
+def quat_product(q, r):
+  t = np.zeros(4)
+  t[0] = r[0] * q[0] - r[1] * q[1] - r[2] * q[2] - r[3] * q[3]
+  t[1] = r[0] * q[1] + r[1] * q[0] - r[2] * q[3] + r[3] * q[2]
+  t[2] = r[0] * q[2] + r[1] * q[3] + r[2] * q[0] - r[3] * q[1]
+  t[3] = r[0] * q[3] - r[1] * q[2] + r[2] * q[1] + r[3] * q[0]
+  return t
+
+
+def rot_matrix(roll, pitch, yaw):
+  cr, sr = np.cos(roll), np.sin(roll)
+  cp, sp = np.cos(pitch), np.sin(pitch)
+  cy, sy = np.cos(yaw), np.sin(yaw)
+  rr = array([[1,0,0],[0, cr,-sr],[0, sr, cr]])
+  rp = array([[cp,0,sp],[0, 1,0],[-sp, 0, cp]])
+  ry = array([[cy,-sy,0],[sy, cy,0],[0, 0, 1]])
+  return ry.dot(rp.dot(rr))
+
+
+def rot(axis, angle):
+  # Rotates around an arbitrary axis
+  ret_1 = (1 - np.cos(angle)) * array([[axis[0]**2, axis[0] * axis[1], axis[0] * axis[2]], [
+    axis[1] * axis[0], axis[1]**2, axis[1] * axis[2]
+  ], [axis[2] * axis[0], axis[2] * axis[1], axis[2]**2]])
+  ret_2 = np.cos(angle) * np.eye(3)
+  ret_3 = np.sin(angle) * array([[0, -axis[2], axis[1]], [axis[2], 0, -axis[0]],
+                              [-axis[1], axis[0], 0]])
+  return ret_1 + ret_2 + ret_3
+
+
+def ecef_euler_from_ned(ned_ecef_init, ned_pose):
+  '''
+  Got it from here:
+  Using Rotations to Build Aerospace Coordinate Systems
+  -Don Koks
+  '''
+  converter = LocalCoord.from_ecef(ned_ecef_init)
+  x0 = converter.ned2ecef([1, 0, 0]) - converter.ned2ecef([0, 0, 0])
+  y0 = converter.ned2ecef([0, 1, 0]) - converter.ned2ecef([0, 0, 0])
+  z0 = converter.ned2ecef([0, 0, 1]) - converter.ned2ecef([0, 0, 0])
+
+  x1 = rot(z0, ned_pose[2]).dot(x0)
+  y1 = rot(z0, ned_pose[2]).dot(y0)
+  z1 = rot(z0, ned_pose[2]).dot(z0)
+
+  x2 = rot(y1, ned_pose[1]).dot(x1)
+  y2 = rot(y1, ned_pose[1]).dot(y1)
+  z2 = rot(y1, ned_pose[1]).dot(z1)
+
+  x3 = rot(x2, ned_pose[0]).dot(x2)
+  y3 = rot(x2, ned_pose[0]).dot(y2)
+  #z3 = rot(x2, ned_pose[0]).dot(z2)
+
+  x0 = array([1, 0, 0])
+  y0 = array([0, 1, 0])
+  z0 = array([0, 0, 1])
+
+  psi = np.arctan2(inner(x3, y0), inner(x3, x0))
+  theta = np.arctan2(-inner(x3, z0), np.sqrt(inner(x3, x0)**2 + inner(x3, y0)**2))
+  y2 = rot(z0, psi).dot(y0)
+  z2 = rot(y2, theta).dot(z0)
+  phi = np.arctan2(inner(y3, z2), inner(y3, y2))
+
+  ret = array([phi, theta, psi])
+  return ret
+
+
+def ned_euler_from_ecef(ned_ecef_init, ecef_poses):
+  '''
+  Got the math from here:
+  Using Rotations to Build Aerospace Coordinate Systems
+  -Don Koks
+
+  Also accepts array of ecef_poses and array of ned_ecef_inits.
+  Where each row is a pose and an ecef_init.
+  '''
+  ned_ecef_init = array(ned_ecef_init)
+  ecef_poses = array(ecef_poses)
+  output_shape = ecef_poses.shape
+  ned_ecef_init = np.atleast_2d(ned_ecef_init)
+  if ned_ecef_init.shape[0] == 1:
+    ned_ecef_init = np.tile(ned_ecef_init[0], (output_shape[0], 1))
+  ecef_poses = np.atleast_2d(ecef_poses)
+
+  ned_poses = np.zeros(ecef_poses.shape)
+  for i, ecef_pose in enumerate(ecef_poses):
+    converter = LocalCoord.from_ecef(ned_ecef_init[i])
+    x0 = array([1, 0, 0])
+    y0 = array([0, 1, 0])
+    z0 = array([0, 0, 1])
+
+    x1 = rot(z0, ecef_pose[2]).dot(x0)
+    y1 = rot(z0, ecef_pose[2]).dot(y0)
+    z1 = rot(z0, ecef_pose[2]).dot(z0)
+
+    x2 = rot(y1, ecef_pose[1]).dot(x1)
+    y2 = rot(y1, ecef_pose[1]).dot(y1)
+    z2 = rot(y1, ecef_pose[1]).dot(z1)
+
+    x3 = rot(x2, ecef_pose[0]).dot(x2)
+    y3 = rot(x2, ecef_pose[0]).dot(y2)
+    #z3 = rot(x2, ecef_pose[0]).dot(z2)
+
+    x0 = converter.ned2ecef([1, 0, 0]) - converter.ned2ecef([0, 0, 0])
+    y0 = converter.ned2ecef([0, 1, 0]) - converter.ned2ecef([0, 0, 0])
+    z0 = converter.ned2ecef([0, 0, 1]) - converter.ned2ecef([0, 0, 0])
+
+    psi = np.arctan2(inner(x3, y0), inner(x3, x0))
+    theta = np.arctan2(-inner(x3, z0), np.sqrt(inner(x3, x0)**2 + inner(x3, y0)**2))
+    y2 = rot(z0, psi).dot(y0)
+    z2 = rot(y2, theta).dot(z0)
+    phi = np.arctan2(inner(y3, z2), inner(y3, y2))
+    ned_poses[i] = array([phi, theta, psi])
+
+  return ned_poses.reshape(output_shape)
+
+
+def ecef2car(car_ecef, psi, theta, points_ecef, ned_converter):
+  """
+  TODO: add roll rotation
+  Converts an array of points in ecef coordinates into
+  x-forward, y-left, z-up coordinates
+  Parameters
+  ----------
+  psi: yaw, radian
+  theta: pitch, radian
+  Returns
+  -------
+  [x, y, z] coordinates in car frame
+  """
+
+  # input is an array of points in ecef cocrdinates
+  # output is an array of points in car's coordinate (x-front, y-left, z-up)
+
+  # convert points to NED
+  points_ned = []
+  for p in points_ecef:
+    points_ned.append(ned_converter.ecef2ned_matrix.dot(array(p) - car_ecef))
+
+  points_ned = np.vstack(points_ned).T
+
+  # n, e, d -> x, y, z
+  # Calculate relative postions and rotate wrt to heading and pitch of car
+  invert_R = array([[1., 0., 0.], [0., -1., 0.], [0., 0., -1.]])
+
+  c, s = np.cos(psi), np.sin(psi)
+  yaw_R = array([[c, s, 0.], [-s, c, 0.], [0., 0., 1.]])
+
+  c, s = np.cos(theta), np.sin(theta)
+  pitch_R = array([[c, 0., -s], [0., 1., 0.], [s, 0., c]])
+
+  return dot(pitch_R, dot(yaw_R, dot(invert_R, points_ned)))
@@ -1,20 +0,0 @@
-import numpy
-
-from Cython.Build import cythonize
-from distutils.core import Extension, setup  # pylint: disable=import-error,no-name-in-module
-from common.cython_hacks import BuildExtWithoutPlatformSuffix
-
-setup(
-  name='Cython transformations wrapper',
-  cmdclass={'build_ext': BuildExtWithoutPlatformSuffix},
-  ext_modules=cythonize(
-    Extension(
-      "transformations",
-      sources=["transformations.pyx"],
-      language="c++",
-      extra_compile_args=["-std=c++1z", "-Wno-cpp"],
-      include_dirs=[numpy.get_include()],
-    ),
-    nthreads=4,
-  )
-)
@@ -1,72 +0,0 @@
-#cython: language_level=3
-from libcpp cimport bool
-
-cdef extern from "orientation.cc":
-  pass
-
-cdef extern from "orientation.hpp":
-  cdef cppclass Quaternion "Eigen::Quaterniond":
-    Quaternion()
-    Quaternion(double, double, double, double)
-    double w()
-    double x()
-    double y()
-    double z()
-
-  cdef cppclass Vector3 "Eigen::Vector3d":
-    Vector3()
-    Vector3(double, double, double)
-    double operator()(int)
-
-  cdef cppclass Matrix3 "Eigen::Matrix3d":
-    Matrix3()
-    Matrix3(double*)
-
-    double operator()(int, int)
-
-  Quaternion euler2quat(Vector3)
-  Vector3 quat2euler(Quaternion)
-  Matrix3 quat2rot(Quaternion)
-  Quaternion rot2quat(Matrix3)
-  Vector3 rot2euler(Matrix3)
-  Matrix3 euler2rot(Vector3)
-  Matrix3 rot_matrix(double, double, double)
-  Vector3 ecef_euler_from_ned(ECEF, Vector3)
-  Vector3 ned_euler_from_ecef(ECEF, Vector3)
-
-
-cdef extern from "coordinates.cc":
-  cdef struct ECEF:
-    double x
-    double y
-    double z
-
-  cdef struct NED:
-    double n
-    double e
-    double d
-
-  cdef struct Geodetic:
-    double lat
-    double lon
-    double alt
-    bool radians
-
-  ECEF geodetic2ecef(Geodetic)
-  Geodetic ecef2geodetic(ECEF)
-
-  cdef cppclass LocalCoord_c "LocalCoord":
-    Matrix3 ned2ecef_matrix
-    Matrix3 ecef2ned_matrix
-
-    LocalCoord_c(Geodetic, ECEF)
-    LocalCoord_c(Geodetic)
-    LocalCoord_c(ECEF)
-
-    NED ecef2ned(ECEF)
-    ECEF ned2ecef(NED)
-    NED geodetic2ned(Geodetic)
-    Geodetic ned2geodetic(NED)
-
-cdef extern from "coordinates.hpp":
-  pass
@@ -1,172 +0,0 @@
-from transformations cimport Matrix3, Vector3, Quaternion
-from transformations cimport ECEF, NED, Geodetic
-
-from transformations cimport euler2quat as euler2quat_c
-from transformations cimport quat2euler as quat2euler_c
-from transformations cimport quat2rot as quat2rot_c
-from transformations cimport rot2quat as rot2quat_c
-from transformations cimport euler2rot as euler2rot_c
-from transformations cimport rot2euler as rot2euler_c
-from transformations cimport rot_matrix as rot_matrix_c
-from transformations cimport ecef_euler_from_ned as ecef_euler_from_ned_c
-from transformations cimport ned_euler_from_ecef as ned_euler_from_ecef_c
-from transformations cimport geodetic2ecef as geodetic2ecef_c
-from transformations cimport ecef2geodetic as ecef2geodetic_c
-from transformations cimport LocalCoord_c
-
-
-import cython
-import numpy as np
-cimport numpy as np
-
-cdef np.ndarray[double, ndim=2] matrix2numpy(Matrix3 m):
-    return np.array([
-        [m(0, 0), m(0, 1), m(0, 2)],
-        [m(1, 0), m(1, 1), m(1, 2)],
-        [m(2, 0), m(2, 1), m(2, 2)],
-    ])
-
-cdef Matrix3 numpy2matrix(np.ndarray[double, ndim=2, mode="fortran"] m):
-    assert m.shape[0] == 3
-    assert m.shape[1] == 3
-    return Matrix3(<double*>m.data)
-
-cdef ECEF list2ecef(ecef):
-    cdef ECEF e;
-    e.x = ecef[0]
-    e.y = ecef[1]
-    e.z = ecef[2]
-    return e
-
-cdef NED list2ned(ned):
-    cdef NED n;
-    n.n = ned[0]
-    n.e = ned[1]
-    n.d = ned[2]
-    return n
-
-cdef Geodetic list2geodetic(geodetic):
-    cdef Geodetic g
-    g.lat = geodetic[0]
-    g.lon = geodetic[1]
-    g.alt = geodetic[2]
-    return g
-
-def euler2quat_single(euler):
-    cdef Vector3 e = Vector3(euler[0], euler[1], euler[2])
-    cdef Quaternion q = euler2quat_c(e)
-    return [q.w(), q.x(), q.y(), q.z()]
-
-def quat2euler_single(quat):
-    cdef Quaternion q = Quaternion(quat[0], quat[1], quat[2], quat[3])
-    cdef Vector3 e = quat2euler_c(q);
-    return [e(0), e(1), e(2)]
-
-def quat2rot_single(quat):
-    cdef Quaternion q = Quaternion(quat[0], quat[1], quat[2], quat[3])
-    cdef Matrix3 r = quat2rot_c(q)
-    return matrix2numpy(r)
-
-def rot2quat_single(rot):
-    cdef Matrix3 r = numpy2matrix(np.asfortranarray(rot, dtype=np.double))
-    cdef Quaternion q = rot2quat_c(r)
-    return [q.w(), q.x(), q.y(), q.z()]
-
-def euler2rot_single(euler):
-    cdef Vector3 e = Vector3(euler[0], euler[1], euler[2])
-    cdef Matrix3 r = euler2rot_c(e)
-    return matrix2numpy(r)
-
-def rot2euler_single(rot):
-    cdef Matrix3 r = numpy2matrix(np.asfortranarray(rot, dtype=np.double))
-    cdef Vector3 e = rot2euler_c(r)
-    return [e(0), e(1), e(2)]
-
-def rot_matrix(roll, pitch, yaw):
-    return matrix2numpy(rot_matrix_c(roll, pitch, yaw))
-
-def ecef_euler_from_ned_single(ecef_init, ned_pose):
-    cdef ECEF init = list2ecef(ecef_init)
-    cdef Vector3 pose = Vector3(ned_pose[0], ned_pose[1], ned_pose[2])
-
-    cdef Vector3 e = ecef_euler_from_ned_c(init, pose)
-    return [e(0), e(1), e(2)]
-
-def ned_euler_from_ecef_single(ecef_init, ecef_pose):
-    cdef ECEF init = list2ecef(ecef_init)
-    cdef Vector3 pose = Vector3(ecef_pose[0], ecef_pose[1], ecef_pose[2])
-
-    cdef Vector3 e = ned_euler_from_ecef_c(init, pose)
-    return [e(0), e(1), e(2)]
-
-def geodetic2ecef_single(geodetic):
-    cdef Geodetic g = list2geodetic(geodetic)
-    cdef ECEF e = geodetic2ecef_c(g)
-    return [e.x, e.y, e.z]
-
-def ecef2geodetic_single(ecef):
-    cdef ECEF e = list2ecef(ecef)
-    cdef Geodetic g = ecef2geodetic_c(e)
-    return [g.lat, g.lon, g.alt]
-
-
-cdef class LocalCoord:
-    cdef LocalCoord_c * lc
-
-    def __init__(self, geodetic=None, ecef=None):
-        assert (geodetic is not None) or (ecef is not None)
-        if geodetic is not None:
-            self.lc = new LocalCoord_c(list2geodetic(geodetic))
-        elif ecef is not None:
-            self.lc = new LocalCoord_c(list2ecef(ecef))
-
-    @property
-    def ned2ecef_matrix(self):
-        return matrix2numpy(self.lc.ned2ecef_matrix)
-
-    @property
-    def ecef2ned_matrix(self):
-        return matrix2numpy(self.lc.ecef2ned_matrix)
-
-    @property
-    def ned_from_ecef_matrix(self):
-        return self.ecef2ned_matrix
-
-    @property
-    def ecef_from_ned_matrix(self):
-        return self.ned2ecef_matrix
-
-    @classmethod
-    def from_geodetic(cls, geodetic):
-        return cls(geodetic=geodetic)
-
-    @classmethod
-    def from_ecef(cls, ecef):
-        return cls(ecef=ecef)
-
-    def ecef2ned_single(self, ecef):
-        assert self.lc
-        cdef ECEF e = list2ecef(ecef)
-        cdef NED n = self.lc.ecef2ned(e)
-        return [n.n, n.e, n.d]
-
-    def ned2ecef_single(self, ned):
-        assert self.lc
-        cdef NED n = list2ned(ned)
-        cdef ECEF e = self.lc.ned2ecef(n)
-        return [e.x, e.y, e.z]
-
-    def geodetic2ned_single(self, geodetic):
-        assert self.lc
-        cdef Geodetic g = list2geodetic(geodetic)
-        cdef NED n = self.lc.geodetic2ned(g)
-        return [n.n, n.e, n.d]
-
-    def ned2geodetic_single(self, ned):
-        assert self.lc
-        cdef NED n = list2ned(ned)
-        cdef Geodetic g = self.lc.ned2geodetic(n)
-        return [g.lat, g.lon, g.alt]
-
-    def __dealloc__(self):
-        del self.lc
@@ -1,56 +0,0 @@
-Make a python script translatable
--
-1. add following codes to the top
-```python
-# This Python file uses the following encoding: utf-8
-# -*- coding: utf-8 -*-
-from common.i18n import events
-_ = events()
-``` 
-2. wrap ```_()``` function around the string.
-
-3. generate pot file (template file)
-```bash
-# e.g. xgettext -d base -o /data/openpilot/selfdrive/assets/locales/events.pot /data/openpilot/selfdrive/controls/lib/events.py
-xgettext -d base -o <openpilot_path>/selfdrive/assets/locales/<template_file>.pot <file_to_translate>.py
-```
-
-4. add po file to languages
-```bash
-# e.g. cp /data/openpilot/selfdrive/assets/locales/events.pot /data/openpilot/selfdrive/assets/locales/zh-TW/LC_MESSAGES/events.po
-cp <openpilot_path>/selfdrive/assets/locales/<template_file>.pot <openpilot_path>/selfdrive/assets/locales/<language>/LC_MESSAGES/<template_file>.po
-```
-
-5. translate po file with your favorite editor.
-
-6. generate mo file.
-```bash
-# e.g. msgfmt -o /data/openpilot/selfdrive/assets/locales/zh-TW/LC_MESSAGES/events.mo /data/openpilot/selfdrive/assets/locales/zh-TW/LC_MESSAGES/events
-msgfmt -o <openpilot_path>/selfdrive/assets/locales/<language>/LC_MESSAGES/<template_file>.mo <openpilot_path>/selfdrive/assets/locales/<language>/LC_MESSAGES/<template_file>
-```
-
-Update translations
---
-1. add ```_()``` around the new strings
-
-2. generate pot file again (template file)
-```bash
-# e.g. xgettext -d base -o /data/openpilot/selfdrive/assets/locales/events.pot /data/openpilot/selfdrive/controls/lib/events.py
-xgettext -d base -o <openpilot_path>/selfdrive/assets/locales/<template_file>.pot <file_to_translate>.py
-```
-
-3. update already translated file (merge)
-```bash
-# e.g. msgmerge --update /data/openpilot/selfdrive/assets/locales/zh-TW/LC_MESSAGES/events.po /data/openpilot/selfdrive/assets/locales/events.pot
-msgmerge --update <openpilot_path>/selfdrive/assets/locales/<language>/LC_MESSAGES/<file_to_translate>.po <openpilot_path>/selfdrive/assets/locales/<template_file>.pot
-```
-
-4. generate mo file again.
-```bash
-# e.g. msgfmt -o /data/openpilot/selfdrive/assets/locales/zh-TW/LC_MESSAGES/events.mo /data/openpilot/selfdrive/assets/locales/zh-TW/LC_MESSAGES/events
-msgfmt -o <openpilot_path>/selfdrive/assets/locales/<language>/LC_MESSAGES/<template_file>.mo <openpilot_path>/selfdrive/assets/locales/<language>/LC_MESSAGES/<template_file>
-```
-
-Reference
---
-https://simpleit.rocks/python/how-to-translate-a-python-project-with-gettext-the-easy-way/
@@ -36,14 +36,6 @@
 update_font=0
 remove_old_font=0

-# temp fix for lib change
-if [ ! -f "/system/comma/usr/lib/libcapnp-0.6.1.so" ]; then
-    mount -o remount,rw /system
-    ln -sf /system/comma/usr/lib/libcapnp.so /system/comma/usr/lib/libcapnp-0.6.1.so
-    ln -sf /system/comma/usr/lib/libkj.so /system/comma/usr/lib/libkj-0.6.1.so
-    mount -o remount,r /system
-fi
-
 # check regular font
 if [ ! -f "/system/fonts/NotoSansCJKtc-Regular.otf" ]; then
    update_font=1
@@ -10,7 +10,7 @@ WARN_FLAGS = -Werror=implicit-function-declaration \
             -Werror=format-extra-args

 CFLAGS = -std=gnu11 -g -fPIC -O2 $(WARN_FLAGS)
-CXXFLAGS = -std=c++1z -g -fPIC -O2 $(WARN_FLAGS)
+CXXFLAGS = -std=c++11 -g -fPIC -O2 $(WARN_FLAGS)

 CURL_FLAGS = -I$(PHONELIBS)/curl/include
 CURL_LIBS = $(PHONELIBS)/curl/lib/libcurl.a \
@@ -34,7 +34,6 @@ all: updater
 OBJS = opensans_regular.ttf.o \
 			 opensans_semibold.ttf.o \
 			 opensans_bold.ttf.o \
-       ../../selfdrive/common/util.o \
       ../../selfdrive/common/touch.o \
       ../../selfdrive/common/framebuffer.o \
       $(PHONELIBS)/json11/json11.o \
@@ -1,7 +1,7 @@
 {
-  "ota_url": "https://commadist.azureedge.net/neosupdate/ota-signed-e85f507777cb6b22f88ba1c8be6bbaa2630c484b971344b645fca2d1c461cd47.zip",
-  "ota_hash": "e85f507777cb6b22f88ba1c8be6bbaa2630c484b971344b645fca2d1c461cd47",
-  "recovery_url": "https://commadist.azureedge.net/neosupdate/recovery-db31ffe79dfd60be966fba6d1525a5081a920062b883644dc8f5734bcc6806bb.img",
-  "recovery_len": 15926572,
-  "recovery_hash": "db31ffe79dfd60be966fba6d1525a5081a920062b883644dc8f5734bcc6806bb"
+  "ota_url": "https://commadist.azureedge.net/neosupdate/ota-signed-efdf7de63b1aef63d68301e6175930991bf9a5927d16ec6fcc69287e2ee7ca4a.zip",
+  "ota_hash": "efdf7de63b1aef63d68301e6175930991bf9a5927d16ec6fcc69287e2ee7ca4a",
+  "recovery_url": "https://commadist.azureedge.net/neosupdate/recovery-97c27e6ed04ed6bb0608b845a2d4100912093f9380c3f2ba6b56bccd608e5f6e.img",
+  "recovery_len": 15861036,
+  "recovery_hash": "97c27e6ed04ed6bb0608b845a2d4100912093f9380c3f2ba6b56bccd608e5f6e"
 }
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Rick Lan	50ea09ae59	add 2018 skoda kodiaq fp	2020-07-03 11:35:47 +10:00
Rick Lan	c991f68bd7	dragonpilot 0.7.5.4 ======================== * Removed Steering monitor timer adjustment in code (timer in settings will no longer work.) * Updated Dynamic Follow model. (Special Thanks to @ShaneSmiskol for the feature and @cgw1968 for testing)	2020-06-22 11:50:30 +10:00
Rick Lan	0286285ec9	dragonpilot 0.7.5.3 ======================== * Updated Dynamic Follow to ShaneSmiskol:stock_additions 0.7.5. (Special Thanks to @ShaneSmiskol for the feature and @Wei for testing) * Optimize Lexus GSH steering. (Thanks to @簡銘佑) * C2 to support auto shutdown "DragonAutoShutDownAt" param. (Thanks to @cgw1968) * Fixed issue showing "pedalPressed" error. (Thanks to @Wei) * Added brake indicator to dp infobar. * Fixed "Temp monitor" indicator. * Added delay config to "Disable Lat Control on Blinker". (Thanks to @wabes) * Added Brazil 2020 Corolla Hybrid FPv2. (Thanks to @berno22)	2020-05-31 16:36:15 +10:00
Rick Lan	4b13bc6850	dragonpilot 0.7.5.2 ======================== * Added support to VW MQB/PQ platform. (Thanks to @dingliangxue) * Updated to stop charging after 3 hrs. (Thanks to @Wei) * Removed Impact Detection in Dashcam. * Fixed a bug caused by enabling "Noctua Mod". (Thanks to @阿濤) * Fixed alert issue in waze mode. (Thanks to @axandres) * Fixed unable to display "UPDATING" icon issue. * Added "Allow Continuous Auto Lane Change" Toggle. (Thanks to @阿濤) * Fixed setting page crash issue. (Thanks to @salmankhan, @Wei) * Fixed issue that keep showing update prompt. (Thanks to @Sky Chang)	2020-05-25 22:29:03 +10:00
Rick Lan	8502b57831	* Added Upload Over Mobile Network toggle. * Added Upload Over Hotspot toggle. * Added Canada 2018 Toyota Sienna LTD fingerprint (v1). (Thanks to 明峰) * Added China Volkswagen Golf GTI fingerprint (v1). (Thanks to easyeiji) * Added DragonMaxSpeedLimit parameter (mph), op will stop controlling when car speed is high than the value. (Thanks to @Anthony) * Updated appd to use cnpmjs to download APKs. * Fixed Update Service. (Thanks to @Wei) * Added Canada 2018 Toyota Sienna LTD fingerprint (v2). (Thanks to 明峰) * Fixed acceleration issue * Added missing param init value	2020-05-19 23:00:20 +10:00
Rick Lan	e81e6393e6	dragonpilot v0.7.5	2020-05-13 23:45:27 +10:00