more

* ci: add documentation sync and publishing workflows

- Added `sync_docs.rb` to automate syncing and processing documentation.
- Integrated GitHub Actions workflows to validate and publish docs.
- Supports dynamic title generation via Gemini API and proper formatting conversions.

* ci: add documentation sync and publishing workflows

- Added `sync_docs.rb` to automate syncing and processing documentation.
- Integrated GitHub Actions workflows to validate and publish docs.
- Supports dynamic title generation via Gemini API and proper formatting conversions.

* no need and fix

* maybe

* i think it wants this

* i think it wants this

* i think it wants this

* send it

* Fix the link

* Improve CI documentation processing logic and link handling

- Updated MkDocs conversions for "tabs" and callout styles for better Obsidian compatibility.
- Enhanced internal markdown link rewriting to resolve Discourse topic links.
- Reduced rate limit for Gemini API requests, improving call stability.
- Fixed GitHub link in generated documents to include specific document paths.

.github/workflows/forum-docs.yml

@@ -0,0 +1,46 @@
+name: Publish Docs
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+  workflow_dispatch:
+    inputs:
+      publish:
+        description: 'Set to true to publish to forum'
+        required: false
+        type: boolean
+
+env:
+  DOCS_CATEGORY_ID: 114
+  DOCS_DATA_EXPLORER_QUERY_ID: 2
+  DOCS_TARGET: https://forum.sunnypilot.ai
+  DOCS_API_KEY: ${{ secrets.DISCOURSE_API_KEY }}
+  GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
+
+jobs:
+#  dry_run:
+#    runs-on: ubuntu-latest
+#    steps:
+#      - uses: actions/checkout@v4
+#      - uses: ruby/setup-ruby@v1
+#        with:
+#          ruby-version: "3.3"
+#          bundler-cache: true
+#          working-directory: ./release/ci
+#      - run: bundle exec ./sync_docs.rb --dry-run
+#        working-directory: ./release/ci
+
+  publish:
+#    if: github.ref == 'refs/heads/main' || (github.event_name == 'workflow_dispatch' && github.event.inputs.publish == 'true')
+#    needs: dry_run
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: "3.3"
+          bundler-cache: true
+          working-directory: ./release/ci
+      - run: bundle exec ./sync_docs.rb
+        working-directory: ./release/ci

.gitignore

@@ -16,6 +16,7 @@ a.out
 .cache/
 
 /docs_site/
+/docs_sp_site/
 
 *.mp4
 *.dylib

docs_sp/SAFETY.md

@@ -0,0 +1,36 @@
+# Safety
+
+openpilot is an Adaptive Cruise Control (ACC) and Automated Lane Centering (ALC) system.
+Like other ACC and ALC systems, openpilot is a failsafe passive system and it requires the
+driver to be alert and to pay attention at all times.
+
+In order to enforce driver alertness, openpilot includes a driver monitoring feature
+that alerts the driver when distracted.
+
+However, even with an attentive driver, we must make further efforts for the system to be
+safe. We repeat, **driver alertness is necessary, but not sufficient, for openpilot to be
+used safely** and openpilot is provided with no warranty of fitness for any purpose.
+
+openpilot is developed in good faith to be compliant with FMVSS requirements and to follow
+industry standards of safety for Level 2 Driver Assistance Systems. In particular, we observe
+ISO26262 guidelines, including those from [pertinent documents](https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/13498a_812_573_alcsystemreport.pdf)
+released by NHTSA. In addition, we impose strict coding guidelines (like [MISRA C : 2012](https://www.misra.org.uk/what-is-misra/))
+on parts of openpilot that are safety relevant. We also perform software-in-the-loop,
+hardware-in-the-loop and in-vehicle tests before each software release.
+
+Following Hazard and Risk Analysis and FMEA, at a very high level, we have designed openpilot
+ensuring two main safety requirements.
+
+1. The driver must always be capable to immediately retake manual control of the vehicle,
+   by stepping on the brake pedal or by pressing the cancel button.
+2. The vehicle must not alter its trajectory too quickly for the driver to safely
+   react. This means that while the system is engaged, the actuators are constrained
+   to operate within reasonable limits[^1]. 
+
+For additional safety implementation details, refer to [panda safety model](https://github.com/commaai/panda#safety-model). For vehicle specific implementation of the safety concept, refer to [panda/board/safety/](https://github.com/commaai/panda/tree/master/board/safety).
+
+**Extra note**: comma.ai strongly discourages the use of openpilot forks with safety code either missing or
+  not fully meeting the above requirements.
+
+[^1]: For these actuator limits we observe ISO11270 and ISO15622. Lateral limits described there translate to 0.9 seconds of maximum actuation to achieve a 1m lateral deviation. 
+

docs_sp/assets/sp_logo.svg

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:675116a9b50eb713b2266396aff1460eed8263738455b1d49365f48168b4d4f9
+size 1698

docs_sp/branches/definitions.md

@@ -0,0 +1,13 @@
+# Definitions
+
+|     Branch     | Definition | Supported Devices                                                                                                                                                                                                                                 | Description                                                                                                                                                                                                                   | Stability/Readiness |
+|:--------------:|------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---|
+|   `release`    | Release branch | Comma 3X                                                                                                                                                                                                                                          | Stable release branch. After testing on `staging`, updates are pushed here and published publicly.                                                                                                                            | **Ready to Use:** Highly stable, recommended for most users.                                                                                                                                                             |
+|   `staging`    | Staging branch | Comma 3X                                                                                                                                                                                                                                          | Pre-release testing branch. Community feedback is essential to identify issues before public release.                                                                                                                         | **Varied Stability:** Generally stable, but intended for testing before public release.                 |
+|     `dev`      | Development branches | Comma 3X                                                                                                                                                                                                                                          | Experimental branch with the latest features and bug fixes brought in manually. Expect bugs and braking changes.                                                                                                              | **Experimental:** Least stable, suitable for testers and developers.                                    |
+|    `master`    | Primary development branch | Comma 3X | All Pull Requests are merged here for future releases. CI automatically strips, minifies, and pushes changes to `staging`. Running the `master` branch is suitable for development purposes but not recommended for non-development use. | **For Development Use:** Suitable for developers, may be unstable for general use.                                 |
+| `release-tici` | Release branch | Comma 3                                                                                                                                                                                                                                           | Stable release branch. After testing on `staging-tici`, updates are pushed here and published publicly.                                                                                                                       | **Ready to Use:** Highly stable, recommended for most users.                                                                                                                                                             |
+| `staging-tici` | Staging branch | Comma 3                                                                                                                                                                                                                                           | Pre-release testing branch. Community feedback is essential to identify issues before public release.                                                                                                                         | **Varied Stability:** Generally stable, but intended for testing before public release.                 |
+
+!!! tip
+    Your feedback is invaluable. Testers, even without software development experience, are encourage to run `dev` or `staging` and report issues.

docs_sp/branches/recommended-branches.md

@@ -0,0 +1,17 @@
+# Recommended Branches
+
+=== "Comma 3X"
+    
+    |  Branch   | Installation URL                      | Change Logs                                                                     |
+    |:---------:|-----------------------------------    |---------------------------------------------------------------------------------|
+    | `release` | **Coming Soon**                       | **Coming Soon**                                                                 |
+    | `staging` | [install.sunnypilot.ai/staging]()     | [CHANGELOG](https://github.com/sunnyhaibin/sunnypilot/blob/staging/RELEASES.md) |
+    |   `dev`   | [install.sunnypilot.ai/dev]()         | [CHANGELOG](https://github.com/sunnyhaibin/sunnypilot/blob/dev/RELEASES.md)     |
+
+=== "Comma 3"
+    
+    |  Branch   | Installation URL                              | Change Logs                                                                     |
+    |:---------:|---------------------------------------        |---------------------------------------------------------------------------------|
+    | `release-tici` | **Coming Soon**                          | **Coming Soon**                                                                 |
+    | `staging-tici` | [install.sunnypilot.ai/staging-tici]()   | [CHANGELOG](https://github.com/sunnyhaibin/sunnypilot/blob/staging/RELEASES.md) |
+

docs_sp/community/CONTRIBUTING.md

@@ -0,0 +1,68 @@
+# How to contribute
+
+Our software is open source so you can solve your own problems without needing help from others. And if you solve a problem and are so kind, you can upstream it for the rest of the world to use. Check out our [post about open-sourcing and externalization](https://www.sunnypilot.ai/blog/july/a-new-chapter-transparency/). Development activity is coordinated through our [GitHub Issues](https://github.com/sunnypilot/sunnypilot/issues), [GitHub Discussions](https://github.com/sunnypilot/sunnypilot/discussions), and [Discord](https://discord.sunnypilot.ai).
+
+### Getting Started
+
+* Setup your [development environment](https://github.com/sunnypilot/sunnypilot/tree/master/tools)
+* Read about the [development workflow](WORKFLOW.md)
+* Join our [Discord](https://discord.sunnypilot.ai)
+* Docs are at [https://docs.sunnypilot.ai](https://docs.sunnypilot.ai) and [https://www.sunnypilot.ai/blog](https://www.sunnypilot.ai/blog)
+
+## What contributions are we looking for?
+
+**sunnypilot's priorities are [safety](../SAFETY.md), stability, quality, and features, in that order.** Aligning with comma's ideals, part of sunnypilot's mission is to *solve self-driving cars while delivering shippable intermediaries*, and **all** development is towards that goal.
+
+### What gets merged?
+
+The probability of a pull request being merged is a function of its value to the project and the effort it will take us to get it merged.
+If a PR offers *some* value but will take lots of time to get merged, it will be closed.
+Simple, well-tested bug fixes are the easiest to merge, and new features are the hardest to get merged. 
+
+All of these are examples of good PRs:
+
+* [typo fix](https://github.com/commaai/openpilot/pull/30678)
+* [removing unused code](https://github.com/commaai/openpilot/pull/30573)
+* [simple car model port](https://github.com/commaai/openpilot/pull/30245)
+* [car brand port](https://github.com/commaai/openpilot/pull/23331)
+* [UI design changes](https://github.com/sunnypilot/sunnypilot/commit/84f6fce90639135611ec568c4d39a352a300bede)
+* [new features](https://github.com/sunnypilot/sunnypilot/commit/68e1379003bfdb599921cf9cd5684bfb762fd676)
+
+### What doesn't get merged?
+
+* **arbitrary style changes**: code is art, and it's up to the author to make it beautiful
+* **500+ line PRs**: clean it up, break it up into smaller PRs, or both
+* **PRs without a clear goal**: every PR must have a singular and clear goal
+
+### First contribution
+
+Check out any [good first issue from commaai's openpilot](https://github.com/commaai/openpilot/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) to get started.
+
+### What do I need to contribute?
+
+A lot of sunnypilot work requires only a PC, and some requires a comma device.
+Most car-related contributions require access to that car, plus a comma device installed in the car.
+
+## Pull Requests
+
+Pull requests should be against the [`master`](https://github.com/sunnypilot/sunnypilot) branch. If you're unsure about a contribution, feel free to open a discussion, issue, or draft PR to discuss the problem you're trying to solve.
+
+A good pull request has all of the following:
+
+- a clearly stated purpose
+- every line changed directly contributes to the stated purpose
+- verification, i.e. how did you test your PR?
+- justification
+
+    * if you've optimized something, post benchmarks to prove it's better
+    * if you've improved your car's tuning, post before and after plots
+
+- passes the CI tests
+
+## Contributing without Code
+
+* Report bugs in [GitHub issues](https://github.com/sunnypilot/sunnypilot/issues).
+* Report driving issues in the `#general` Discord channel.
+* Consider opting into driver camera uploads to improve the driver monitoring model.
+* Connect your device to Wi-Fi regularly, so that comma can pull data for training better driving models.
+* Run the `staging-c3` branch and report issues. This branch is like `master` but it's built just like a release.

docs_sp/community/WORKFLOW.md

@@ -0,0 +1,43 @@
+# sunnypilot development workflow
+
+Aside from the ML models, most tools used for sunnypilot development are in this repo.
+
+Most development happens on normal Ubuntu workstations, and not in cars or directly on comma devices. See the [setup guide](https://github.com/sunnypilot/sunnypilot/tree/master/tools) for getting your PC setup for sunnypilot development.
+
+## Quick start
+
+```bash
+# get the latest stuff
+git pull
+git lfs pull
+git submodule update --init --recursive
+
+# update dependencies
+tools/ubuntu_setup.sh
+
+# build everything
+scons -j$(nproc)
+
+# build just the ui with either of these
+scons -j8 selfdrive/ui/
+cd selfdrive/ui/ && scons -u -j8
+
+# test everything
+pytest
+
+# test just logging services
+cd system/loggerd && pytest .
+
+# run the linter
+op lint
+```
+
+## Testing
+
+### Automated Testing
+
+All PRs and commits are automatically checked by GitHub Actions. Check out `.github/workflows/` for what GitHub Actions runs. Any new tests should be added to GitHub Actions.
+
+### Code Style and Linting
+
+Code is automatically checked for style by GitHub Actions as part of the automated tests. You can also run these tests yourself by running `pre-commit run --all`.

docs_sp/community/reporting-a-bug.md

@@ -0,0 +1,274 @@
+# Bug Reports
+
+sunnypilot is an actively maintained project that we constantly strive to improve. With project of this size and complexity, 
+bugs may occur. If you think you have discovered a bug, you can help us by submitting an issue 
+in [comma's public issue tracker][comma's issue tracker], 
+[sunnypilot's public issue tracker][sunnypilot's issue tracker] or on our [Discord][discord], following this guide.
+
+  [comma's issue tracker]: https://github.com/commaai/openpilot/issues
+  [sunnypilot's issue tracker]: https://github.com/sunnypilot/sunnypilot/issues
+  [discord]: https://discord.sunnypilot.ai
+
+## Before creating an issue
+
+With more than 2,500 users, issues are created frequently. The maintainers of this project are trying very hard to keep 
+the number of open issues and reports down by fixing bugs as fast as possible. By following this guide, you will know 
+exactly what information we need to help you quickly.
+
+**But first, please do the following things before creating an issue.**
+
+### Upgrade to the latest version
+
+Chances are that the bug you discovered was already fixed in a subsequent version. Thus, before reporting an issue, 
+ensure that you're running the [latest release version](https://github.com/sunnypilot/sunnypilot/releases) of sunnypilot. 
+Please consult our [installation guide](../setup/read-before-installing.md) to learn how to upgrade to the latest version.
+
+!!! warning "Bug fixes are not backported"
+    Please understand that only bugs that occur in the latest version of sunnypilot will be addressed. Also, to reduce 
+    duplicate efforts, fixes cannot be backported to earlier versions.
+
+### Remove customizations
+
+If you're using customized features, such as your own tweaks of the features, please remove them from the branch 
+you are testing from before reporting a bug. We can't offer official support for bugs that might hide in your implementations, 
+so make sure to omit any customizations from the version being tested.
+
+If, after removing the customizations, the bug is gone, the bug is likely caused by your customizations. A good idea is 
+to add them back gradually to narrow down the root cause of the problem If you did a major version upgrade, make sure 
+you adjusted all customizations you have implemented.
+
+!!! tip
+    If you are an advanced user, you could also utilize `git bisect` 
+    to perform a binary search in the history to find a particular regression.
+
+!!! warning "Customizations mentioned in our documentation"
+    A handful of the features sunnypilot offers can only be implemented with customizations. if you find a bug in any of 
+    the customizations that our documentations explicitly mentioned, you are, of course, encouraged to report it.
+
+**Don't be shy to ask on our [Discord][discord] for help if you run into problems.**
+
+### Search for solutions
+
+At this stage, we know that the problem persists in the latest version and is not caused by any of your customizations. 
+However, the problem might result from a small typo or a syntactical error in the source code, e.g., `selfdrive/car/interfaces.py`.
+
+Now, before you go through the trouble of creating a bug report that is answered and closed right away with a link to
+the relevant documentation section or another already reported or closed issue or discussion, you can save time for us 
+and yourself by doing some research:
+
+1. [Search our documentation] and look for the relevant sections that could be related to your problem. If found, make
+    sure that the settings are configured correctly.
+2. [Search our Discord][discord] to learn if other users are struggling with similar problems and work together with 
+   our 
+   great community towards a solution. Many problems are solved there.
+3. [Search comma's openpilot issue tracker][comma's issue tracker], as another user might 
+   already have reported the same problem that may exist in
+   stock openpilot, and there might even be a known workaround or fix for it. Thus, no need to create a new issue.
+4. [Search sunnypilot's issue tracker][sunnypilot's issue tracker], as another user might already have 
+   reported the same problem, and there
+   might even be a known workaround or fix for it. Thus, no need to create a new issue.
+
+[Search our documentation]: ?q=
+
+**Keep track of all <u>search terms</u> and <u>relevant links</u>, you'll need them in the bug report.**[^1]
+
+  [^1]:
+    We might be using terminology in our documentation different from yours, but we mean the same. When you include the 
+    search terms and related links in your bug report, you help us to adjust and improve the documentation. 
+
+---
+
+At this point, when you still haven't found a solution to your problem, we encourage you to report the issue on our 
+[Discord][discord] because it's now very likely that you stumbled over something we don't know 
+yet. Read the following section 
+to learn how to create a complete and helpful bug report.
+
+## Issue template
+
+We have created an issue template to make the bug reporting process as simple as possible, and more efficient for our
+community and us.
+
+- [Title]
+- [Context]<small>optional</small>
+- [Bug description]
+- [Related links]
+- [Reproduction]
+- [Steps to reproduce]
+- [Checklist]
+
+  [Title]: #title
+  [Context]: #context
+  [Bug description]: #bug-description
+  [Related links]: #related-links
+  [Reproduction]: #reproduction
+  [Steps to reproduce]: #steps-to-reproduce
+  [Checklist]: #checklist
+
+### Title
+
+A good title is short and descriptive. It should be a one-sentence executive summary of the issue, so the impact and
+severity of the bug you want to report can be inferred from the title.
+
+| <!-- --> | Example                                                                                                      |
+| -------- |--------------------------------------------------------------------------------------------------------------|
+| :material-check:{ style="color: #4DB6AC" } __Clear__ | Speed Limit Control (SLC) stuck in `preActive` when engaged                                                  |
+| :material-close:{ style="color: #EF5350" } __Wordy__ | The Speed Limit Control (SLC) remains in the `preActive` state when longitudinal it's supposed to be engaged |
+| :material-close:{ style="color: #EF5350" } __Unclear__ | SLC does not work                                                                                            |
+| :material-close:{ style="color: #EF5350" } __Useless__ | Help                                                                                                         |
+
+### Context <small>optional</small> { #context }
+
+Before describing the bug, you can provide additional context for us to understand what you were trying to achieve.
+Explain the circumstances in which you're using sunnypilot, and what you _think_ might be relevant. Don't write 
+about the bug here.
+
+!!! note "__Why this might be helpful__"
+    Some errors only manifest in specific settings, environments or edge cases, for example, when the feature is not available
+    to certain cars.
+
+### Bug description
+
+Now, to the bug you want to report. Provide a clear, focused, specific, and concise summary of the bug you encountered. 
+Explain why you think this is a bug that should be reported to sunnypilot, and not to one of its dependencies.[^3]
+Adhere to the following principles:
+
+  [^3]:
+    Sometimes, users report bugs on our [sunnypilot's issue tracker] or [Discord][discord] 
+    that are caused by one of our upstream dependencies, including [comma's openpilot], [comma's panda],
+    or other openpilot forks' dependencies. A good rule of thumb is
+    to reproduce the issue with stock openpilot in the same conditions and
+    check if the problem persists. If it does, the problem is likely not
+    related to sunnypilot and should be reported upstream. When in
+    doubt, use our [Discord][discord] to ask for help.
+
+  [comma's openpilot]: https://github.com/commaai/openpilot
+  [comma's panda]: https://github.com/commaai/panda
+
+-   __Explain the <u>what</u>, not the <u>how</u>__ – don't explain
+    [how to reproduce the bug][Steps to reproduce] here, we're getting there.
+    Focus on articulating the problem and its impact as clearly as possible.
+
+-   __Keep it short and concise__ – if the bug can be precisely explained in one
+    or two sentences, perfect. Don't inflate it – maintainers and future users
+    will be grateful for having to read less.
+
+-   __One bug at a time__ – if you encounter several unrelated bugs, please
+    create separate issues for them. Don't report them in the same issue, as
+    this makes attribution difficult.
+
+---
+
+:material-run-fast: __Stretch goal__ – if you found a workaround or a way to fix
+the bug, you can help other users temporarily mitigate the problem before
+we maintainers can fix the bug in our code base.
+
+!!! note "__Why we need this__"
+    In order for us to understand the problem, we need a clear description of it and quantify its impact, which is 
+    essential for triage and prioritization.
+
+### Related links
+
+Of course, prior to reporting a bug, you have read our documentation and
+[could not find a working solution][search for solutions]. Please share links
+to all sections of our documentation that might be relevant to the bug, as it
+helps us gradually improve it.
+
+Additionally, since you have searched [comma's issue tracker], [sunnypilot's issue tracker] or [Discord][discord]
+before reporting an issue, and have possibly found several issues or
+discussions, include those as well. Every link to an issue or discussion creates
+a backlink, guiding us maintainers and other users in the future.
+
+---
+
+:material-run-fast: __Stretch goal__ – if you also include the search terms you
+used when [searching for a solution][search for solutions] to your problem, you
+make it easier for us maintainers to improve the documentation.
+
+  [search for solutions]: #search-for-solutions
+
+### Reproduction
+
+A minimal reproduction is at the heart of every well-written bug report, as
+it allows us maintainers to instantly recreate the necessary conditions to
+inspect the bug to quickly find its root cause. It's a proven fact that issues
+with concise and small reproductions can be fixed much faster.
+
+After you have created the reproduction, take note of your <u>__comma Dongle ID__</u>. It will be used during the bug 
+report.
+
+!!! note "__Why we need this__"
+    If an issue contains no minimal reproduction or just a link to a repository with thousands of files, the 
+    maintainers would need to invest a lot of time into trying to recreate the right conditions to even inspect the 
+    bug, let alone fix it.
+
+!!! warning "Don't share links to repositories"
+    While we know that it is a good practice among developers to include a link
+    to a repository with the bug report, we currently don't support those in our
+    process. The reason is that the reproduction, which is automatically
+    produced by the <u>__route ID__</u> contains all the necessary
+    environment information that is often forgotten to be included.
+
+    Additionally, there are many non-technical users of sunnypilot that
+    have trouble creating repositories.
+
+### Steps to reproduce
+
+At this point, you provided us with enough information to understand the bug
+and provided us with a reproduction that we could run and inspect. However, when
+we check your reproduction, it might not be immediately apparent how we can see
+the bug in action.
+
+Thus, please list the specific steps we should follow when running your
+reproduction to observe the bug. Keep the steps short and concise, and make sure
+not to leave anything out. Use simple language as you would explain it to a
+five-year-old, and focus on continuity.
+
+!!! note "__Why we need this__"
+    We must know how to navigate your reproduction in order
+    to observe the bug, as some bugs only occur at certain viewports or in
+    specific conditions.
+
+### Uploading logs and preserving routes
+
+After reproducing the bug, please follow these steps to upload the necessary logs and preserve the routes.
+
+1. Ensure the route is fully uploaded at [comma Connect]. We cannot look 
+  into issues without routes, or at least a comma Dongle ID.
+
+    1. Visit [comma Connect], select the route with the issue reproduced.
+    2. Under the "Files" button, locate "All logs". Click "Upload x files".
+    3. View the upload queue, and confirm that all raw logs are uploaded.
+
+    !!! note
+        Sometimes when the qlogs of the route are still being uploaded, some raw logs may not be available to 
+        request for upload. Refresh the page a few times once you have confirmed all qlogs have been uploaded, 
+        then try to upload all raw logs again if available.
+
+2. Share your Dongle ID with sunnypilot on [comma Connect].
+
+    1. Visit [comma Connect], navigate to the gear icon.
+    2. Select "Share by email", and enter `support@sunnypilot.ai`.
+    3. Confirm the sharing by clicking the share icon again.
+    4. Set the device name to your vehicle's year/make/model and your Discord username, so it can be easily identified.
+
+3. Once all raw logs are uploaded, click "More info" and enable the "Preserved" option to preserve the route.
+4. Attach the route ID in your issue submission.
+
+  [comma Connect]: https://connect.comma.ai
+
+### Checklist
+
+Thanks for following the guide and creating a high-quality and complete bug
+report – you are almost done. The checklist ensures that you have read this guide
+and have worked to your best knowledge to provide us with everything we need to
+know to help you.
+
+- [ ] I have upgraded to the latest release version of sunnypilot.
+- [ ] I have removed or disable any customizations and confirmed the bug persists.
+- [ ] I have searched the documentation, issue trackers, and Discord for similar issues.
+- [ ] I have created a minimal reproduction and noted my comma Dongle ID.
+- [ ] I have shared my Dongle ID with sunnypilot at `support@sunnypilot.ai`.
+- [ ] I have filled out all required sections of the issue template.
+- [ ] I have followed this guide and ensured all necessary information is included.
+
+__We'll take it from here.__

docs_sp/community/reporting-a-docs-issue.md

@@ -0,0 +1,97 @@
+# Documentation issues
+
+Our documentation is composed of many pages and includes extensive
+information on features, configurations, customizations, and much more. If you
+have found an inconsistency or see room for improvement, please follow this
+guide to submit an issue on our [issue tracker].
+
+  [issue tracker]: https://github.com/sunnypilot/sunnypilot/issues
+
+## Issue template
+
+Reporting a documentation issue is usually less involved than reporting a bug.
+Please thoroughly read this guide before creating a new documentation issue,
+and provide the following information as part of the issue:
+
+- [Title]
+- [Description]
+- [Related links]
+- [Proposed change] <small>optional</small>
+- [Checklist]
+
+  [Title]: #title
+  [Description]: #description
+  [Related links]: #related-links
+  [Proposed change]: #proposed-change
+  [Checklist]: #checklist
+
+### Title
+
+A good title should be a short, one-sentence description of the issue, contain
+all relevant information and, in particular, keywords to simplify the search in
+our issue tracker.
+
+| <!-- --> | Example  |
+| -------- | -------- |
+| :material-check:{ style="color: #4DB6AC" } __Clear__ | Clarify Speed Limit Control engagement
+| :material-close:{ style="color: #EF5350" } __Unclear__ | Missing information in the docs
+| :material-close:{ style="color: #EF5350" } __Useless__ | Help
+
+### Description
+
+Provide a clear and concise summary of the inconsistency or issue you
+encountered in the documentation or the documentation section that needs
+improvement. Explain why you think the documentation should be adjusted and
+describe the severity of the issue:
+
+-   __Keep it short and concise__ – if the inconsistency or issue can be
+    precisely explained in one or two sentences, perfect. Maintainers and future
+    users will be grateful for having to read less.
+
+-   __One issue at a time__ – if you encounter several unrelated inconsistencies,
+    please create separate issues for them. Don't report them in the same issue
+    – it makes attribution difficult.
+
+!!! note "__Why we need this__"
+    Describing the problem clearly and concisely is a prerequisite for improving
+    our documentation – we need to understand what's wrong, so we can fix it.
+
+### Related links
+
+After you described the documentation section that needs to be adjusted above,
+we now ask you to share the link to this specific documentation section and
+other possibly related sections. Make sure to use anchor links (permanent links)
+where possible, as it simplifies discovery.
+
+!!! note "__Why we need this__"
+    Providing the links to the documentation help us understand which sections
+    of our documentation need to be adjusted, extended, or overhauled.
+
+### Proposed change <small>optional</small> { #proposed-change }
+
+Now that you have provided us with the description and links to the
+documentation sections, you can help us, maintainers, and the community by
+proposing an improvement. You can sketch out rough ideas or write a concrete
+proposal. This field is optional but very helpful.
+
+!!! note "__Why we need this__"
+    An improvement proposal can be beneficial for other users who encounter
+    the same issue, as they offer solutions before we maintainers can update
+    the documentation.
+
+### Checklist
+
+Thanks for following the guide and providing valuable feedback for our
+documentation – you are almost done. The checklist ensures that you have read
+this guide and have worked to your best knowledge to provide us with every piece
+of information we need to improve it.
+
+- [ ] I have provided a clear and descriptive title for the documentation issue.
+- [ ] I have summarized the inconsistency or issue concisely in the description.
+- [ ] I have included links to the specific documentation section(s) that need
+      adjustments.
+- [ ] (Optional) I have proposed a change or improvement to the documentation.
+- [ ] I have followed this guide and ensured all necessary information is included.
+
+__We'll take it from here.__
+

docs_sp/features/auto-lane-change.md

@@ -0,0 +1,24 @@
+---
+title: Auto Lane Change
+description: Detailed documentation on the Auto Lane Change feature in sunnypilot.
+---
+
+# Auto Lane Change
+
+sunnypilot's Auto Lane Change feature allows the vehicle to automatically change lanes.
+
+## Key Setting and How to Use
+
+- **Nudge to Confirm:** This is the default method. To perform a lane change, activate the turn signal in the desired direction and then gently turn the steering wheel (nudges) in the same direction to confirm the maneuver.
+- **Nudge-less Lane Change:** For a more automated experience, users can enable a "nudge-less" option. With this setting, you only need to activate the turn signal to initiate a lane-change.
+- **Nudge-less with Delay** This is same as Nudge-less mode, but with a fixed delay. Once the turn-indicator is activated, sunnypilot will wait for the selected delay before initiating the lane change.
+
+### Additional Related Settings
+- **Delay with Blind Spot:** This crucial safety feature adds a delay to the lane change if the vehicle's blind spot monitoring (BSM) system detects an object. The lane change will only proceed after the blind spot is clear.
+
+## Important Considerations
+
+- **Manual Override:** The driver can regain control at any time by taking control of the steering wheel.
+- **Driver Monitoring:** sunnypilot utilizes driver monitoring to ensure the driver remains attentive.
+- **Object Detection:** The system is primarily designed to follow lane lines and may not detect all objects. The driver must always be prepared to take control.
+- **Lane Markings:** The feature's performance is dependent on clear and visible lane markings.

docs_sp/features/custom-acc-increments.md

@@ -0,0 +1,13 @@
+---
+title: Custom ACC Increments
+description: Detailed documentation on the Custom ACC Increments feature in sunnypilot.
+---
+
+# Custom ACC Increments
+
+sunnypilot offers customization to the set speed increments for Adaptive Cruise Control (ACC) when sunnypilot is controlling the vehicle's longitudinal control. This allows for more precise speed adjustments compared to the default behavior of many vehicles.
+
+- **Short press**: This controls the speed adjustment when the "RES+" or "SET-" button is pressed for a short period of time.
+  - Allowed values: 1 through 10
+- **Long press**: This controls the speed adjustment when the "RES+" or "SET-" button is pressed for a longer duration.
+  - Allowed values: 1, 5, 10

docs_sp/features/dynamic-experimental-control.md

@@ -0,0 +1,14 @@
+---
+title: Dynamic Experimental Control
+description: Detailed documentation on the Dynamic Experimental Control feature in sunnypilot.
+---
+
+# Dynamic Experimental Control (DEC)
+
+Dynamic Experimental Control (DEC) intelligently switches between the standard adaptive cruise control (ACC) and the end-to-end longitudinal control of Experimental Mode based on the driving conditions.
+
+**Chill Mode** is the standard adaptive cruise control method. It is designed to provide a smooth, predictable driving experience. However, it does not provide advanced driving capabilities like stopping at red lights and stop signs.
+
+**Experimental Mode** allows the system to control the vehicle's speed and tries to drive like a human, enabling it to slow down for turns, stop at red lights, and stop signs. While capable and experimental, this mode can sometimes be overly cautious, especially on highways.
+
+***Dynamic Experimental Control*** aims to provide the best of both worlds: the advanced capabilities of Experimental Mode when needed, and the smooth, predictable behavior of the standard system for less complex driving scenarios. This allows for a more natural driving experience by using Experimental Mode in situations where it excels, such as city driving and tight turns, while reverting to the default behavior for highway driving.

docs_sp/features/hyundai-longitudinal-tuning.md

@@ -0,0 +1,97 @@
+
+# **How custom longitudinal tuning works for Hyundai/Kia/Genesis vehicles in sunnypilot.**
+
+To begin this documentation, I would like to first present the safety guidelines followed to create the tune:
+
+Our main safety guideline considered is [ISO 15622:2018](https://www.iso.org/obp/ui/en/#iso:std:iso:15622:ed-3:v1:en)
+This provides the groundwork of safety limits this tune must adhere too, and therefore, must be followed.
+For example, in our jerk calculations throughout this tune, you will see how maximum jerk is clipped using the equation provided.
+
+In the tuning you will see a set of equations, the first being jerk, **but what exactly is jerk?**
+Jerk is calculated by taking current acceleration (in the form of m/s^2), subtracting that by previous acceleration, and
+dividing that by time. In our tune you will see the following equation:
+
+    planned_accel = CC.actuators.accel
+    current_accel = CS.out.aEgo
+    blended_value = 0.67 * planned_accel + 0.33 * current_accel
+    delta = blended_value - self.state.accel_last_jerk
+
+    self.state.jerk = math.copysign(delta * delta, delta)
+    self.state.accel_last_jerk = blended_value
+
+Instead of using a hardcoded time, we are focused on making jerk parabolic. First we have our planned acceleration from longitudinal_planner.
+Then we have our current carstate acceleration. These are then blended together 67:33 = 100% to form our blended value.
+Following this, we have our delta which subtracts our blended_value from our previous acceleration `self.state.accel_last_jerk`
+Lastly, we have our finalized jerk calculation, which squares the delta to create a parabolic response while retaining the original sign,
+which could be positive or negative (e.g., 5.0 or -5.0). This then goes through our minimum and maximum clipping
+which forces a value between our set min and max, which I discuss later in this readme.
+
+Moving on, the accel_last_jerk, stores current accel after each iteration and uses that in the calculation as previous accel for
+our jerk calculations. Now we see the calculation of jerk max and jerk min. 
+
+### Let's dive into how jerk lower limit max is calculated:
+
+     velocity = CS.out.vEgo
+    if velocity < 5.0:
+      decel_jerk_max = self.car_config.jerk_limits[1]
+    elif velocity > 20.0:
+      decel_jerk_max = 2.5
+    else:
+      decel_jerk_max = 3.64284 - 0.05714 * velocity
+
+This equation above is set by ISO 15622, and dictates that jerk lower limit can only be five when below 5 m/s. In our equation,
+
+    self.car_config.jerk_limits[1] 
+
+Jerk_limits[1] represents a jerk value of 3.3 m/s^3, which is the maximum analyzed lower jerk rate seen on stock SCC CAN.
+Between 5 m/s and 20 m/s jerk is capped using the calculation:
+
+    decel_jerk_max = 3.64284 - 0.05714 * velocity
+
+This equation calculates the linear jerk from 6m/s to 19m/s, scaling down from 3.3 to 2.5 m/s^3.
+This means that if current velocity is say, 15 m/s the final jerk max value would be capped at 2.78 m/s^3.
+Anything above 20 m/s is capped to a lower jerk max of 2.5 m/s^3. This allows for a smoother jerk range, while complying to ISO standards to a tee.
+The current jerk Lower Limit you will see in openpilot before this tune, is 5.0 m/s^3; Which as you can see from using the above calculation,
+the 5.0 m/s^3 technically does not comply with ISO standards at any speed above 5.0 m/s^3.
+Having our jerk max be clipped to these values not only allows for better consistency with ISO standards, 
+but also enables us to have a much smoother braking experience.
+
+### Getting into our next topic, I would like to explain how our minimum jerk was chosen.
+
+Minimum jerk was chosen based off of the following guideline proposed by Handbook of Intellegent Vehicles (2012):
+`Ride comfort may be sacrificed only under emergency conditions when vehicle and occupant safety consideration may preclude comfort.`
+
+### What the value of 0.53 m/s^3 of the jerk lower limit was chosen based off of
+
+[Carlowitz et al. (2024).](https://www.researchgate.net/publication/382274551_User_evaluation_of_comfortable_deceleration_profiles_for_highly_automated_driving_Findings_from_a_test_track_study)
+This research study identified the average lower jerk used in comfortable driving settings, which is 0.53 m/s^3.
+This is then inputted to jerk_limits[0] as 0.53 m/s^3 represents the value used in upper jerk absolute minimum.
+
+     min_lower_jerk = self.car_config.jerk_limits[0]
+
+As shown above, lower jerk minimum of 0.53 is used for our lower_jerk minimum bounds.
+
+### Why our minimum upper jerk is conditional
+
+Our minimum upper band jerk is conditional as well and is denoted below:
+
+    min_upper_jerk = self.car_config.jerk_limits[0] if (velocity > 3.611) else 0.60
+
+This means that for speeds under 3.611 m/s (8.077 mph/ 13 kph) we have a minimum jerk of 0.60. This allows for smooth
+takeoffs while not causing lag. For all other speeds, we use our normal jerk_limit for minimum, which is 0.53.
+
+### Next, we have our acceleration limiting
+
+For acceleration limiting, we use TCS signal brakeLightsDEPRECATED to measure when to enact the standstill delay 
+which stock SCC uses to allow smoother transitions in acceleration.
+
+### Lastly, we have our accel value calculations for hyundaican.py
+
+For our accel value calculations we have the following:
+
+   `self.accel_value = np.clip(self.accel_raw, self.state.accel_last - jerk_number, self.state.accel_last + jerk_number)`
+
+This essentially means that we have our accel_raw, which is acceleration (m/s^2), followed by our clipping variables. 
+jerk_number in this equation represents exactly `0.1`, which is subtracted or added by self.state.accel_last, which is 
+previous calculated accel_value. Furthermore, we have `self.state.accel_last`, which is calculated as the stored accel from
+the above calculations.

docs_sp/features/icbm.md

@@ -0,0 +1,17 @@
+---
+title: ICBM (Intelligent Cruise Button Management)
+description: Detailed documentation on the ICBM feature in sunnypilot.
+---
+
+# ICBM (Intelligent Cruise Button Management)
+
+Intelligent Cruise Button Management (ICBM) is a system designed to manage the vehicle's cruise control set speed by sending cruise control button commands via CAN bus to the car.
+
+ICBM is particularly useful in vehicles openpilot Longitudinal Control is not available or not desirable to use. By simulating button presses to adjust the stock cruise control set speed, sunnypilot's ICBM can manage the car's speed while keeping native safety features active, such as Forward Collision Warning (FCA) and Automatic Emergency Braking (AEB).
+
+ICBM also allows the vehicle to use the following features:
+
+- **[Smart Cruise Control - Vision (SCC-V)]()**
+- **[Smart Cruise Control - Map (SCC-M)]()**
+- **[Speed Limit Assist (SLA)]()**
+- **[Custom ACC Increments]()**

docs_sp/features/mads.md

@@ -0,0 +1,17 @@
+# Modular Assistive Driving System (M.A.D.S.)
+
+Modular Assistive Driving System (MADS) aims to elevate the user's driving experience by modifying the behaviors of 
+driving assist engagements.
+
+!!! note
+    This feature aligns closely with comma.ai's safety rules.
+
+## Independent Engagement
+
+MADS allows users to engage sunnypilot Automatic Lane Centering (ALC) for lateral control and Adaptive Cruise Control 
+(ACC) or Smart Cruise Control (SCC) for longitudinal control independently.
+
+!!! note "Why This Feature Exists"
+    While newer car models allow for independent engagement of lateral (steering) and longitudinal (speed) control, 
+    many older vehicle models and stock openpilot enforce engaging both controls together. MADS introduces this modern 
+    convenience to older vehicle models, effectively backporting a feature found in newer cars and providing users more flexibility.

docs_sp/features/nnlc.md

@@ -0,0 +1,15 @@
+---
+title: Neural Network Lateral Control (NNLC)
+description: Detailed documentation on the NNLC feature in sunnypilot.
+---
+
+# Neural Network Lateral Control (NNLC)
+
+sunnypilot's Neural Network Lateral Control (NNLC) is a feature that enhances the system's ability to steer a vehicle. It enhances the standard lateral controller with one based on a neural network trained on the vehicle's torque data, aiming for smoother and more precise steering adjustments.
+
+## Key Aspects of NNLC
+
+- **Improved Accuracy:** The neural network is trained using driving data specific to each vehicle, which allows for more accurate control.
+- **Smoother Turns:** NNLC inputs past curvature data into its driving model to achieve smoother and more precise lateral control, especially noticeable when taking curves on a highway. Users report that it reduces the oversteering and understeering corrections.
+
+Formerly known as "NNFF" (Neural Network Feedforward), NNLC aims to make the driving experience feel more natural and less "jittery" in turns.

docs_sp/features/scc-m.md

@@ -0,0 +1,9 @@
+# Smart Cruise Control - Map (SCC-M)
+
+Smart Cruise Control - Map (SCC-M) leverages map data to proactively adjust your vehicle's speed by calculating curvature in the road ahead.
+
+!!! note
+    This feature is only available when sunnypilot is actively controlling the vehicle's longitudinal control.
+
+!!! note
+    This feature requires OpenStreetMap data for the current location to be downloaded.

docs_sp/features/scc-v.md

@@ -0,0 +1,6 @@
+# Smart Cruise Control - Vision (SCC-V)
+
+Smart Cruise Control - Vision (SCC-V) leverages camera vision data to proactively adjust your vehicle's speed by calculating curvature in the road ahead.
+
+!!! note
+    This feature is only available when sunnypilot is actively controlling the vehicle's longitudinal control.

docs_sp/features/speed-limit-assist.md

@@ -0,0 +1,26 @@
+---
+title: Speed Limit Assist
+description: Detailed documentation on the Speed Limit Assist feature in sunnypilot.
+---
+
+# Speed Limit Assist (SLA)
+
+Speed Limit Assist (SLA) is a comprehensive framework in sunnypilot that adjusts the vehicle's set cruise speed according to detected speed limits. It reconciles data from multiple sources to determine the current speed limit.
+
+## Core Functionality
+
+SLA utilizes various data sources to determine the speed limit for the current road:
+
+- **OpenStreetMap (OSM):** Utilizes map data to fetch speed limits. sunnypilot has a refactored OSM implementation for lower resource usage and provides weekly updates.
+- **Car's Stock System:** On some compatible vehicles, it can pull speed limit data directly from the car's native system.
+
+## Key Features and Customization
+
+sunnypilot offers extensive customization for its speed control features:
+
+- **Selectable Speed Limit Source:** Users can define the priority of data sources (e.g., OSM, Navigation, Vision) for determining the speed limit.
+- **Configurable Offsets:** You can set an offset above the detected speed limit, either as a fixed value (e.g., +5 mph) or a percentage.
+- **Engagement Modes:**
+    - **Auto:** Automatically adjusts the cruise speed when a new speed limit is detected.
+    - **User Confirm:** Informs the driver of the new speed limit and waits for them to confirm the change before adjusting the speed.
+- **Alerts and Warnings:** The system provides alerts to inform the driver before a speed adjustment occurs.

docs_sp/getting-started/develop-sunnypilot.md

@@ -0,0 +1,9 @@
+# To start developing sunnypilot
+
+sunnypilot is a fork of [commaai's openpilot](https://github.com/commaai/openpilot), developed by [sunnypilot](https://sunnypilot.ai) and by users like you. 
+We welcome both pull requests and issues on [GitHub](http://github.com/sunnypilot/sunnypilot).
+
+* Join the [community Discord](https://discord.sunnypilot.ai)
+* Check out [the contributing docs](../community/CONTRIBUTING.md)
+* Check out the [openpilot tools](https://github.com/sunnypilot/sunnypilot/tree/master/tools)
+* Read about the [development workflow](../community/WORKFLOW.md)

docs_sp/getting-started/use-sunnypilot-in-a-car.md

@@ -0,0 +1,16 @@
+# To start using sunnypilot in a car
+
+To use sunnypilot in a car, you need four things:
+
+1. **Supported Device:** a comma 3/3X, available at [comma.ai/shop](https://comma.ai/shop/comma-3x).
+
+2. **Software:** The setup procedure for the comma 3/3X allows users to enter a URL for custom software. Use the URL `release-c3.sunnypilot.ai` to install the release version.
+
+3. **Supported Car:** Ensure that you have one of [the 275+ supported cars](https://github.com/sunnypilot/sunnypilot/blob/master/docs/CARS.md).
+
+4. **Car Harness:** You will also need a [car harness](https://comma.ai/shop/car-harness) to connect your comma 3/3X to your car.
+
+[comma.ai](https://comma.ai) have detailed instructions for [how to install the harness and device in a car](https://comma.ai/setup).
+
+!!! note
+    It's possible to run sunnypilot on [other hardware](https://blog.comma.ai/self-driving-car-for-free/), although it's not plug-and-play.

docs_sp/getting-started/what-is-sunnypilot.md

@@ -0,0 +1,11 @@
+# What is sunnypilot?
+
+sunnypilot is a fork of [comma.ai's openpilot](https://github.com/commaai/openpilot), an open source driver assistance system. sunnypilot offers the user a unique driving experience for over 250+ supported car makes and models with modified behaviors of driving assist engagements. sunnypilot complies with comma.ai's safety rules as accurately as possible.
+
+## How do I use it?
+
+sunnypilot is designed to be used on the comma 3/3X.
+
+## How does it work?
+
+In short, sunnypilot uses the car's existing APIs for the built-in [ADAS](https://en.wikipedia.org/wiki/Advanced_driver-assistance_system) system and simply provides better acceleration, braking, and steering inputs than the stock system.

docs_sp/index.md

@@ -0,0 +1,3 @@
+The documentation here is as best-effort sync from the official https://sunnypilot.github.io/ for ease of access and for AI enhancement when users asks on the forum.
+
+

docs_sp/safety-information/prohibited-safety-modifications.md

@@ -0,0 +1,14 @@
+# Prohibited Safety Modifications
+
+All [official sunnypilot branches](https://github.com/sunnyhaibin/sunnypilot/branches) strictly adhere to [comma.ai's safety policy](https://github.com/commaai/openpilot/blob/master/docs/SAFETY.md). Any changes that go against 
+this policy will result in your fork and your device being banned from both comma.ai and sunnypilot channels.
+
+The following changes are **VIOLATIONS** of the safety policy and **ARE NOT** supported in any official sunnypilot branches:
+
+!!! danger "Driver Monitoring"
+    - "Nerfing" or reducing monitoring parameters.
+
+!!! danger "Panda Safety"
+    - No preventing disengaging of <ins>**longitudinal control**</ins> (positive/negative acceleration) on brake pedal press.
+    - No auto re-engaging of <ins>**longitudinal control**</ins> (positive/negative acceleration) on brake pedal release.
+    - No disengaging on `CRUISE MAIN` in `OFF` state.

docs_sp/setup/read-before-installing.md

@@ -0,0 +1,20 @@
+# 🚨 Read Before Installing
+
+It is recommended to read the <u>**entire documentation**</u> before proceeding. This will ensure that you fully understand each added feature in sunnypilot. This also ensures that you are choosing the correct settings and branch for your car to have the best driving experience.
+
+!!! warning
+    By installing this software, you accept all responsibility for anything that might occur while you use it. sunnypilot and all contributors to sunnypilot are not liable.
+
+    **Use at your own risk.**
+
+## Installation
+
+Please refer to the [Recommended Branches](../branches/recommended-branches.md) to find your preferred/supported branch. This guide will assume you want to install the latest `release-c3` branch.
+
+You can install sunnypilot on your comma 3/3X using one of the following methods:
+
+- ### [URL Method (Directly on Device)](url-method.md)
+This method allows you to install sunnypilot directly from your device's screen using a provided URL. It's simple and user-friendly, requiring no additional tools or external devices.
+
+- ### [SSH Method (Command Line)](ssh-method.md)
+This method is for advanced users who prefer to use SSH to clone the sunnypilot repository and install it manually via the command line. It offeres greater control over the installation process.

docs_sp/setup/ssh-method.md

@@ -0,0 +1,28 @@
+# SSH Method
+
+If you are looking to install sunnypilot via SSH, run the following commands in an SSH terminal after connecting to your comma 3/3X:
+
+1. Navigate to `data` directory
+    ```sh 
+    cd /data
+    rm -rf openpilot
+    ```
+
+2. Clone sunnypilot
+
+    !!! example ""
+        `staging` branch is used in this step as an example.
+    ```sh
+    git clone -b staging --recurse-submodules https://github.com/sunnypilot/sunnypilot.git openpilot
+    ```
+
+3. Git LFS
+    ```sh
+    cd openpilot
+    git lfs pull
+    ```
+
+4. Reboot
+    ```sh
+    sudo reboot
+    ```

docs_sp/setup/url-method.md

@@ -0,0 +1,18 @@
+# URL Method
+
+The URL installation method can be done in two ways, depending on your device & if you already have sunnypilot installed.
+
+=== "sunnypilot not installed"
+
+    1. [Factory reset/uninstall](https://github.com/commaai/openpilot/wiki/FAQ#how-can-i-reset-the-device) the previous software if you have another software/fork installed.
+    2. After factory reset/uninstall, upon reboot, select `Custom Software` when given the option.
+    3. Input the **Installation URL** per [Recommended Branches](../branches/recommended-branches.md).
+    4. Complete the rest of the installation by following the onscreen instructions.
+
+
+=== "sunnypilot already installed"
+
+    1. On the comma 3/3X, go to `Settings` → `Software`.
+    2. At the `Download` option, press `CHECK`. This will fetch the list of latest branches from the sunnypilot repository on GitHub.
+    3. At the `Target Branch` option, press `SELECT` to open the `Target Branch` selector.
+    4. Scroll and select the **Desired Branch** per Recommended Branches.

docs_sp/stylesheets/style.css

@@ -0,0 +1,6 @@
+/**
+ * Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+ *
+ * This file is part of sunnypilot and is licensed under the MIT License.
+ * See the LICENSE.md file in the root directory for more details.
+ */

mkdocs-sp.yml

@@ -0,0 +1,150 @@
+site_name: sunnypilot docs
+repo_name: sunnypilot/sunnypilot
+repo_url: https://github.com/sunnypilot/sunnypilot/
+site_description: sunnypilot Documentation
+site_url: https://docs.sunnypilot.ai
+edit_uri: blob/new-docs/docs_sp
+
+exclude_docs: README.md
+
+strict: true
+docs_dir: docs_sp
+site_dir: docs_sp_site/
+
+theme:
+  name: material
+  palette:
+    - media: "(prefers-color-scheme)"
+      toggle:
+        icon: material/brightness-auto
+        name: Switch to light mode
+    - scheme: default
+      media: "(prefers-color-scheme: light)"
+      primary: deep purple
+      accent: teal
+      toggle:
+        icon: material/brightness-7
+        name: Switch to dark mode
+    - scheme: slate
+      media: "(prefers-color-scheme: dark)"
+      primary: deep purple
+      accent: teal
+      toggle:
+        icon: material/brightness-4
+        name: Switch to system preference
+  font:
+    text: Open Sans
+    code: Fira Code
+  logo: assets/sp_logo.svg
+  favicon: assets/sp_logo.svg
+  features:
+    - content.code.copy
+    - navigation.tabs
+    - navigation.tabs.sticky
+
+extra_css:
+  - stylesheets/style.css
+
+markdown_extensions:
+  - admonition
+  - attr_list
+  - def_list
+  - footnotes
+  - md_in_html
+  - tables
+  - pymdownx.details
+  - pymdownx.emoji:
+      emoji_index: !!python/name:material.extensions.emoji.twemoji
+      emoji_generator: !!python/name:material.extensions.emoji.to_svg
+  - pymdownx.highlight:
+      anchor_linenums: true
+      line_spans: __span
+      pygments_lang_class: true
+  - pymdownx.inlinehilite
+  - pymdownx.magiclink:
+      normalize_issue_symbols: true
+      repo_url_shorthand: true
+      user: sunnypilot
+      repo: sunnypilot
+  - pymdownx.snippets
+  - pymdownx.superfences
+  - pymdownx.tabbed:
+      alternate_style: true
+  - pymdownx.tasklist:
+      custom_checkbox: true
+      clickable_checkbox: true
+  - toc:
+      permalink: true
+
+plugins:
+  - git-authors:
+      show_email_address: false
+  - git-committers:
+      repository: sunnypilot/sunnypilot
+      branch: master
+      enabled: !ENV [CI, false]
+  - git-revision-date-localized:
+      enable_creation_date: true
+  - search
+  - redirects:
+      redirect_maps:
+        'index.md': 'getting-started/what-is-sunnypilot.md'
+
+extra:
+  social:
+    - icon: fontawesome/brands/github
+      link: https://github.com/sunnypilot/sunnypilot
+    - icon: fontawesome/brands/discord
+      link: https://discord.sunnypilot.ai
+#  analytics:
+#    provider: google
+#    property: !ENV GOOGLE_ANALYTICS_KEY
+#    feedback:
+#      title: Was this page helpful?
+#      ratings:
+#        - icon: material/emoticon-happy-outline
+#          name: This page was helpful
+#          data: 1
+#          note: >-
+#            Thanks for your feedback!
+#        - icon: material/emoticon-sad-outline
+#          name: This page could be improved
+#          data: 0
+#          note: >-
+#            Thanks for your feedback! Help us improve this page by
+#            using our <a href="..." target="_blank" rel="noopener">feedback form</a>.
+
+nav:
+  - Getting Started:
+      - What is sunnypilot?: getting-started/what-is-sunnypilot.md
+      - Use sunnypilot in a car: getting-started/use-sunnypilot-in-a-car.md
+      - Develop sunnypilot: getting-started/develop-sunnypilot.md
+      - Setup:
+          - 🚨 Read before installing 🚨: setup/read-before-installing.md
+          - Installation:
+              - URL Method: setup/url-method.md
+              - SSH Method: setup/ssh-method.md
+  - Features:
+      - Auto Lane Change: features/auto-lane-change.md
+      - Custom ACC Increments: features/custom-acc-increments.md
+      - Dynamic Experimental Control: features/dynamic-experimental-control.md
+      - Hyundai Longitudinal Tuning: features/hyundai-longitudinal-tuning.md
+      - Intelligent Cruise Button Management: features/icbm.md
+      - Modular Assistive Driving System: features/mads.md
+      - Neutral Network Lateral Control: features/nnlc.md
+      - Smart Cruise Control - Map: features/scc-m.md
+      - Smart Cruise Control - Vision: features/scc-v.md
+      - Speed Limit - Assist: features/speed-limit-assist.md
+  - Community:
+      - Contributing: community/CONTRIBUTING.md
+      - Workflow: community/WORKFLOW.md
+      - Reporting a bug: community/reporting-a-bug.md
+      - Reporting a docs issue: community/reporting-a-docs-issue.md
+      - Discord Community: https://discord.sunnypilot.ai
+  - Safety Information:
+      - Safety: SAFETY.md
+      - Prohibited safety modifications: safety-information/prohibited-safety-modifications.md
+  - References:
+      - Branches:
+          - Recommended Branches: branches/recommended-branches.md
+          - Branch Definitions: branches/definitions.md

pyproject.toml

@@ -80,6 +80,13 @@ docs = [
   "Jinja2",
   "natsort",
   "mkdocs",
+  "mkdocs-material",
+  "mkdocs-material-extensions",
+  "mkdocs-git-revision-date-localized-plugin",
+  "mkdocs-git-committers-plugin-2",
+  "mkdocs-git-authors-plugin",
+  "mkdocs-glightbox",
+  "mkdocs-redirects",
 ]
 
 testing = [

release/ci/Gemfile

@@ -0,0 +1,5 @@
+source "https://rubygems.org"
+gem "faraday"
+gem "faraday-retry"
+gem "faraday-multipart"
+gem "listen"

release/ci/lib/api.rb

@@ -0,0 +1,112 @@
+# frozen_string_literal: true
+
+module API
+  def self.client
+    @client ||=
+      Faraday.new(url: DOCS_TARGET) do |conn|
+        conn.request :multipart
+        conn.request :url_encoded
+        conn.request :retry,
+                     {
+                       methods: %i[get post delete put],
+                       retry_statuses: [429],
+                       max: 3,
+                       retry_block: ->(env:, options:, retry_count:, exception:, will_retry_in:) do
+                         puts "Rate limited... will retry in #{will_retry_in}s"
+                       end,
+                       exceptions: [Faraday::TooManyRequestsError]
+                     }
+        conn.response :json, content_type: "application/json"
+        conn.response :raise_error
+        conn.adapter Faraday.default_adapter
+        conn.headers["Api-Key"] = DOCS_API_KEY
+      end
+  end
+
+  def self.edit_post(post_id:, raw:, title: nil, category: nil)
+    if dry_run?
+      puts "  DRY RUN: skipping PUT /posts/#{post_id}"
+      return
+    end
+
+    params = {
+      post: {
+        raw: raw,
+        edit_reason: "Synced from github.com/discourse/discourse-developer-docs"
+      }
+    }
+    params[:title] = title if title
+    params[:category] = category if category
+    client.put("/posts/#{post_id}", params)
+  end
+
+  def self.create_topic(external_id:, raw:, category:, title:)
+    if dry_run?
+      puts "  DRY RUN: skipping POST /posts"
+      return
+    end
+    client.post("/posts", { title: title, raw: raw, external_id: external_id, category: category })
+  end
+
+  def self.trash_topic(topic_id:)
+    if dry_run?
+      puts "  DRY RUN: skipping DELETE /t/#{topic_id}.json"
+      return
+    end
+
+    client.delete("/t/#{topic_id}.json")
+  end
+
+  def self.fetch_current_state
+    result =
+      client.post(
+        "/admin/plugins/explorer/queries/#{DATA_EXPLORER_QUERY_ID}/run",
+        { params: { category_id: CATEGORY_ID.to_s }.to_json }
+      ).body
+
+    raise "Data explorer query failed" if result["success"] != true
+
+    if result["columns"] != %w[t_id first_p_id external_id title raw deleted_at is_index_topic]
+      raise "Data explorer query returned unexpected columns: #{result["columns"].inspect}"
+    end
+
+    result["rows"].map do |row|
+      {
+        topic_id: row[0],
+        first_post_id: row[1],
+        external_id: row[2],
+        title: row[3],
+        raw: row[4],
+        deleted_at: row[5],
+        is_index_topic: row[6]
+      }
+    end
+  end
+
+  def self.restore_topic(topic_id:)
+    path = "/t/#{topic_id}/recover.json"
+
+    if dry_run?
+      puts "  DRY RUN: skipping PUT #{path}"
+      return
+    end
+
+    client.put(path)
+  end
+
+  def self.upload_file(path)
+    if dry_run?
+      puts "  DRY RUN: skipping POST /uploads.json"
+      return { "short_url" => "upload://placeholder" }
+    end
+
+    client.post(
+      "/uploads.json",
+      { type: "composer", synchronous: true, file: Faraday::UploadIO.new(path, "image/png") }
+    ).body
+  end
+
+  def self.dry_run?
+    [nil, true].include?(DRY_RUN)
+  end
+end

release/ci/lib/local_doc.rb

@@ -0,0 +1,108 @@
+# frozen_string_literal: true
+
+Asset = Struct.new(:local_path, :local_sha1, :remote_short_url, keyword_init: true)
+
+class LocalDoc
+  attr_accessor :path,
+                :frontmatter,
+                :content,
+                :topic_id,
+                :first_post_id,
+                :remote_content,
+                :remote_title,
+                :remote_deleted,
+                :assets
+
+  def initialize(**kwargs)
+    kwargs.each { |k, v| send("#{k}=", v) }
+    self.assets ||= []
+  end
+
+  def external_id
+    "DOC-#{frontmatter["id"]}"
+  end
+
+  def section
+    path_segments = path.split("/")
+    path_segments[0] if path_segments.size > 1
+  end
+
+  def content_with_uploads
+    unused_assets = assets.dup
+
+    result =
+      content.gsub(/![^\]]+\]\(([^)]+)\)/) do |match|
+        path = $1
+        next match if !path.start_with?("/")
+
+        resolved = File.expand_path("#{__dir__}/../#{path}")
+        assets_dir = File.expand_path("#{__dir__}/../assets/")
+        raise "Invalid path: #{resolved}" if !resolved.start_with?(assets_dir)
+
+        digest = Digest::SHA1.file(resolved).hexdigest
+
+        asset = assets.find { |a| a.local_sha1 == digest }
+        unused_assets.delete(asset)
+        if !asset
+          puts "  Uploading #{path}..."
+          result = API.upload_file(resolved)
+          raise "File upload failed: #{result.inspect}" if !result["short_url"]
+          asset =
+            Asset.new(local_path: path, local_sha1: digest, remote_short_url: result["short_url"])
+          assets.push(asset)
+        end
+
+        short_url = asset.remote_short_url
+
+        match.gsub(path, short_url)
+      end
+
+    unused_assets.each { |asset| assets.delete(asset) }
+
+    result = <<~MD
+      #{result}
+
+      ---
+
+      <small>This document is version controlled - suggest changes [on github](https://github.com/sunnypilot/sunnypilot/blob/master/docs_sp/#{path}).</small>
+    MD
+
+    if assets.size == 0
+      result
+    else
+      <<~MD
+        #{result}
+        <!-- START DOCS ASSET MAP
+        #{serialized_assets}
+        END DOCS ASSET MAP -->
+      MD
+    end
+  end
+
+  def serialized_assets
+    JSON.pretty_generate(
+      assets.map do |asset|
+        {
+          local_path: asset.local_path,
+          local_sha1: asset.local_sha1,
+          remote_short_url: asset.remote_short_url
+        }
+      end
+    )
+  end
+
+  def remote_content=(value)
+    if value.match(/<!-- START DOCS ASSET MAP\n(.+?)\nEND DOCS ASSET MAP -->/m)
+      self.assets = JSON.parse($1).map { |raw_asset| Asset.new(**raw_asset) }
+    end
+
+    value.gsub!(/![^\]]+\]\(([^)]+)\)/) do |match|
+      url = $1
+      found_asset = assets.find { |a| a.remote_short_url == url }
+      match.sub!(path, found_asset.local_path) if found_asset
+      match
+    end
+
+    @remote_content = value
+  end
+end

release/ci/lib/util.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module Util
+  def self.parse_md(raw)
+    if match = raw.match(/\A---\s*\n(.+?)\n---\n?(.*)\z/m)
+      raw_frontmatter, content = match.captures
+      frontmatter = YAML.safe_load(raw_frontmatter)
+    else
+      content = raw
+      frontmatter = {}
+    end
+
+    [frontmatter, content]
+  end
+end

release/ci/sync_docs.rb

@@ -0,0 +1,762 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "yaml"
+require "faraday"
+require "faraday/retry"
+require "faraday/multipart"
+require "listen"
+require "json"
+require "digest"
+
+CATEGORY_ID = ENV["DOCS_CATEGORY_ID"].to_i
+DATA_EXPLORER_QUERY_ID = ENV["DOCS_DATA_EXPLORER_QUERY_ID"].to_i
+DOCS_TARGET = ENV["DOCS_TARGET"]
+DOCS_API_KEY = ENV["DOCS_API_KEY"]
+GEMINI_API_KEY = ENV["GEMINI_API_KEY"]
+
+VERBOSE = ARGV.include?("-v")
+WATCH = ARGV.include?("--watch")
+DRY_RUN = ARGV.include?("--dry-run")
+
+require_relative "lib/local_doc"
+require_relative "lib/api"
+require_relative "lib/util"
+
+# Gemini API client for title generation
+class GeminiClient
+  GEMINI_API_URL = "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash-lite:generateContent"
+  #GEMINI_API_URL = "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent"  
+#   MAX_REQUESTS_PER_MINUTE = 15
+  MAX_REQUESTS_PER_MINUTE = 9
+  RATE_LIMIT_WINDOW = 60 # seconds
+
+  def initialize(api_key)
+    @api_key = api_key
+    @request_timestamps = []
+    @mutex = Mutex.new
+  end
+
+  def generate_titles(file_path, content)
+    return nil unless @api_key
+
+    wait_for_rate_limit
+
+    prompt = build_prompt(file_path, content)
+    
+    response = Faraday.post(
+      "#{GEMINI_API_URL}?key=#{@api_key}",
+      { contents: [{ parts: [{ text: prompt }] }] }.to_json,
+      "Content-Type" => "application/json"
+    )
+
+    parse_response(response)
+  rescue => e
+    puts "Error calling Gemini API: #{e.message}"
+    nil
+  end
+
+  private
+
+  def wait_for_rate_limit
+    @mutex.synchronize do
+      now = Time.now
+      
+      # Remove timestamps older than the rate limit window
+      @request_timestamps.reject! { |ts| now - ts > RATE_LIMIT_WINDOW }
+      
+      # If we've hit the limit, wait until the oldest request expires
+      if @request_timestamps.length >= MAX_REQUESTS_PER_MINUTE
+        oldest_request = @request_timestamps.first
+        sleep_time = RATE_LIMIT_WINDOW - (now - oldest_request) + 0.1 # Add small buffer
+        
+        if sleep_time > 0
+          puts "Rate limit reached (#{MAX_REQUESTS_PER_MINUTE}/min). Waiting #{sleep_time.round(1)}s..."
+          sleep(sleep_time)
+          
+          # Clean up again after waiting
+          now = Time.now
+          @request_timestamps.reject! { |ts| now - ts > RATE_LIMIT_WINDOW }
+        end
+      end
+      
+      # Record this request
+      @request_timestamps << Time.now
+    end
+  end
+
+  def build_prompt(file_path, content)
+    <<~PROMPT
+      You are helping to generate documentation metadata. Given a markdown file path and its content, generate appropriate titles.
+
+      File path: #{file_path}
+      
+      Content preview:
+      #{content[0..500]}
+
+      Please analyze the file path and content, then provide:
+      1. A full title (3-8 words, descriptive and professional, MUST be at least 15 characters long)
+      2. A short title (1-3 words, concise, MUST be at least 15 characters long)
+
+      CRITICAL: Both titles MUST be at least 15 characters long. If a title would be shorter, expand it with relevant context.
+
+      Respond ONLY with valid JSON in this exact format:
+      {
+        "title": "Your Full Title Here",
+        "short_title": "Short Title Here"
+      }
+
+      Do not include any other text, explanation, or markdown formatting.
+    PROMPT
+  end
+
+  def parse_response(response)
+    body = JSON.parse(response.body)
+    text = body.dig("candidates", 0, "content", "parts", 0, "text")
+    
+    return nil unless text
+
+    # Extract JSON from potential markdown code blocks
+    json_text = text.strip.gsub(/^```json\n/, "").gsub(/\n```$/, "").strip
+    
+    parsed = JSON.parse(json_text)
+    
+    # Validate minimum length
+    if parsed["title"] && parsed["title"].length < 15
+      parsed["title"] = parsed["title"].ljust(15)
+    end
+    
+    if parsed["short_title"] && parsed["short_title"].length < 15
+      parsed["short_title"] = parsed["short_title"].ljust(15)
+    end
+    
+    parsed
+  rescue => e
+    puts "Error parsing Gemini response: #{e.message}"
+    nil
+  end
+end
+
+# Convert MkDocs "=== Tabs" sections to Obsidian callouts
+def convert_mkdocs_tabs_to_callouts(content, debug: false)
+  lines = content.lines
+  result = []
+  i = 0
+  match_count = 0
+
+  while i < lines.length
+    line = lines[i]
+
+    # Detect a MkDocs tab start, e.g., === "sunnypilot not installed"
+    if line =~ /^===\s+"([^"]+)"\s*$/
+      match_count += 1
+      tab_title = $1.strip
+
+      # Collect all indented lines following the tab
+      body_lines = []
+      i += 1
+      while i < lines.length
+        current_line = lines[i]
+        
+        # Check if line is indented (4+ spaces or tab)
+        if current_line =~ /^(?:    |\t)/
+          body_lines << current_line
+          i += 1
+        # Check if it's a blank line - peek ahead like we do for callouts
+        elsif current_line =~ /^\s*$/
+          peek_index = i + 1
+          has_more_indented = false
+          while peek_index < lines.length
+            if lines[peek_index] =~ /^(?:    |\t)/
+              has_more_indented = true
+              break
+            elsif lines[peek_index] =~ /^\s*$/
+              peek_index += 1
+            else
+              break
+            end
+          end
+          
+          if has_more_indented
+            body_lines << current_line
+            i += 1
+          else
+            break
+          end
+        else
+          # Non-indented, non-blank line (could be next tab!) - stop
+          break
+        end
+      end
+
+      puts "DEBUG: Tab '#{tab_title}', lines: #{body_lines.length}" if debug
+
+      # Convert the tab section to a callout
+      result << "> [!note] #{tab_title}\n"
+      body_lines.each do |body_line|
+        # Remove the first level of indentation (4 spaces or 1 tab) and add > prefix
+        stripped = body_line.sub(/^(?:    |\t)/, '')
+        if stripped.strip.empty?
+          result << ">\n"
+        else
+          result << ">#{stripped}"
+        end
+      end
+    else
+      result << line
+      i += 1
+    end
+  end
+
+  puts "DEBUG: Total tab matches: #{match_count}" if debug
+  result.join
+end
+
+
+# New implementation of MkDocs Material callout to Obsidian converter
+def convert_mkdocs_to_obsidian_callouts(content, debug: false)
+  # Map of MkDocs callout types to Obsidian equivalents
+  callout_map = {
+    'note' => 'note',
+    'tip' => 'tip',
+    'important' => 'important',
+    'warning' => 'warning',
+    'caution' => 'caution',
+    'info' => 'info',
+    'success' => 'success',
+    'question' => 'question',
+    'failure' => 'failure',
+    'danger' => 'danger',
+    'bug' => 'bug',
+    'example' => 'example',
+    'quote' => 'quote',
+    'abstract' => 'abstract',
+    'summary' => 'summary',
+    'tldr' => 'tldr'
+  }
+  
+  lines = content.lines
+  result = []
+  i = 0
+  match_count = 0
+  
+  while i < lines.length
+    line = lines[i]
+    
+    # Check if this line starts a callout (can be indented or not)
+    # Capture any leading indentation
+    if line =~ /^(\s*)!!!\s+(#{callout_map.keys.map { |k| Regexp.escape(k) }.join('|')})(?:\s+"([^"]*)")?\s*$/
+      leading_indent = $1
+      mkdocs_type = $2
+      custom_title = $3
+      match_count += 1
+      obsidian_type = callout_map[mkdocs_type]
+      
+      # Determine the base indentation level (number of spaces/tabs before !!!)
+      base_indent_size = leading_indent.length
+      
+      # Collect all lines that have MORE indentation than the base
+      body_lines = []
+      i += 1
+      while i < lines.length
+        current_line = lines[i]
+        
+        # Check if line has the required base indentation plus more (for body content)
+        # Body content should be indented relative to the !!! line
+        required_indent = leading_indent + "    " # base + 4 more spaces
+        alt_required_indent = leading_indent + "\t" # base + 1 tab
+        
+        if current_line.start_with?(required_indent) || current_line.start_with?(alt_required_indent)
+          body_lines << current_line
+          i += 1
+        # Check if line is blank - might be between paragraphs
+        elsif current_line =~ /^\s*$/
+          peek_index = i + 1
+          has_more_indented = false
+          while peek_index < lines.length
+            peek_line = lines[peek_index]
+            if peek_line.start_with?(required_indent) || peek_line.start_with?(alt_required_indent)
+              has_more_indented = true
+              break
+            elsif peek_line =~ /^\s*$/
+              peek_index += 1
+            else
+              break
+            end
+          end
+          
+          if has_more_indented
+            body_lines << current_line
+            i += 1
+          else
+            break
+          end
+        else
+          # Line doesn't have required indentation - stop
+          break
+        end
+      end
+      
+      puts "DEBUG: Match #{match_count} - indent: #{base_indent_size} spaces, type: #{mkdocs_type}, title: #{custom_title.inspect}, body lines: #{body_lines.length}" if debug
+      
+      # Build the converted callout, preserving the base indentation
+      if body_lines.empty?
+        if custom_title && !custom_title.empty?
+          result << "#{leading_indent}> [!#{obsidian_type}] \"#{custom_title}\"\n"
+        else
+          result << "#{leading_indent}> [!#{obsidian_type}]\n"
+        end
+      else
+        if custom_title && !custom_title.empty?
+          result << "#{leading_indent}> [!#{obsidian_type}] \"#{custom_title}\"\n"
+        else
+          result << "#{leading_indent}> [!#{obsidian_type}]\n"
+        end
+        # Add > prefix to each body line, removing the extra level of indentation
+        body_lines.each do |body_line|
+          # Remove the base indent + one level (4 spaces or tab)
+          stripped = body_line.sub(/^#{Regexp.escape(leading_indent)}(?:    |\t)/, '')
+          result << "#{leading_indent}>#{stripped}"
+        end
+      end
+    else
+      result << line
+      i += 1
+    end
+  end
+  
+  puts "DEBUG: Total matches found: #{match_count}" if debug
+  
+  result.join
+end
+
+# Convert MkDocs Material icons to standard emojis
+def convert_material_icons_to_emojis(content)
+  # Map of common Material icons to emoji equivalents
+  icon_map = {
+    # Check/success icons
+    ':material-check:' => '✅',
+    ':material-check-circle:' => '✅',
+    ':material-check-bold:' => '✅',
+    
+    # Close/error icons
+    ':material-close:' => '❌',
+    ':material-close-circle:' => '❌',
+    ':material-alert-circle:' => '⚠️',
+    
+    # Info icons
+    ':material-information:' => 'ℹ️',
+    ':material-information-outline:' => 'ℹ️',
+    ':material-help-circle:' => '❓',
+    
+    # Arrow icons
+    ':material-arrow-right:' => '→',
+    ':material-arrow-left:' => '←',
+    ':material-arrow-up:' => '↑',
+    ':material-arrow-down:' => '↓',
+    
+    # Other common icons
+    ':material-lightbulb:' => '💡',
+    ':material-star:' => '⭐',
+    ':material-heart:' => '❤️',
+    ':material-fire:' => '🔥',
+    ':material-flag:' => '🚩',
+    ':material-link:' => '🔗',
+    ':material-pencil:' => '✏️',
+    ':material-delete:' => '🗑️',
+    ':material-calendar:' => '📅',
+    ':material-clock:' => '🕐',
+    ':material-email:' => '📧',
+    ':material-phone:' => '📞',
+  }
+  
+  # Replace material icons with emojis, ignoring any style attributes
+  icon_map.each do |material_icon, emoji|
+    # Match the icon with optional style attributes like { style="color: #EF5350" }
+    content.gsub!(/#{Regexp.escape(material_icon)}\{\s*style="[^"]*"\s*\}/, emoji)
+    # Also match without style attributes
+    content.gsub!(material_icon, emoji)
+  end
+  
+  content
+end
+
+# Helper method to generate frontmatter from file path
+def generate_frontmatter_from_path(path, content = nil, gemini_client = nil)
+  # Remove .md extension and get the base name
+  base_name = File.basename(path, ".md")
+  
+  # Generate id from the full path (without extension)
+  # Replace / with -
+  # IMPORTANT: The LocalDoc#external_id method adds "DOC-" prefix (4 chars)
+  # So we need to limit the base ID to 46 chars to stay under the 50 char API limit
+  full_id = path.sub(/\.md$/, "").gsub("/", "-")
+  
+  # Maximum length for the base ID (50 char API limit - 4 char "DOC-" prefix)
+  max_base_id_length = 46
+  
+  if full_id.length > max_base_id_length
+    # Take first 37 chars and append an 8-char hash for uniqueness (37 + 1 dash + 8 = 46)
+    hash_suffix = Digest::MD5.hexdigest(path)[0..7]
+    id = "#{full_id[0..36]}-#{hash_suffix}"
+  else
+    id = full_id
+  end
+  
+  # Try to use Gemini for title generation
+  if gemini_client && content
+    gemini_titles = gemini_client.generate_titles(path, content)
+    
+    if gemini_titles
+      return {
+        "id" => id,
+        "title" => gemini_titles["title"],
+        "short_title" => gemini_titles["short_title"]
+      }
+    end
+  end
+  
+  # Fallback to original logic if Gemini fails or is not available
+  title = base_name.split(/[-_]/).map(&:capitalize).join(" ")
+  short_title = base_name.split(/[-_]/).map(&:capitalize).join(" ")
+  
+  # Ensure minimum length
+  title = title.ljust(15) if title.length < 15
+  short_title = short_title.ljust(15) if short_title.length < 15
+  
+  {
+    "id" => id,
+    "title" => title,
+    "short_title" => short_title
+  }
+end
+
+# Helper method to generate index.md content for a folder
+def generate_folder_index(folder_name)
+  # Convert folder name to a nice title (e.g., "my-folder" -> "My Folder")
+  title = folder_name.split(/[-_]/).map(&:capitalize).join(" ")
+  
+  "---\ntitle: #{title}\n---\n"
+end
+
+# Convert internal markdown links (.md) to Discourse topic links
+def rewrite_internal_links(content, docs)
+  require "uri"
+
+  content.gsub(/\]\(([^)]+\.md)(#[^)]+)?\)/) do |match|
+    raw_link = $1
+    anchor = $2 || ""
+    # Strip any ./ or ../ from the beginning, but preserve subfolders
+    normalized = raw_link.gsub(%r{^\./}, "").gsub(%r{^\.\./}, "")
+    # Remove trailing .md
+    normalized = normalized.gsub(/\.md$/, "")
+
+    # Try percent-decoding (handles %20 etc)
+    begin
+      normalized_decoded = URI.decode_www_form_component(normalized)
+    rescue
+      normalized_decoded = normalized
+    end
+
+    candidates = []
+
+    # Strategy 1: exact match against doc.path without .md
+    candidates += docs.select { |d| d.path.sub(/\.md$/, "") == normalized_decoded }
+
+    # Strategy 2: ends_with (useful if docs have a different root)
+    if candidates.empty?
+      candidates += docs.select { |d| d.path.end_with?("#{normalized_decoded}.md") }
+    end
+
+    # Strategy 3: match by basename (e.g., linking to index.md or same-named file in subfolder)
+    if candidates.empty?
+      basename = File.basename(normalized_decoded)
+      candidates += docs.select { |d| File.basename(d.path, ".md") == basename }
+    end
+
+    # Strategy 4: index.md handling — if link pointed to a folder/index.md, allow folder match
+    if candidates.empty? && normalized_decoded.end_with?("/index")
+      folder = normalized_decoded.sub(/\/index$/, "")
+      candidates += docs.select { |d| File.dirname(d.path) == folder && File.basename(d.path, ".md") == "index" }
+    end
+
+    # Pick the best candidate (prefer exact match)
+    target_doc =
+      if candidates.any?
+        # prefer exact equality if present
+        exact = candidates.find { |d| d.path.sub(/\.md$/, "") == normalized_decoded }
+        exact || candidates.first
+      else
+        nil
+      end
+
+    if target_doc && target_doc.topic_id
+      # Return a Discourse link preserving the anchor
+      "](/t/-/#{target_doc.topic_id}?silent=true#{anchor})"
+    else
+      if VERBOSE
+        puts "⚠️ rewrite_internal_links: unresolved '#{raw_link}' -> normalized='#{normalized_decoded}'"
+        # show up to 10 possible docs to help debugging
+        sample = docs.first(10).map(&:path).join(", ")
+        puts "  sample docs: #{sample}"
+      end
+      # Return original match unchanged so the link doesn't become invalid text
+      match
+    end
+  end
+end
+
+# Initialize Gemini client if API key is available
+gemini_client = GEMINI_API_KEY ? GeminiClient.new(GEMINI_API_KEY) : nil
+
+if gemini_client
+  puts "✓ Gemini API configured for title generation"
+else
+  puts "⚠ GEMINI_API_KEY not set - using fallback title generation"
+end
+
+docs = []
+
+puts "Reading local docs..."
+BASE = "#{__dir__}/../../docs_sp/"
+
+# Generate index.md for each folder that doesn't have one
+folders_needing_index = Set.new
+Dir.glob("**/", base: BASE).each do |folder|
+  next if folder == "./" || folder.empty?
+  
+  folder_path = folder.chomp("/")
+  index_path = File.join(BASE, folder_path, "index.md")
+  
+  unless File.exist?(index_path)
+    folders_needing_index.add(folder_path)
+    
+    # Get the folder name (last component of the path)
+    folder_name = File.basename(folder_path)
+    
+    # Generate the index.md content
+    index_content = generate_folder_index(folder_name)
+    
+    # Write the index.md file
+    File.write(index_path, index_content)
+    puts "Generated index.md for folder: #{folder_path}" if VERBOSE
+  end
+end
+
+puts "Generated #{folders_needing_index.size} index.md files" if folders_needing_index.any?
+
+Dir
+  .glob("**/*.md", base: BASE)
+  .each do |path|
+    next if path.end_with?("index.md")
+    next if path.include?("SAFETY")
+
+    content = File.read(File.join(BASE, path))
+
+    frontmatter, content = Util.parse_md(content)
+    
+    # Convert MkDocs Material callouts to Obsidian format
+    content = convert_mkdocs_tabs_to_callouts(content)
+    content = convert_mkdocs_to_obsidian_callouts(content)
+    content = convert_material_icons_to_emojis(content)
+
+    # Generate missing frontmatter fields dynamically
+    generated = generate_frontmatter_from_path(path, content, gemini_client)
+    
+    # Apply the generated values, ensuring ID is limited to 50 chars
+    frontmatter["id"] = generated["id"]
+    frontmatter["title"] ||= generated["title"]
+    frontmatter["short_title"] ||= generated["short_title"]
+
+    puts "Generated frontmatter for '#{path}': id='#{frontmatter["id"]}', title='#{frontmatter["title"]}'" if VERBOSE
+
+    docs.push(LocalDoc.new(frontmatter:, path:, content:))
+  end
+
+puts "Rewriting internal links..."
+docs.each do |doc|
+  doc.content = rewrite_internal_links(doc.content, docs)
+end
+
+puts "Validating local docs..."
+docs
+  .group_by { |doc| doc.external_id }
+  .each do |id, docs|
+    if docs.size > 1
+      puts "- duplicate external_id '#{id}' found in:"
+      docs.each { |doc| puts "- #{doc.path}" }
+      exit 1
+    end
+  end
+
+exit 0 if !DOCS_API_KEY
+
+puts "Fetching remote info via data-explorer..."
+remote_topics = API.fetch_current_state
+
+puts "Mapping to existing topics..."
+map_to_remote =
+  lambda do
+    docs.each do |doc|
+      puts "- checking '#{doc.external_id}'..." if VERBOSE
+      if topic_info = remote_topics.find { |t| t[:external_id] == doc.external_id }
+        doc.topic_id = topic_info[:topic_id]
+        doc.first_post_id = topic_info[:first_post_id]
+        doc.remote_title = topic_info[:title]
+        doc.remote_content = topic_info[:raw]
+        doc.remote_deleted = topic_info[:deleted_at]
+        puts "  found topic_id: #{doc.topic_id}" if VERBOSE
+      else
+        puts " not found" if VERBOSE
+      end
+    end
+  end
+map_to_remote.call
+
+puts "Deleting topics if necessary..."
+
+cat_desc_topic = remote_topics.find { |t| t[:is_index_topic] }
+if cat_desc_topic.nil?
+  puts "Docs category is missing an index topic"
+  exit 1
+end
+cat_desc_topic_id = cat_desc_topic[:topic_id]
+
+remote_topics
+  .reject { |remote_doc| remote_doc[:deleted_at] }
+  .reject { |remote_doc| docs.any? { |doc| doc.topic_id == remote_doc[:topic_id] } }
+  .reject { |remote_doc| remote_doc[:topic_id] == cat_desc_topic_id }
+  .each do |remote_doc|
+    id = remote_doc[:topic_id]
+    puts "- deleting topic #{id}..."
+    API.trash_topic(topic_id: id)
+  end
+
+puts "Restoring topics if necessary..."
+docs
+  .filter(&:remote_deleted)
+  .each do |doc|
+    puts "- restoring '#{doc.external_id}'..."
+    API.restore_topic(topic_id: doc.topic_id)
+  end
+
+puts "Creating missing topics..."
+created_any = false
+docs.each do |doc|
+  next if doc.topic_id
+
+  created_any = true
+  puts "- creating '#{doc.external_id} with title '#{doc.frontmatter["title"]}'..."
+  converted_content = convert_mkdocs_to_obsidian_callouts(doc.content_with_uploads)
+  API.create_topic(
+    external_id: doc.external_id,
+    raw: converted_content,
+    category: CATEGORY_ID,
+    title: doc.frontmatter["title"]
+  )
+rescue Faraday::UnprocessableEntityError => e
+  puts " 422 error: #{e.response[:body]}"
+  raise e
+end
+
+if created_any
+  puts "Re-fetching remote info..."
+  remote_topics = API.fetch_current_state
+  map_to_remote.call
+end
+
+puts "Updating content..."
+docs.each do |doc|
+  if doc.topic_id.nil?
+    next if DRY_RUN
+    raise "Topic ID not found for '#{doc.external_id}'. Something went wrong with creating it?"
+  end
+
+  # Convert callouts in the content before comparison and upload
+  converted_content = convert_mkdocs_to_obsidian_callouts(doc.content_with_uploads)
+
+  if converted_content.strip == doc.remote_content.strip &&
+       doc.frontmatter["title"] == doc.remote_title
+    puts "- no changes required for '#{doc.external_id}' (topic_id: #{doc.topic_id})" if VERBOSE
+    next
+  end
+
+  puts "- updating '#{doc.external_id}' (topic_id: #{doc.topic_id})... new title: '#{doc.frontmatter["title"]}'"
+  API.edit_post(
+    post_id: doc.first_post_id,
+    raw: converted_content,
+    title: doc.frontmatter["title"],
+    category: CATEGORY_ID
+  )
+rescue Faraday::UnprocessableEntityError => e
+  puts " 422 error: #{e.response[:body]}"
+  raise e
+end
+
+puts "Building index..."
+_, index_content = Util.parse_md(File.read("#{BASE}index.md"))
+index_content += "\n\n"
+docs
+  .group_by { |doc| doc.section }
+  .each do |section, section_docs|
+    if section
+      section_frontmatter, _ = Util.parse_md(File.read("#{BASE}#{section}/index.md"))
+      index_content += "## #{section_frontmatter["title"]}\n\n"
+    end
+
+    section_docs.each do |doc|
+      index_content +=
+        "- #{doc.frontmatter["short_title"]}: [#{doc.frontmatter["title"]}](/t/-/#{doc.topic_id}?silent=true)\n"
+    end
+    index_content += "\n"
+  end
+
+index_post_info = remote_topics.find { |t| t[:topic_id] == cat_desc_topic_id }
+
+if index_post_info[:raw].strip == index_content.strip
+  puts "- no changes required for index"
+else
+  puts "- updating index..."
+  API.edit_post(post_id: index_post_info[:first_post_id], raw: index_content)
+end
+
+if WATCH
+  puts "Watching for changes to files..."
+
+  Listen
+    .to("#{__dir__}/docs") do |modified, added, removed|
+      if added.size > 0 || removed.size > 0
+        puts "Files added/removed. Restarting sync..."
+        exec("ruby", "#{__dir__}/sync_docs", *ARGV)
+      end
+
+      modified.each do |path|
+        relative = path.sub(BASE, "")
+        doc = docs.find { |d| d.path == relative }
+        raise "Modified file not recognized: #{relative}" if doc.nil?
+
+        print "- updating '#{doc.external_id}' (topic_id: #{doc.topic_id})..."
+        new_frontmatter, new_content = Util.parse_md(File.read(path))
+        if %w[id short_title].any? { |key| doc.frontmatter[key] != new_frontmatter[key] }
+          puts "Frontmatter changed. Restarting sync..."
+          exec("ruby", "#{__dir__}/sync_docs", *ARGV)
+        end
+        doc.content, doc.frontmatter = new_content, new_frontmatter
+
+        # Convert callouts before uploading
+        converted_content = convert_mkdocs_to_obsidian_callouts(doc.content_with_uploads)
+
+        API.edit_post(
+          post_id: doc.first_post_id,
+          raw: converted_content,
+          title: doc.frontmatter["title"]
+        )
+        puts " done"
+      end
+    end
+    .start
+
+  sleep
+else
+  puts "Done."
+end