# Project export: TORQ

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2026
- Tagline: Turn any car into a self-driving vehicle with the Jetson Thor. Retrofit autonomy, enable rideshare, and deliver smart AI feedback. From real-time reflexes to deep reasoning.
- Devpost: https://devpost.com/software/torq
- GitHub: https://github.com/bryandong24/treehacks2026
- Demo: https://github.com/JoelGrayson/Treehacks2026
- Video: https://www.youtube.com/embed/zEDVGfCVhaw?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner ([NVIDIA] Edge AI Track)
- Team: 1 GitHub contributor(s) — Bryan Dong (26 commits)

## Devpost submission (written by the team)

### Overview

Our project is for retrofitting any existing car to make it self-driving. Self-driving should not just be limited to new vehicles, Teslas, and Waymos, but cars bought before self-driving was available too. In our project, we take a 2018 Honda Accord and give it the ability to drive itself using commercially available hardware and open-source software. Using our app, you can request a ride and the car will come to where you are and drive you there. How We Built It The central design problem is that driving requires two very different kinds of intelligence. Understanding a scene - recognizing that a pedestrian is about to cross, or that a lane is ending - requires slow, contextual reasoning over visual input. Actually holding a lane and applying the brakes smoothly requires fast, high-frequency control. Trying to do both in a single system forces a tradeoff between intelligence and reaction speed, so we separated them into two layers. The high-level reasoning layer runs NVIDIA's Alpamayo R1, a 10.5-billion parameter vision-language-action model. It takes in camera frames from a wide-angle and a telephoto camera along with a short history of the car's own motion, and produces high-level driving plans. Because Alpamayo is a language model at its core, it also generates natural language explanations of its decisions - the same model that decides to yield to oncoming traffic can tell a passenger why it's yielding. This dual capability is what powers the transparency features in our rider-facing iOS app. The low-level control layer is derived from sunnypilot, a fork of comma.ai's openpilot. It runs a vision model that processes camera frames at 20 Hz and a control loop that actuates steering and acceleration at 100 Hz. These fast reflexes handle the moment-to-moment driving - lane holding, smooth braking, correcting for disturbances - while the reasoning layer above sets the overall plan. Both layers, along with several supporting processes, run independently on our compute platform and communicate through comma.ai's cereal IPC messaging framework. To physically control the car, we use the comma.ai Red Panda, a bidirectional CAN bus adapter connected to the Honda through a vehicle-specific wiring harness. Our software translates driving commands into CAN bus frames that the Honda's systems understand — steering torque, acceleration, braking — and sends them to the Red Panda, which transmits them onto the car's internal network. Vehicle state flows back through the same path, keeping the software in sync with what the car is actually doing. The system runs on an NVIDIA Jetson AGX Thor, with cameras connected through the Holoscan Sensor Bridge, which routes uncompressed video directly into GPU memory for minimal latency processing. The iOS app serves as both a rideshare dispatch system and a transparency interface. Riders request pickups, and the car navigates to them. During the ride, the app displays a live feed of the system's reasoning — what it sees, what it's doing, and why — drawn directly from Alpamayo's language output. In a driverless context, this kind of visibility is a practical necessity. Challenges One of the most difficult challenges was striking the balance between a performant and optimized model and one that actually performed well. Our hope was to run Chain of Causation models in real time on the AGX Thor with around 10hz control frequency, but these models were well over 2B parameters which made it essentially impossible to run under the 100ms per cycle compute time limit. Especially given the memory bandwidth cap of 276GB/s this made memory loading time on autoregressive chain of causation models to be a massive bottleneck. Without a custom ptx kernel that would do essentially what Flash Attention did - completely removing multiple memory read and write steps to work around memory bottlenecks. The hardest part of this project was integration. Each individual component — the reasoning model, the control software, the CAN bus interface, the camera pipeline — works on its own. Getting them all to work together reliably was where the real difficulty lay. The most fundamental challenge was bridging the two timescales of our architecture. The reasoning model takes hundreds of milliseconds to process a scene. The control loop needs to respond every ten milliseconds. Designing the handoff so that slow plan updates translate into smooth, continuous actuation — without jerks or gaps — required careful work on buffering, timing, and interpolation between the two systems. Hardware integration was equally demanding. Our stack spans four ecosystems — NVIDIA, Lattice Semiconductor, comma.ai, and Honda — each with its own data formats, protocols, and assumptions. Sunnypilot was built for Android with comma.ai's own cameras; we had to adapt it to Linux, swap in different camera sensors with different calibrations, route data through the Holoscan bridge, and make it all talk to the same CAN bus interface. Every boundary between ecosystems was its own set of problems. Finally, we had to ensure that generating natural language explanations from the reasoning model never interfered with the driving task. The language output runs as a secondary, asynchronous process — useful for riders, but never in the critical path of vehicle control. The car, which is controlled by the Jetson Thor, uses MQTT to communicate with the iPhone app through an MQTT broker, which is a Google Compute Engine instance. Safety Building a system that physically controls a moving vehicle carries obvious responsibility, and safety considerations informed our architecture from the start. The Red Panda firmware validates every CAN frame before transmitting it to the car. Malformed or out-of-range commands are rejected at the hardware level before they ever reach the vehicle's systems. On the camera side, the Holoscan Sensor Bridge and Lattice FPGA board provide a deterministic data path from the IMX274 cameras over MIPI to GPU memory - there is no software bottleneck or unpredictable CPU scheduling in the way of incoming visual data, which reduces the risk of stale or dropped frames reaching the driving model. On the software side, sunnypilot inherits openpilot's safety model: the driver can always override the system by touching the steering wheel or pressing the brake, which immediately disengages autonomous control. The system monitors for driver attentiveness and will alert and disengage if the driver is unresponsive for too long. Our two-layer architecture also provides a natural safety boundary. The low-level controller operates independently of the reasoning model - if Alpamayo stalls or produces an unreasonable plan, the fast control loop continues to hold the lane and maintain safe following distance using its own visual perception. The reasoning layer can fail gracefully without the car losing basic control. For the rideshare context, the iOS app gives passengers direct access to the reasoning layer. Riders can see a live interpretation of the system's actions - why it's slowing down, why it chose a particular lane, what it's anticipating ahead. This goes beyond passive status updates: because Alpamayo is a language model, passengers can actually converse with the system, ask questions about its decisions, and provide feedback. If a rider prefers a different route or wants to understand why the car is taking a particular path, they can say so, and the model can process that input as part of its planning. The app also enhances navigation by surfacing the model's contextual awareness - not just turn-by-turn directions, but an understanding of traffic conditions, road geometry, and obstacles that inform routing decisions. The result is that riders don't just observe autonomy - they interact with it, understand its thinking, and have a channel to influence it. We treat this project as a research prototype, not a production deployment. All testing was conducted in controlled conditions with a safety driver behind the wheel at all times. What We Learned Hardware is hard. There's a bunch of firmware issues, driver issues, and just a bunch of other risks that pose challenges. We developed a working familiarity with the NVIDIA autonomous driving ecosystem - Alpamayo, Cosmos, Holoscan, and JetPack - and with the comma.ai open-source stack for vehicle control. The gap between a model that works in simulation and a car that physically turns its steering wheel is large, and it is almost entirely composed of integration engineering. Our goal is to distill Alpamayo down to a model small enough to run directly on the Jetson AGX Thor in real time. W For on-device inference, we are using Thunder Kittens and TensorRT-Edge-LLM to write hyper-optimized CUDA kernels targeting a quantized version of the distilled model in NVFP4 (4-bit floating point). This combination should allow us to hit real-time inference on the Thor's Blackwell GPU, as well as cheaper hardware. We should note that the main bottleneck we'll be solving with custom kernels is memory bandwidth, and not compute. Stay tuned for future updates.

## README (from the GitHub repository)

# treehacks2026

Autonomous vehicle platform running on the **NVIDIA Jetson AGX Thor** to drive a **Honda Bosch** vehicle. Built at TreeHacks 2026.

## Hardware

- **Compute**: NVIDIA Jetson AGX Thor (JetPack 7.1, CUDA 13.0, L4T R38.4.0)
- **Cameras**: Two IMX274 cameras via Holoscan Sensor Bridge — 90° FOV (road) and 120° FOV (wide)
- **IMU**: LSM6DSOX via Arduino → USB-UART at 104 Hz
- **GPS**: Adafruit Ultimate GPS FeatherWing via USB-UART at 10 Hz
- **Vehicle**: Honda Bosch platform via red panda OBD-II adapter

## Architecture

```
Cameras → CUDA warp/YUV (CuPy) → driving_vision.onnx → driving_policy.onnx → controlsd → pandad → CAN bus → Honda
                                                                                    ↑
IMU (104Hz) → locationd (Kalman) → livePose ────────────────────────────────────────┘
GPS (10Hz)  → navd (Valhalla offline routing) → NavDesire → desire_helper ──────────┘
```

## What We Built

### ONNX Runtime on CUDA
- Built a compatible onnxruntime wheel for Jetson Thor (aarch64, CUDA 13.0) using [jetson-containers](https://github.com/dusty-nv/jetson-containers)
- All three models run on GPU with `ORT_ENABLE_ALL` graph optimization + `EXHAUSTIVE` cuDNN algo search

| Model | Avg Latency | Output |
|---|---|---|
| `driving_vision.onnx` | 7.06 ms | [1,1576] fp16 |
| `driving_policy.onnx` | 0.93 ms | [1,1000] fp16 |
| `dmonitoring_model.onnx` | 3.83 ms | [1,551] fp16 |
| **Total driving pipeline** | **~8 ms** | Well within 50ms (20Hz) budget |

### CUDA Preprocessing
Replaced the entire OpenCL pipeline with CuPy CUDA kernels:
- `warpPerspective` CUDA RawKernel — bilinear interpolation perspective warp
- `loadyuv` — YUV420 channel packing via CuPy array slicing
- `DrivingModelFrame` / `MonitoringModelFrame` — full temporal buffer management

### ONNX-Based Model Runner
Replaced tinygrad/OpenCL model runner with ONNX Runtime + CUDA sessions for both the driving model and driver monitoring model.

### Camera Integration via Holoscan Sensor Bridge ([`holoscan-sensor-bridge/`](holoscan-sensor-bridge/))
Two IMX274 cameras are connected to the Jetson AGX Thor through a [Lattice CPNX100 Holoscan Sensor Bridge](https://www.latticesemi.com/products/developmentboardsandkits/certuspro-nx-sensor-to-ethernet-bridge-board) board. The FPGA bridges MIPI camera data to 10GbE UDP, which ConnectX NICs can write directly into GPU memory via RDMA.

**FPGA firmware flashing:**
The Lattice CPNX100 board requires its FPGA bitstream to be programmed before use. This is done from inside the Holoscan Sensor Bridge Docker container:
```bash
# 1. Connect ethernet from Jetson to the sensor bridge board (J6 for cam 0, J3 for cam 1)
# 2. Verify connectivity
ping 192.168.0.2

# 3. Launch the Holoscan Sensor Bridge Docker container
cd holoscan-sensor-bridge
xhost +
sh docker/demo.sh

# 4. Flash the FPGA bitstream to on-board SPI flash (~50 min)
#    Use --force if upgrading from an older bitstream version
program_lattice_cpnx100 scripts/manifest.yaml

# 5. Program the FPGA from the SPI flash (~1 min)
#    The board must be power-cycled after programming
```

The flash tool (`tools/program_lattice_cpnx100/`) programs both the CLNX17 (MIPI bridge) and CPNX100 (main 10GbE) FPGAs via SPI, with MD5 checksum verification and automatic firmware download.

**Camera pipeline:**
- Holoscan captures frames inside Docker, converts RGBA uint16 → NV12 uint8 via a CuPy CUDA kernel
- Frames are passed to the host through a lock-free `/dev/shm` ring buffer (4 slots, sequence-counter torn-read detection)
- Host-side `jetson_camerad` publishes via VisionIPC + cereal at 20 Hz
- Camera 0 (90° FOV) → road camera, Camera 1 (120° FOV) → wide camera

### IMU Integration
- LSM6DSOX IMU → Arduino (I2C) → CP2104 USB-UART → `/dev/IMU`
- Publishes accelerometer + gyroscope at 104 Hz
- Fully integrated with locationd Kalman filter: `sensorsOK`, `inputsOK`, `valid` all 100%

### GPS Integration
- Adafruit Ultimate GPS via USB-UART → `/dev/GPS`
- Publishes GPS location at 10 Hz with speed and bearing

### Navigation (Valhalla + NavDesire)
- Offline turn-by-turn routing using `pyvalhalla` with Stanford-area OSM data
- GPS map matching → maneuver tracking → desire inputs (turnRight, turnLeft, keepLeft, keepRight)
- Pipeline: GPS → navd → NavDesire → desire_helper → desire_pulse → driving_policy.onnx

### Mobile App — Ride Hailing ([`mobileApp/`](mobileApp/))
A native iOS app (SwiftUI) that lets users hail the autonomous vehicle, similar to the Waymo rider app. The phone and car communicate over MQTT via a VPS broker.

**Ride flow:**
1. User sets a destination and pickup location (search, recommended spots, or map pin)
2. App sends a `from-phone/command-car` MQTT message with pickup/destination coordinates
3. Car drives to pickup — app tracks the car's live GPS position on a map
4. Car sends `from-car/car-arrived` — user taps "Start Driving"
5. App sends `from-phone/start-ride` — car drives to destination
6. Car sends `from-car/ride-finished` — ride complete

**Features:**
- Real-time car location tracking via MQTT GPS updates
- MapKit route visualization and address autocomplete (scoped to Stanford campus)
- Ride phase UI (approaching → arrived → driving → reached destination)
- Car diagnostics tab with live MQTT message log and connection status
- Auto-reconnecting MQTT client (CocoaMQTT)

### H100 Cloud Server ([`mqtt-server/`](mqtt-server/))
We run an **NVIDIA H100 GPU instance on Google Cloud** (A3 machine type, 80GB HBM3) that serves as the central hub connecting the car, the mobile app, and the Alpamayo model.

**Server setup:**
The server runs three services:
1. **Mosquitto MQTT broker** (port 1883) — message bus connecting all components (car, phone, server)
2. **FastAPI application** (port 8000) — WebSocket endpoints for data ingestion and video relay
3. **Alpamayo R1 inference** — loaded at startup, runs periodically on buffered frames

```bash
# On the H100 instance
cd mqtt-server
pip install -r requirements.txt
PYTHONPATH=/path/to/alpamayo/src uvicorn server.main:app --host 0.0.0.0 --port 8000
```

**What the server does:**
- **Data ingestion** (`/ws/thor`): The Jetson Thor streams JPEG camera frames + ego-motion data (orientation quaternion, velocity vector from `livePose`) over a msgpack WebSocket at ~10 Hz. The server buffers these in a thread-safe ring buffer, selecting frames at ~100ms intervals for inference.
- **MQTT relay**: Subscribes to `from-phone/*` and `from-car/*` topics, forwards commands between phone and car (e.g. routing `from-phone/command-car` → `from-server/command-car`), and auto-accepts hail requests with the car's latest GPS position.
- **Alpamayo inference**: Every ~5 seconds, takes a snapshot of 4 buffered frames + 16 ego-motion history steps and runs Alpamayo R1 inference. Publishes Chain-of-Causation reasoning and trajectory predictions to MQTT.
- **Video relay** (`/ws/mobile/video`): Forwards the latest JPEG frame from the car to connected mobile clients at ~5 FPS.

### Cloud Inference with Alpamayo ([`alpamayo/`](alpamayo/))
We run [NVIDIA Alpamayo R1](https://huggingface.co/nvidia/Alpamayo-R1-10B) (10B parameter Vision-Language-Action model) on the H100 server to provide high-level scene reasoning alongside the on-device driving stack.

**How it works:**
1. The Jetson Thor streams camera frames + ego-motion data (orientation, velocity from `livePose`) to the H100 server over a WebSocket at ~10 Hz
2. The server buffers frames and runs Alpamayo inference every few seconds
3. Alpamayo produces two outputs:
   - **Chain-of-Causation (CoC) reasoning** — a natural language explanation of what the car sees, why it's making decisions, and causal relationships between scene elements (e.g. "The lead vehicle is braking because a pedestrian is crossing, so I should decelerate")
   - **Trajectory prediction** — 64 waypoints over a 6.4s horizon at 10 Hz, generated via flow-matching diffusion conditioned on the VLM's reasoning
4. Results are published over MQT

[README truncated for size]

## Detected evidence (automated analysis)

Indexed codebase: 7355 recognized source files, 84265 KB.
- C (language) — detected in the code
- C++ (language) — detected in the code
- CSS (language) — detected in the code
- FastAPI (technology) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- Hugging Face (technology) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- PyTorch (technology) — detected in the code
- Redis (technology) — detected in the code
- Rust (language) — detected in the code
- Swift (language) — detected in the code
- AI coding agent: Claude Code — evidence: config files committed to the repository
- AI coding agent: Codex — evidence: config files committed to the repository

## Codebase structure (from repository index)

### Files (120 of 8929)

```
.gitignore
.gitmodules
alpamayo/.gitignore
alpamayo/CONTRIBUTING.md
alpamayo/LICENSE
alpamayo/notebooks/clip_ids.parquet
alpamayo/notebooks/inference.ipynb
alpamayo/pyproject.toml
alpamayo/README.md
alpamayo/src/alpamayo_r1/__init__.py
alpamayo/src/alpamayo_r1/action_space/__init__.py
alpamayo/src/alpamayo_r1/action_space/action_space.py
alpamayo/src/alpamayo_r1/action_space/discrete_action_space.py
alpamayo/src/alpamayo_r1/action_space/unicycle_accel_curvature.py
alpamayo/src/alpamayo_r1/action_space/utils.py
alpamayo/src/alpamayo_r1/config.py
alpamayo/src/alpamayo_r1/diffusion/__init__.py
alpamayo/src/alpamayo_r1/diffusion/base.py
alpamayo/src/alpamayo_r1/diffusion/flow_matching.py
alpamayo/src/alpamayo_r1/geometry/rotation.py
alpamayo/src/alpamayo_r1/helper.py
alpamayo/src/alpamayo_r1/load_physical_aiavdataset.py
alpamayo/src/alpamayo_r1/models/action_in_proj.py
alpamayo/src/alpamayo_r1/models/alpamayo_r1.py
alpamayo/src/alpamayo_r1/models/base_model.py
alpamayo/src/alpamayo_r1/models/delta_tokenizer.py
alpamayo/src/alpamayo_r1/models/token_utils.py
alpamayo/src/alpamayo_r1/test_inference.py
jetson-containers/.dockerignore
jetson-containers/.editorconfig
jetson-containers/.env.default
jetson-containers/.github/dependabot.yml
jetson-containers/.github/ISSUE_TEMPLATE/bug-report.yml
jetson-containers/.github/ISSUE_TEMPLATE/config.yml
jetson-containers/.github/ISSUE_TEMPLATE/feature-request.yml
jetson-containers/.github/ISSUE_TEMPLATE/question.yml
jetson-containers/.github/workflows/build-monthly-packages.yml
jetson-containers/.github/workflows/build-push.yml
jetson-containers/.github/workflows/pr-on-dev.yml
jetson-containers/.github/workflows/scripts/detect_changed_packages.sh
jetson-containers/.github/workflows/scripts/generate-workflow.sh
jetson-containers/.github/workflows/scripts/README.md
jetson-containers/.github/workflows/scripts/step-analyze-results.sh
jetson-containers/.github/workflows/scripts/step-build-package.sh
jetson-containers/.github/workflows/scripts/step-clean-environment.sh
jetson-containers/.github/workflows/scripts/step-env-info.sh
jetson-containers/.github/workflows/scripts/step-git-status.sh
jetson-containers/.github/workflows/scripts/step-pre-checkout-cleanup.sh
jetson-containers/.github/workflows/scripts/step-system-info.sh
jetson-containers/.github/workflows/scripts/step-test-results-summary.sh
jetson-containers/.github/workflows/scripts/workflow-template.yml
jetson-containers/.github/workflows/sweep-build-matrix.yml
jetson-containers/.gitignore
jetson-containers/.pre-commit-config.yaml
jetson-containers/autotag
jetson-containers/build.sh
jetson-containers/CITATION.cff
jetson-containers/deprecated/auto_awq/build.sh
jetson-containers/deprecated/auto_awq/config.py
jetson-containers/deprecated/auto_awq/Dockerfile
jetson-containers/deprecated/auto_awq/install.sh
jetson-containers/deprecated/auto_awq/README.md
jetson-containers/deprecated/auto_awq/test.py
jetson-containers/deprecated/cuda/cccl/build.sh
jetson-containers/deprecated/cuda/cccl/config.py
jetson-containers/deprecated/cuda/cccl/Dockerfile
jetson-containers/deprecated/cuda/cccl/install.sh
jetson-containers/deprecated/cuda/cccl/test.py
jetson-containers/deprecated/cuda/cuda-python/build.sh
jetson-containers/deprecated/cuda/cuda-python/config.py
jetson-containers/deprecated/cuda/cuda-python/Dockerfile
jetson-containers/deprecated/cuda/cuda-python/install.sh
jetson-containers/deprecated/cuda/cuda-python/README.md
jetson-containers/deprecated/cuda/cuda-python/test_driver.py
jetson-containers/deprecated/cuda/cuda-python/test_runtime.py
jetson-containers/deprecated/cuda/cuda-python/utils.py
jetson-containers/deprecated/cuda/cuda/config.py
jetson-containers/deprecated/cuda/cuda/Dockerfile
jetson-containers/deprecated/cuda/cuda/Dockerfile.builtin
jetson-containers/deprecated/cuda/cuda/Dockerfile.pip
jetson-containers/deprecated/cuda/cuda/Dockerfile.samples
jetson-containers/deprecated/cuda/cuda/install-samples.sh
jetson-containers/deprecated/cuda/cuda/install.sh
jetson-containers/deprecated/cuda/cuda/README.md
jetson-containers/deprecated/cuda/cuda/test-samples.sh
jetson-containers/deprecated/cuda/cuda/test.sh
jetson-containers/deprecated/cuda/cudnn/config.py
jetson-containers/deprecated/cuda/cudnn/cudnn_frontend/build.sh
jetson-containers/deprecated/cuda/cudnn/cudnn_frontend/config.py
jetson-containers/deprecated/cuda/cudnn/cudnn_frontend/Dockerfile
jetson-containers/deprecated/cuda/cudnn/cudnn_frontend/install.sh
jetson-containers/deprecated/cuda/cudnn/cudnn_frontend/test.py
jetson-containers/deprecated/cuda/cudnn/Dockerfile
jetson-containers/deprecated/cuda/cudnn/test.sh
jetson-containers/deprecated/cuda/cudss/config.py
jetson-containers/deprecated/cuda/cudss/Dockerfile
jetson-containers/deprecated/cuda/cudss/install.sh
jetson-containers/deprecated/cuda/cudss/test.sh
jetson-containers/deprecated/cuda/cusparselt/config.py
jetson-containers/deprecated/cuda/cusparselt/Dockerfile
jetson-containers/deprecated/cuda/cusparselt/install.sh
jetson-containers/deprecated/cuda/cusparselt/test_cusparselt.cu
jetson-containers/deprecated/cuda/cusparselt/test.sh
jetson-containers/deprecated/cuda/cutensor/config.py
jetson-containers/deprecated/cuda/cutensor/Dockerfile
jetson-containers/deprecated/cuda/cutensor/install.sh
jetson-containers/deprecated/cuda/cutensor/test.sh
jetson-containers/deprecated/cuda/cutlass/build.sh
jetson-containers/deprecated/cuda/cutlass/config.py
jetson-containers/deprecated/cuda/cutlass/Dockerfile
jetson-containers/deprecated/cuda/cutlass/install.sh
jetson-containers/deprecated/cuda/cutlass/README.md
jetson-containers/deprecated/cuda/cutlass/test_runtime.py
jetson-containers/deprecated/cuda/gdrcopy/build.sh
jetson-containers/deprecated/cuda/gdrcopy/config.py
jetson-containers/deprecated/cuda/gdrcopy/Dockerfile
jetson-containers/deprecated/cuda/gdrcopy/install.sh
jetson-containers/deprecated/cuda/gdrcopy/README.md
jetson-containers/deprecated/cuda/gdrcopy/test.sh
jetson-containers/deprecated/cuda/nccl/build.sh
[8809 more files omitted for size]
```

### Dependencies

- alpamayo/pyproject.toml: accelerate@>=1.12.0, av@>=16.0.1, einops@>=0.8.1, flash-attn@>=2.8.3, hydra-colorlog@>=1.2.0, hydra-core@>=1.3.2, pandas@>=2.3.3, physical_ai_av@>=0.1.0, pillow@>=12.0.0, torch@==2.8.0, torchvision@>=0.23.0, transformers@==4.57.1
- jetson-containers/packages/diffusion/diffusion_policy/requirements.txt: accelerate, av@==16.0.1, boto3@==1.24.96, cffi@==1.15.1, click@==8.3.1, datasets, diffusers, dill@==0.3.5.1, dm-control@==1.0.9, einops@==0.8.1, gdown, gym, h5py, hydra-core@==1.2.0, imagecodecs, imageio@==2.22.0, imageio-ffmpeg@==0.4.7, ipykernel@==7.1.0, matplotlib@==3.6.1, mujoco, numcodecs@==0.16.5, psutil@==7.1.3, pybullet-svl@==3.1.6.4, pygame@==2.1.2, pymunk@==7.1.0, pytorchvideo@==0.1.5, ray[default,tune]@==2.51.1, scikit-image@==0.25.2, scikit-video@==1.1.11, scipy@==1.9.1, shapely@==1.8.4, tensorboard, tensorboardx, termcolor@==3.2.0, threadpoolctl@==3.1.0, tqdm@==4.67.1, wandb@==0.23.0, zarr@==3.1.5
- jetson-containers/packages/llm/llamaspeak/requirements.txt: flask, nvidia-riva-client, termcolor, tones, websockets
- jetson-containers/packages/llm/local_llm/requirements.txt: flask, getch, tabulate, termcolor, tqdm, websockets
- jetson-containers/packages/llm/sudonim/patches/250314/requirements.txt: absl-py, fastapi, pydantic, sse_starlette, starlette, uvicorn
- jetson-containers/packages/pytorch/requirements.txt: build[uv], expecttest@>=0.3.0, filelock, fsspec@>=0.8.5, hypothesis, jinja2, lintrunner, networkx@>=2.5.1, optree@>=0.13.0, psutil, sympy@>=1.13.3, typing-extensions@>=4.13.2, wheel
- jetson-containers/packages/vlm/vila-microservice/src/requirements.txt: fastapi@==0.121.3, flask, pillow, prometheus_client, pydantic-settings, redis, uvicorn[standard], websockets
- jetson-containers/requirements.txt: black@>=23.0.0, flake8@>=6.0.0, git@+https://github.com/Granulate/DockerHub-API.git, packaging@>=20.0, pre-commit@>=3.0.0, pyyaml@>=6, requests, tabulate, termcolor, wget
- mqtt-listen/requirements.txt: paho-mqtt@>=2.0.0
- mqtt-server/requirements.txt: fastapi, msgpack, paho-mqtt, Pillow, scipy, uvicorn[standard], websockets
- openpilot/msgq_repo/pyproject.toml: codespell, coverage, cppcheck, cpplint, Cython, lefthook, numpy, parameterized, pytest, pytest-retry, ruff, scons, setuptools, ty
- openpilot/opendbc_repo/pyproject.toml: cffi, codespell, comma-car-segments@@ https://huggingface.co/datasets/commaai/commaCarSegments/resolve/main/dist/comma_car_segments-0.1.0-py3-none-any.whl, cpplint, crcmod-plus, gcovr, hypothesis@==6.47.*, inputs, Jinja2, lefthook, matplotlib, numpy, parameterized@>=0.8,<0.9, pycapnp@==2.1.0, pycryptodome, pytest@==8.4.2, pytest-coverage, pytest-mock, pytest-randomly, pytest-subtests, pytest-xdist@@ git+https://github.com/sshane/pytest-xdist@2b4372bd62699fb412c4fe2f95bf9f01bd2018da, ruff, scons, tqdm, ty, zstandard
- openpilot/panda/pyproject.toml: cffi, flaky, libusb1, mypy, opendbc@@ git+https://github.com/commaai/opendbc.git@master#egg=opendbc, pycryptodome@>= 3.9.8, pytest, pytest-mock, pytest-randomly, pytest-timeout, ruff, scons, setuptools, spidev
- openpilot/pyproject.toml: aiohttp, aiortc, av, casadi@>=3.6.6, cffi, codespell, coverage, crcmod-plus, Cython, dearpygui@>=2.1.0, dictdiffer, hypothesis@==6.47.*, inputs, jeepney, Jinja2, json-rpc, libusb1, mapbox-earcut, matplotlib, metadrive-simulator@@ https://github.com/commaai/metadrive/releases/download/MetaDrive-minimal-0.4.2.4/metadrive_simulator-0.4.2.4-py3-none-any.whl, mkdocs, numpy@>=2.0, onnx@>= 1.14.0, opencv-python-headless, parameterized@>=0.8, <0.9, pre-commit-hooks, psutil, pyaudio, pyautogui, pycapnp@==2.1.0, pycryptodome, PyJWT, pyopenssl@< 24.3.0, pyserial, pytest, pytest-asyncio, pytest-cpp, pytest-mock, pytest-subtests, pytest-timeout, pytest-xdist@@ git+https://github.com/sshane/pytest-xdist@2b4372bd62699fb412c4fe2f95bf9f01bd2018da, pywinctl, pyzmq, qrcode, raylib@> 5.5.0.3, requests, ruff, scons, sentry-sdk, setproctitle, setuptools, sounddevice, spidev, sympy, tqdm, ty, websocket_client, xattr, zstandard
- openpilot/rednose_repo/requirements.txt: cffi, Cython, numpy, pre-commit, pytest, pytest-xdist, ruff, scipy, scons, sympy
- openpilot/teleoprtc_repo/pyproject.toml: aiohttp@>=3.7.0, aiortc@>=1.6.0, av@>=11.0.0,<13.0.0, numpy@>=1.19.0, parameterized@>=0.8, pre-commit, pytest, pytest-asyncio, pytest-xdist
- openpilot/tinygrad_repo/extra/remu/Cargo.toml: float-cmp@0.9.0, half@2.3.1, num-traits@0.2.17
- openpilot/tinygrad_repo/pyproject.toml: black, blobfile, boto3, bottle, capstone, ggml-python, hypothesis@>=6.148.9, influxdb3-python, librosa, markdown-callouts, markdown-exec[ansi], mkdocs, mkdocs-material, mkdocstrings[python], mypy@==1.19.1, networkx, nibabel, numba@>=0.55, numpy, numpy, numpy, onnx@==1.19.0, onnx2torch, onnxruntime, openai, opencv-python, pandas, pillow, pre-commit, pycocotools, pylint, pytest, pytest-split, pytest-timeout, pytest-xdist, ruff@==0.14.10, safetensors, sentencepiece, tabulate, tiktoken, tinygrad[testing_minimal], tinygrad[testing_unit], torch@==2.9.1, tqdm, transformers, triton-nightly@>=2.1.0.dev20231014192330, typeguard, typing-extensions, unicorn, z3-solver
- sunnypilot/msgq_repo/pyproject.toml: codespell, coverage, cppcheck, cpplint, Cython, lefthook, numpy, parameterized, pytest, pytest-retry, ruff, scons, setuptools, ty
- sunnypilot/opendbc_repo/pyproject.toml: cffi, codespell, comma-car-segments@@ https://huggingface.co/datasets/commaai/commaCarSegments/resolve/main/dist/comma_car_segments-0.1.0-py3-none-any.whl, cpplint, crcmod-plus, gcovr, hypothesis@==6.47.*, inputs, Jinja2, lefthook, matplotlib, numpy, parameterized@>=0.8,<0.9, pycapnp@==2.1.0, pycryptodome, pytest@==8.4.2, pytest-coverage, pytest-mock, pytest-randomly, pytest-subtests, pytest-xdist@@ git+https://github.com/sshane/pytest-xdist@2b4372bd62699fb412c4fe2f95bf9f01bd2018da, ruff, scons, tqdm, ty, zstandard
- sunnypilot/panda/pyproject.toml: cffi, flaky, libusb1, mypy, opendbc@@ git+https://github.com/sunnypilot/opendbc.git@master#egg=opendbc, pycryptodome@>= 3.9.8, pytest, pytest-mock, pytest-randomly, pytest-timeout, ruff, scons, setuptools, spidev
- sunnypilot/pyproject.toml: aiohttp, aiortc, av, casadi@>=3.6.6, cffi, codespell, coverage, crcmod-plus, Cython, dearpygui@>=2.1.0, dictdiffer, hypothesis@==6.47.*, inputs, jeepney, Jinja2, json-rpc, libusb1, mapbox-earcut, matplotlib, metadrive-simulator@@ https://github.com/commaai/metadrive/releases/download/MetaDrive-minimal-0.4.2.4/metadrive_simulator-0.4.2.4-py3-none-any.whl, mkdocs, numpy@>=2.0, onnx@>= 1.14.0, opencv-python-headless, parameterized@>=0.8, <0.9, pre-commit-hooks, psutil, pyaudio, pyautogui, pycapnp@==2.1.0, pycryptodome, PyJWT, pyopenssl@< 24.3.0, pyserial, pytest, pytest-asyncio, pytest-cpp, pytest-mock, pytest-subtests, pytest-timeout, pytest-xdist@@ git+https://github.com/sshane/pytest-xdist@2b4372bd62699fb412c4fe2f95bf9f01bd2018da, pywinctl, pyzmq, qrcode, raylib@> 5.5.0.3, requests, ruff, scons, sentry-sdk, setproctitle, setuptools, sounddevice, spidev, sympy, tqdm, ty, websocket_client, xattr, zstandard
- sunnypilot/rednose_repo/requirements.txt: cffi, Cython, numpy, pre-commit, pytest, pytest-xdist, ruff, scipy, scons, sympy
- sunnypilot/teleoprtc_repo/pyproject.toml: aiohttp@>=3.7.0, aiortc@>=1.6.0, av@>=11.0.0,<13.0.0, numpy@>=1.19.0, parameterized@>=0.8, pre-commit, pytest, pytest-asyncio, pytest-xdist
- sunnypilot/tinygrad_repo/extra/remu/Cargo.toml: float-cmp@0.9.0, half@2.3.1, num-traits@0.2.17

### Recent commits (newest first)

- Add H100 cloud server setup section to README
- Add Alpamayo on-device distillation roadmap to README
- Add Holoscan Sensor Bridge and FPGA flashing docs to README
- Add holoscan-sensor-bridge submodule
- Add Alpamayo cloud inference section to README
- Add mobile app section to README
- Add mobile app submodule
- Remove Treehacks2026 submodule (moving to mobileApp)
- Add Treehacks2026 submodule
- Jetson Thor CUDA/CuPy library path fixes
- new way of doing GPS
- gps working again
- Merge jetson-work into main
- Additional Jetson changes
- Add alpamayo and mqtt-server
- Jetson changes
- before going back
- before going back
- Jetson Thor port: nav, camera, GPS, and platform integration
- timestamps

## Key source files (fetched from GitHub, selected and truncated for size)

### openpilot/SECURITY.md

```markdown
# Security Policy

## Reporting a Vulnerability

Suspected vulnerabilities can be reported to both `adeeb@comma.ai` and `security@comma.ai`.

```

### sunnypilot/SECURITY.md

```markdown
# Security Policy

## Reporting a Vulnerability

Suspected vulnerabilities can be reported to both `adeeb@comma.ai` and `security@comma.ai`.

```

### mqtt-listen/requirements.txt

```
paho-mqtt>=2.0.0

```

### mqtt-server/requirements.txt

```
fastapi
uvicorn[standard]
websockets
paho-mqtt
msgpack
scipy
Pillow

```

### jetson-containers/requirements.txt

```
packaging>=20.0
pyyaml>=6
wget
requests
git+https://github.com/Granulate/DockerHub-API.git
tabulate
termcolor
black>=23.0.0
flake8>=6.0.0
pre-commit>=3.0.0

```

### jetson-containers/pyproject.toml

```
[tool.black]
line-length = 88
target-version = ['py38']
include = '\.pyi?$'
extend-exclude = '''
# A regex preceded with ^/ will apply only to files and directories
# in the root of the project.
^/docs/
'''
skip-string-normalization = true  # This allows single quotes
skip-magic-trailing-comma = false
preview = false

```

### alpamayo/pyproject.toml

```
[project]
name = "alpamayo_r1"
version = "0.1.0"
requires-python = "==3.12.*"
dependencies = [
  "accelerate>=1.12.0",
  "av>=16.0.1",
  "einops>=0.8.1",
  "hydra-colorlog>=1.2.0",
  "hydra-core>=1.3.2",
  "pandas>=2.3.3",
  "physical_ai_av>=0.1.0",
  "pillow>=12.0.0",
  "torch==2.8.0",
  "torchvision>=0.23.0",
  "transformers==4.57.1",
  "flash-attn>=2.8.3",
]

[build-system]
requires = ["uv_build>=0.9.7,<0.10.0"]
build-backend = "uv_build"

[dependency-groups]
dev = [
  "matplotlib>=3.10.7",
  "mediapy>=1.2.4",
  "ipykernel>=6.29.3",
  "ipywidgets>=8.1.8",
]

[tool.uv]
no-build-isolation-package = ["flash-attn"]

[tool.ruff]
line-length = 100

```

### jetson-containers/docker-compose.yaml

```yaml
# -----------------------------------------------------------------------------
# Docker Compose Stack: Local PyPI & APT Servers
#
# This stack provides:
#   - Multiple local PyPI servers (cu126, cu129, cu130) for Jetson CUDA variants
#     with a fallback to Jetson Community PyPI Registry (https://pypi.jetson-ai-lab.io)
#   - A local APT server for hosting debian packages (cu126, cu129, cu130)
#     with a dallback to Jetson Community APT Registry (https://apt.jetson-ai-lab.io)
#
# Designed for compatibility with:
#   - jetson-containers
#
# Usage:
#   - Local wheel/distribution packages are stored under /home/${SCP_UPLOAD_USER}/dist/pypi/{jpX,sbsa}/cuXXX
#   - Local debian packages are stored under /home/${SCP_UPLOAD_USER}/dist/apt/{jpX,sbsa}/cuXXX/{22.04,24.04}
#
# Services automatically fallback to the Jetson Community Registry when needed.
# -----------------------------------------------------------------------------

x-common: &common
  restart: unless-stopped
  healthcheck:
    interval: 30s
    timeout: 10s
    start_period: 10s
    retries: 3
    test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/').status"]

name: jetson-containers-local
services:
  pypi_cu126: # CUDA 12.6 / Orin (default)
    image: pypiserver/pypiserver:edge
    command: >
      run
      --overwrite
      --verbose
      --authenticate .
      --passwords .
      --hash-algo off
      --fallback-url https://pypi.jetson-ai-lab.io/jp6/cu126
      --server gunicorn
      /data/packages
    <<: *common
    container_name: pypi_cu126
    hostname: pypi_cu126
    volumes:
      - /home/${SCP_UPLOAD_USER}/dist/pypi/jp6/cu126:/data/packages:rw
    ports:
      - "8126:8080"

  pypi_cu129: # CUDA 12.9 / Orin
    image: pypiserver/pypiserver:edge
    command: >
      run
      --overwrite
      --verbose
      --authenticate .
      --passwords .
      --hash-algo off
      --fallback-url https://pypi.jetson-ai-lab.io/jp6/cu129
      --server gunicorn
      /data/packages
    <<: *common
    container_name: pypi_cu129
    hostname: pypi_cu129
    volumes:
      - /home/${SCP_UPLOAD_USER}/dist/pypi/jp6/cu129:/data/packages:rw
    ports:
      - "8129:8080"

  pypi_cu130: # CUDA 13.0 / Thor
    image: pypiserver/pypiserver:edge
    command: >
      run
      --overwrite
      --verbose
      --authenticate .
      --passwords .
      --hash-algo off
      --fallback-url https://pypi.jetson-ai-lab.io/sbsa/cu130
      --server gunicorn
      /data/packages
    <<: *common
    container_name: pypi_cu130
    hostname: pypi_cu130
    volumes:
      - /home/${SCP_UPLOAD_USER}/dist/pypi/sbsa/cu130:/data/packages:rw
    ports:
      - "8130:8080"

  apt-server:
    image: python:3-slim
    <<: *common
    container_name: apt-server
    hostname: apt-server
    command: python -m http.server 8080 --bind 0.0.0.0 --directory /dist/apt
    ports:
      - "${LOCAL_DIST_APT_PORT:-8034}:8080"
    volumes:
      - /home/${SCP_UPLOAD_USER}/dist/apt:/dist/apt:rw
```

### openpilot/pyproject.toml

```
[project]
name = "openpilot"
requires-python = ">= 3.12.3, < 3.13"
license = {text = "MIT License"}
version = "0.1.0"
description = "an open source driver assistance system"
authors = [
  {name = "Vehicle Researcher", email="user@comma.ai"}
]

dependencies = [
  # multiple users
  "sounddevice",  # micd + soundd
  "pyserial",     # pigeond + qcomgpsd
  "requests",     # many one-off uses
  "sympy",        # rednose + friends
  "crcmod-plus",  # cars + qcomgpsd
  "tqdm",         # cars (fw_versions.py) on start + many one-off uses

  # core
  "cffi",
  "scons",
  "pycapnp==2.1.0",
  "Cython",
  "setuptools",
  "numpy >=2.0",

  # body / webrtcd
  "aiohttp",
  "aiortc",
  # aiortc does not put an upper bound on pyopenssl and is now incompatible
  # with the latest release
  "pyopenssl < 24.3.0",
  "pyaudio",

  # panda
  "libusb1",
  "spidev; platform_system == 'Linux'",

  # modeld
  "onnx >= 1.14.0",

  # logging
  "pyzmq",
  "sentry-sdk",
  "xattr",  # used in place of 'os.getxattr' for macOS compatibility

  # athena
  "PyJWT",
  "json-rpc",
  "websocket_client",

  # acados deps
  "casadi >=3.6.6",  # 3.12 fixed in 3.6.6

  # joystickd
  "inputs",

  # these should be removed
  "psutil",
  "pycryptodome", # used in updated/casync, panda, body, and a test
  "setproctitle",

  # logreader
  "zstandard",

  # ui
  "raylib > 5.5.0.3",
  "qrcode",
  "mapbox-earcut",
  "jeepney",
]

[project.optional-dependencies]
docs = [
  "Jinja2",
  "mkdocs",
]

testing = [
  "coverage",
  "hypothesis ==6.47.*",
  "ty",
  "pytest",
  "pytest-cpp",
  "pytest-subtests",
  # https://github.com/pytest-dev/pytest-xdist/pull/1229
  "pytest-xdist @ git+https://github.com/sshane/pytest-xdist@2b4372bd62699fb412c4fe2f95bf9f01bd2018da",
  "pytest-timeout",
  "pytest-asyncio",
  "pytest-mock",
  "ruff",
  "codespell",
  "pre-commit-hooks",
]

dev = [
  "av",
  "dictdiffer",
  "matplotlib",
  "opencv-python-headless",
  "parameterized >=0.8, <0.9",
  "pyautogui",
  "pywinctl",
]

tools = [
  "metadrive-simulator @ https://github.com/commaai/metadrive/releases/download/MetaDrive-minimal-0.4.2.4/metadrive_simulator-0.4.2.4-py3-none-any.whl ; (platform_machine != 'aarch64')",
  "dearpygui>=2.1.0; (sys_platform != 'linux' or platform_machine != 'aarch64')", # not vended for linux aarch64
]

[project.urls]
Homepage = "https://github.com/commaai/openpilot"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = [ "." ]

[tool.hatch.metadata]
allow-direct-references = true

[tool.pytest.ini_options]
minversion = "6.0"
addopts = "--ignore=openpilot/ --ignore=opendbc/ --ignore=panda/ --ignore=rednose_repo/ --ignore=tinygrad_repo/ --ignore=teleoprtc_repo/ --ignore=msgq/  -Werror --strict-config --strict-markers --durations=10 -n auto --dist=loadgroup"
cpp_files = "test_*"
cpp_harness = "selfdrive/test/cpp_harness.py"
python_files = "test_*.py"
asyncio_default_fixture_loop_scope = "function"
#timeout = "30"  # you get this long by default
markers = [
  "slow: tests that take awhile to run and can be skipped with -m 'not slow'",
  "tici: tests that are only meant to run on the C3/C3X",
  "skip_tici_setup: mark test to skip tici setup fixture"
]
testpaths = [
  "common",
  "selfdrive",
  "system",
  "tools",
  "cereal",
]

[tool.codespell]
quiet-level = 3
# if you've got a short variable name that's getting flagged, add it here
ignore-words-list = "bu,ro,te,ue,alo,hda,ois,nam,nams,ned,som,parm,setts,inout,warmup,bumb,nd,sie,preints,whit,indexIn,ws,uint,grey,deque,stdio,amin,BA,LITE,atEnd,UIs,errorString,arange,FocusIn,od,tim,relA,hist,copyable,jupyter,thead,TGE,abl,lite"
builtin = "clear,rare,informal,code,names,en-GB_to_en-US"
skip = "./third_party/*, ./tinygrad/*, ./tinygrad_repo/*, ./msgq/*, ./panda/*, ./opendbc/*, ./opendbc_repo/*, ./rednose/*, ./rednose_repo/*, ./teleoprtc/*, ./teleoprtc_repo/*, *.po, uv.lock, *.onnx, ./cereal/gen/*, */c_generated_code/*, docs/assets/*, tools/plotjuggler/layouts/*, selfdrive/assets/offroad/mici_fcc.html"

# https://docs.astral.sh/ruff/configuration/#using-pyprojecttoml
[tool.ruff]
indent-width = 2
lint.select = [
  "E", "F", "W", "PIE", "C4", "ISC", "A", "B",
  "NPY", # numpy
  "UP",  # pyupgrade
  "TRY203", "TRY400", "TRY401", # try/excepts
  "RUF008", "RUF100",
  "TID251",
  "PLE", "PLR1704",
]
lint.ignore = [
  "E741",
  "E402",
  "C408",
  "ISC003",
  "B027",
  "B024",
  "NPY002",  # new numpy random syntax is worse
  "UP045", "UP007",  # these don't play nice with raylib atm
]
line-length = 160
target-version ="py311"
exclude = [
  "body",
  "cereal",
  "panda",
  "opendbc",
  "opendbc_repo",
  "rednose_repo",
  "tinygrad_repo",
  "teleoprtc",
  "teleoprtc_repo",
  "third_party",
  "*.ipynb",
  "generated",
]
lint.flake8-implicit-str-concat.allow-multiline = false

[tool.ruff.lint.flake8-tidy-imports.banned-api]
"selfdrive".msg = "Use openpilot.selfdrive"
"common".msg = "Use openpilot.common"
"system".msg = "Use openpilot.system"
"third_party".msg = "Use openpilot.third_party"
"tools".msg = "Use openpilot.tools"
"pytest.main".msg = "pytest.main requires special handling that is easy to mess up!"
"unittest".msg = "Use pytest"
"time.time".msg = "Use time.monotonic"

# raylib banned APIs
"pyray.measure_text_ex".msg = "Use openpilot.system.ui.lib.text_measure"
"pyray.is_mouse_button_pressed".msg = "This can miss events. Use Widget._handle_mouse_press"
"pyray.is_mouse_button_released".msg = "This can miss events. Use Widget._handle_mouse_release"
"pyray.draw_text".msg = "Use a function (such as rl.draw_font_ex) that takes font as an argument"

[tool.ruff.format]
quote-style = "preserve"

[tool.ty.src]
exclude = [
  "cereal/",
  "msgq/",
  "msgq_repo/",
  "opendbc/",
  "opendbc_repo/",
  "panda/",
  "rednose/",
  "rednose_repo/",
  "tinygrad/",
  "tinygrad_repo/",
  "teleoprtc/",
  "teleoprtc_repo/",
  "third_party/",
]

[tool.ty.rules]
# Ignore unresolved imports for Cython-compiled modules (.pyx)
unresolved-import = "ignore"
#
[truncated — 928 more characters]
```

### sunnypilot/pyproject.toml

```
[project]
name = "openpilot"
requires-python = ">= 3.12.3, < 3.13"
license = {text = "MIT License"}
version = "0.1.0"
description = "an open source driver assistance system"
authors = [
  {name = "Vehicle Researcher", email="user@comma.ai"}
]

dependencies = [
  # multiple users
  "sounddevice",  # micd + soundd
  "pyserial",     # pigeond + qcomgpsd
  "requests",     # many one-off uses
  "sympy",        # rednose + friends
  "crcmod-plus",  # cars + qcomgpsd
  "tqdm",         # cars (fw_versions.py) on start + many one-off uses

  # core
  "cffi",
  "scons",
  "pycapnp==2.1.0",
  "Cython",
  "setuptools",
  "numpy >=2.0",

  # body / webrtcd
  "aiohttp",
  "aiortc",
  # aiortc does not put an upper bound on pyopenssl and is now incompatible
  # with the latest release
  "pyopenssl < 24.3.0",
  "pyaudio",

  # panda
  "libusb1",
  "spidev; platform_system == 'Linux'",

  # modeld
  "onnx >= 1.14.0",

  # logging
  "pyzmq",
  "sentry-sdk",
  "xattr",  # used in place of 'os.getxattr' for macOS compatibility

  # athena
  "PyJWT",
  "json-rpc",
  "websocket_client",

  # acados deps
  "casadi >=3.6.6",  # 3.12 fixed in 3.6.6

  # joystickd
  "inputs",

  # these should be removed
  "psutil",
  "pycryptodome", # used in updated/casync, panda, body, and a test
  "setproctitle",

  # logreader
  "zstandard",

  # ui
  "raylib > 5.5.0.3",
  "qrcode",
  "mapbox-earcut",
  "jeepney",
]

[project.optional-dependencies]
docs = [
  "Jinja2",
  "mkdocs",
]

testing = [
  "coverage",
  "hypothesis ==6.47.*",
  "ty",
  "pytest",
  "pytest-cpp",
  "pytest-subtests",
  # https://github.com/pytest-dev/pytest-xdist/pull/1229
  "pytest-xdist @ git+https://github.com/sshane/pytest-xdist@2b4372bd62699fb412c4fe2f95bf9f01bd2018da",
  "pytest-timeout",
  "pytest-asyncio",
  "pytest-mock",
  "ruff",
  "codespell",
  "pre-commit-hooks",
]

dev = [
  "av",
  "dictdiffer",
  "matplotlib",
  "opencv-python-headless",
  "parameterized >=0.8, <0.9",
  "pyautogui",
  "pywinctl",
]

tools = [
  "metadrive-simulator @ https://github.com/commaai/metadrive/releases/download/MetaDrive-minimal-0.4.2.4/metadrive_simulator-0.4.2.4-py3-none-any.whl ; (platform_machine != 'aarch64')",
  "dearpygui>=2.1.0; (sys_platform != 'linux' or platform_machine != 'aarch64')", # not vended for linux aarch64
]

[project.urls]
Homepage = "https://github.com/commaai/openpilot"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = [ "." ]

[tool.hatch.metadata]
allow-direct-references = true

[tool.pytest.ini_options]
minversion = "6.0"
addopts = "--ignore=openpilot/ --ignore=opendbc/ --ignore=panda/ --ignore=rednose_repo/ --ignore=tinygrad_repo/ --ignore=teleoprtc_repo/ --ignore=msgq/  -Werror --strict-config --strict-markers --durations=10 -n auto --dist=loadgroup"
cpp_files = "test_*"
cpp_harness = "selfdrive/test/cpp_harness.py"
python_files = "test_*.py"
asyncio_default_fixture_loop_scope = "function"
#timeout = "30"  # you get this long by default
markers = [
  "slow: tests that take awhile to run and can be skipped with -m 'not slow'",
  "tici: tests that are only meant to run on the C3/C3X",
  "skip_tici_setup: mark test to skip tici setup fixture"
]
testpaths = [
  "common",
  "selfdrive",
  "system",
  "tools",
  "cereal",
  "sunnypilot",
]

[tool.codespell]
quiet-level = 3
# if you've got a short variable name that's getting flagged, add it here
ignore-words-list = "bu,ro,te,ue,alo,hda,ois,nam,nams,ned,som,parm,setts,inout,warmup,bumb,nd,sie,preints,whit,indexIn,ws,uint,grey,deque,stdio,amin,BA,LITE,atEnd,UIs,errorString,arange,FocusIn,od,tim,relA,hist,copyable,jupyter,thead,TGE,abl,lite"
builtin = "clear,rare,informal,code,names,en-GB_to_en-US"
skip = "./third_party/*, ./tinygrad/*, ./tinygrad_repo/*, ./msgq/*, ./panda/*, ./opendbc/*, ./opendbc_repo/*, ./rednose/*, ./rednose_repo/*, ./teleoprtc/*, ./teleoprtc_repo/*, *.po, uv.lock, *.onnx, ./cereal/gen/*, */c_generated_code/*, docs/assets/*, tools/plotjuggler/layouts/*, selfdrive/assets/offroad/mici_fcc.html"

# https://docs.astral.sh/ruff/configuration/#using-pyprojecttoml
[tool.ruff]
indent-width = 2
lint.select = [
  "E", "F", "W", "PIE", "C4", "ISC", "A", "B",
  "NPY", # numpy
  "UP",  # pyupgrade
  "TRY203", "TRY400", "TRY401", # try/excepts
  "RUF008", "RUF100",
  "TID251",
  "PLE", "PLR1704",
]
lint.ignore = [
  "E741",
  "E402",
  "C408",
  "ISC003",
  "B027",
  "B024",
  "NPY002",  # new numpy random syntax is worse
  "UP045", "UP007",  # these don't play nice with raylib atm
]
line-length = 160
target-version ="py311"
exclude = [
  "body",
  "cereal",
  "panda",
  "opendbc",
  "opendbc_repo",
  "rednose_repo",
  "tinygrad_repo",
  "teleoprtc",
  "teleoprtc_repo",
  "third_party",
  "*.ipynb",
  "generated",
]
lint.flake8-implicit-str-concat.allow-multiline = false

[tool.ruff.lint.flake8-tidy-imports.banned-api]
"selfdrive".msg = "Use openpilot.selfdrive"
"common".msg = "Use openpilot.common"
"system".msg = "Use openpilot.system"
"third_party".msg = "Use openpilot.third_party"
"tools".msg = "Use openpilot.tools"
"pytest.main".msg = "pytest.main requires special handling that is easy to mess up!"
"unittest".msg = "Use pytest"
"time.time".msg = "Use time.monotonic"

# raylib banned APIs
"pyray.measure_text_ex".msg = "Use openpilot.system.ui.lib.text_measure"
"pyray.is_mouse_button_pressed".msg = "This can miss events. Use Widget._handle_mouse_press"
"pyray.is_mouse_button_released".msg = "This can miss events. Use Widget._handle_mouse_release"
"pyray.draw_text".msg = "Use a function (such as rl.draw_font_ex) that takes font as an argument"

[tool.ruff.format]
quote-style = "preserve"

[tool.ty.src]
exclude = [
  "cereal/",
  "msgq/",
  "msgq_repo/",
  "opendbc/",
  "opendbc_repo/",
  "panda/",
  "rednose/",
  "rednose_repo/",
  "tinygrad/",
  "tinygrad_repo/",
  "teleoprtc/",
  "teleoprtc_repo/",
  "third_party/",
]

[tool.ty.rules]
# Ignore unresolved imports for Cython-compiled modules (.pyx)
unresolved-imp
[truncated — 944 more characters]
```

[6777 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]