# Project export: Tempus This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: AI-powered cellular intelligence for faster, more accurate analysis. - Devpost: https://devpost.com/software/tempus-wvheua - GitHub: https://github.com/yordanoskassa/tempus - Video: https://www.youtube.com/embed/lhRbN7fKgU4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 1 GitHub contributor(s) — Yordanos (38 commits) ## Devpost submission (written by the team) ### Overview A Real-Time Veterinary Microscopy Copilot Tempus transforms a Raspberry Pi 5 running QNX into an intelligent microscopy workstation. It captures live specimens, detects blood cells, measures visible structures, and gives veterinary laboratory technicians a dedicated interface for reviewing and correcting results. Those human-verified corrections create a feedback loop that can continuously improve future versions of the model. Tempus does not just analyze specimens—it learns from the people who understand them. ### Inspiration Veterinary microscopy remains highly manual. Laboratory technicians inspect specimens, count cells, document observations, and transfer their findings between disconnected systems. This takes time, introduces inconsistency, and makes collaboration difficult. We asked: Could an affordable, real-time microscope assistant accelerate this workflow while still keeping the laboratory technician in control? That question became Tempus. What It Does Tempus can: Stream live microscope imagery Detect red blood cells, white blood cells, and platelets Draw real-time detection overlays Count visible cellular structures Measure cell area and specimen coverage Describe basic morphology Generate AI-assisted observations Let technicians confirm, reject, or correct detections Save verified annotations for future model training Preserve findings for reports and historical comparison Tempus is a decision-support and documentation platform—not an autonomous diagnostic system. System Architecture The Raspberry Pi and QNX handle deterministic camera acquisition and hardware control. The connected workstation currently runs the YOLO model, OpenCV analysis, API, and Tempus dashboard. A human-in-the-loop feedback system allows laboratory technicians to verify the model’s output before corrections are included in a future training dataset. Data Flow QNX captures a raw NV12 camera frame. A custom C bridge packages the frame and its metadata. The frame travels through an authenticated SSH tunnel. FastAPI converts it into an OpenCV image. YOLO detects visible blood cells. OpenCV calculates measurements and morphology features. Tempus displays the results to the technician. The technician confirms, rejects, or corrects each result. Verified corrections are saved as training annotations. Curated annotations improve future model versions. Human Feedback Loop The laboratory technician remains the final authority. The review page is designed to let a technician: Confirm a correct detection Change an incorrect cell class Adjust a bounding box Remove a false detection Add a cell the model missed Attach a note to an unusual structure Mark the specimen as unsuitable for training Corrections are not immediately learned from blindly. They first enter a curated dataset where they can be reviewed, anonymized, versioned, and validated before retraining. This prevents a single mistake from silently changing the production model. Why the Feedback Loop Matters Blood-cell appearance varies across: Species Staining methods Microscope optics Camera settings Lighting conditions Specimen quality Clinical environments A model trained on one dataset may not generalize perfectly to every veterinary clinic. Tempus turns normal laboratory review into a structured learning opportunity. Over time, verified corrections can help the system become more representative of real veterinary specimens and local laboratory conditions. The AI assists the technician, and the technician improves the AI. Native QNX Camera Pipeline Tempus uses QNX 8.0 on a Raspberry Pi 5 to control the camera and embedded hardware. The QNX layer handles: Sony IMX708 sensor initialization 2304×1296 NV12 frame capture Camera buffers and timestamps Y and UV image planes GPIO control SPI communication Experimental local display output Python camera libraries were unavailable on the QNX image, so the acquisition system was implemented using QNX’s native C camera API. Secure Frame Transport The camera produces raw NV12 data rather than browser-ready images. We created a binary protocol containing: Frame identifier Width and height Y-plane stride UV-plane offset UV-plane stride Pixel format Payload size Raw image data Frames travel through an authenticated SSH tunnel rather than an exposed camera service. Blood-Cell Detection Tempus uses a custom YOLO model to identify: RBC WBC Platelet The model produces: Bounding boxes Cell classifications Confidence scores Total cell counts Counts by class OpenCV provides additional measurements, including: Visible area Specimen coverage Circularity Aspect ratio Solidity Contour irregularity These measurements provide descriptive visual information without claiming medical diagnosis. Technician Experience The Tempus dashboard provides: Live microscope video Detection overlays RBC, WBC, and platelet counts Confidence controls Area and coverage measurements Morphology summaries Capture and report controls The technician review page adds: Detection confirmation Class correction Bounding-box editing False-positive removal Missed-cell annotation Review notes Training-data consent controls Embedded Display Tempus also includes an experimental display path for a two-inch 240×320 ST7789 screen. The QNX display driver uses: SPI communication GPIO-controlled reset Data/command signaling Backlight control NV12-to-RGB565 conversion Real-time image downscaling Built With QNX 8.0, Raspberry Pi 5, Raspberry Pi Camera Module 3, Sony IMX708, QNX Sensor Framework, C, Python, FastAPI, WebSockets, OpenCV, Ultralytics YOLO, NumPy, React, Vite, SSH tunneling, SPI, GPIO, ST7789 display The Original Microscope Camera Challenge Our original AmScope MU1000 camera used a proprietary ToupTek USB protocol instead of standard UVC. We successfully: Identified its USB endpoints Queried its firmware and hardware versions Read its calibration EEPROM Reproduced its vendor handshake Received the expected 0x08 acknowledgement Full streaming still required an undocumented model-specific sensor initialization sequence. We completed the prototype using the officially supported Camera Module 3 while preserving the ToupTek work for a future native QNX driver. Challenges QNX Camera Integration Python camera libraries were unavailable, so we built a native C acquisition bridge using the QNX camera API. Raw Image Reconstruction The camera produced NV12 buffers with platform-specific offsets and strides. Tempus reconstructs each frame using the metadata supplied by QNX. Secure Networking Direct access to the custom frame port was unavailable. We transported frames through an authenticated SSH tunnel. Real-Time AI Camera acquisition and AI inference have different performance requirements. Separating them keeps QNX focused on reliable hardware control while the workstation handles inference. Trustworthy Feedback Human corrections must not be used as training data without review. Tempus separates technician feedback, dataset curation, model training, validation, and release. Accomplishments Booted QNX 8.0 on Raspberry Pi 5 Enabled live IMX708 camera capture Built a native C frame bridge Reconstructed full-color NV12 frames Connected the feed to a blood-cell YOLO model Created real-time detection analytics Built an interactive veterinary dashboard Designed a technician feedback loop Implemented an experimental ST7789 driver Reverse-engineered the initial ToupTek protocol What’s Next Finish the technician annotation interface Add authenticated reviewer accounts Store consented and anonymized specimens Add annotation quality checks Track dataset and model versions Measure model improvement across feedback cycles Run ONNX inference directly on QNX Complete the AmScope MU1000 driver Train on veterinary-specific species and specimens Add calibrated micrometer measurements Validate results with veterinary professionals Export findings into patient records Generate PDF reports Compare historical specimens Potential Impact Tempus points toward an affordable microscopy platform for: Faster laboratory documentation More consistent cell counting Remote collaboration Structured veterinary datasets Continuous expert-guided improvement Better access for smaller veterinary practices The most important part of Tempus is not that an AI can make predictions. It is that every prediction can be reviewed, corrected, and transformed into better knowledge. Tempus is a microscope, an AI assistant, and a learning system built around human expertise. Responsible Use Tempus is an experimental decision-support platform. It does not provide medical diagnoses and must not replace professional veterinary judgment, validated laboratory equipment, or appropriate clinical testing. Technician feedback should be curated and validated before being used for model training. Patient and clinic information must be protected through appropriate consent, access controls, and anonymization. ## README (from the GitHub repository) No README available. ## Detected evidence (automated analysis) Indexed codebase: 17 recognized source files, 172 KB. - C (language) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code ## Codebase structure (from repository index) ### Files (25 of 25) ``` .firecrawl/medium-blood-cell-yolo.md .firecrawl/roboflow-bccd.md .firecrawl/search-blood-cell-model.json .firecrawl/search-hf-bccd.json .firecrawl/search-keremberke.json .firecrawl/search-roboflow-bccd.json .gitignore backend/detector.py backend/main.py backend/requirements.txt cam_stream.c camera_bridge.c frontend/.gitignore frontend/eslint.config.js frontend/index.html frontend/package.json frontend/README.md frontend/src/App.css frontend/src/App.jsx frontend/src/index.css frontend/src/main.jsx frontend/src/VoiceAgent.jsx frontend/vite.config.js lcd_display.py pi_streamer.py ``` ### Dependencies - backend/requirements.txt: fastapi@==0.115.6, google-genai@>=1.0.0, numpy@==2.2.1, opencv-python-headless@==4.11.0.86, python-dotenv@==1.2.1, python-multipart@==0.0.20, uvicorn@==0.34.0, websockets@==15.0.1 - frontend/package.json: @eslint/js@^10.0.1, @types/react@^19.2.14, @types/react-dom@^19.2.3, @vitejs/plugin-react@^6.0.1, eslint@^10.3.0, eslint-plugin-react-hooks@^7.1.1, eslint-plugin-react-refresh@^0.5.2, globals@^17.6.0, react@^19.2.6, react-dom@^19.2.6, vite@^8.0.12 ### Recent commits (newest first) - Replace placeholder demo image with real microscope blood smears - Add demo mode for lab demos without Pi camera - Draw YOLO annotations on captures, center-crop/sharpen camera feed, seamless start/stop - Replace ffmpeg pipeline with on-device JPEG encoding via turbojpeg - Fix warm-up detection: check R/B channels instead of brightness - Skip dark/green frames during camera auto-exposure warm-up - Add CSS styles for center tabs, capture button, captures gallery, and capture detail modal - Replace continuous detection with capture-based workflow, add captures gallery and Gemini analysis display - Add onCaptureResult prop to VoiceAgent for voice-triggered captures - Add capture workflow: analytics helper, Gemini integration, /capture/analyze endpoint, voice function calling - Use MJPEG compression for Pi camera streaming over SSH - Add google-genai dependency - Working Pi camera via SSH-piped cam_stream (no tunnel needed) - Replace SSH tunnel camera bridge with direct cam_stream NV12 capture - Fix Deepgram voice settings: correct type, provider structure, model name - Fix Deepgram voice agent endpoint to v1/agent/converse - Implement Pi camera streaming via SSH tunnel + MJPEG - Add Tempus brand icon, increase text visibility throughout - Rename to Tempus, add charts/medical terms, fix voice diagnostics - Fix backend freeze: resolve mDNS with timeout in camera receiver ## Key source files (fetched from GitHub, selected and truncated for size) ### .firecrawl/roboflow-bccd.md ```markdown [Universe](https://universe.roboflow.com/ "Universe Home") [Explore](https://universe.roboflow.com/browse) Datasets Models [Trending](https://universe.roboflow.com/trending) Sign in [See all 364 images](https://universe.roboflow.com/joseph-nelson/bccd/browse) [![BCCD cells sample showing RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/0V0DvUEa8YldxooiDHvk/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/0V0DvUEa8YldxooiDHvk) [![BCCD cells sample showing RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/0a1kYH9LRwiUQiSxmmpF/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/0a1kYH9LRwiUQiSxmmpF) [![BCCD cells sample showing Platelets, RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/0eLdBBE42VesQ08rN3UF/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/0eLdBBE42VesQ08rN3UF) [![BCCD cells sample showing RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/0jCQjHTIcoCt4n5CcmfT/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/0jCQjHTIcoCt4n5CcmfT) [![BCCD cells sample showing Platelets, RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/1G9BOtdAqQ9TGrc3068q/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/1G9BOtdAqQ9TGrc3068q) [![BCCD cells sample showing RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/1MWGXEaVn858tSNYefcH/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/1MWGXEaVn858tSNYefcH) [![BCCD cells sample showing RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/1O8AFRMKFsokgH3VkwPa/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/1O8AFRMKFsokgH3VkwPa) [![BCCD cells sample showing RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/1g5pDH9DbjZi6RCLfZf9/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/1g5pDH9DbjZi6RCLfZf9) [![BCCD cells sample showing RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/1gBGZG8hUMcNrxPNPbMU/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/1gBGZG8hUMcNrxPNPbMU) [![BCCD cells sample showing Platelets, RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/1p5SiOq9SQKAoZrAoRrx/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/1p5SiOq9SQKAoZrAoRrx) [![BCCD cells sample showing Platelets, RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/1r8qKBgmJMCjKr9GAij0/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/1r8qKBgmJMCjKr9GAij0) [![BCCD cells sample showing RBC, WBC](https://source.roboflow.com/Ly2DeBzbwsemGd2ReHk4BFxy8683/1yJCZ0Litzk8wfNDfEjl/thumb.jpg)](https://universe.roboflow.com/joseph-nelson/bccd/images/1yJCZ0Litzk8wfNDfEjl) ## About BCCD Dataset # Overview This is a dataset of blood cells photos, originally open sourced by [cosmicad](https://github.com/cosmicad/dataset) and [akshaylambda](https://github.com/akshaylamba/all_ [truncated — 5256 more characters] ``` ### .firecrawl/medium-blood-cell-yolo.md ```markdown [Sitemap](https://medium.com/sitemap/sitemap.xml) [Open in app](https://play.google.com/store/apps/details?id=com.medium.reader&referrer=utm_source%3DmobileNavBar&source=post_page---top_nav_layout_nav-----------------------------------------) Sign up [Sign in](https://medium.com/m/signin?operation=login&redirect=https%3A%2F%2Fmedium.com%2F%40mmasdar%2Ffast-and-accurate-blood-cell-counting-with-yolo-v7-a-state-of-the-art-object-detection-algorithm-ad1d3169fe77&source=post_page---top_nav_layout_nav-----------------------global_nav------------------) [Medium Logo](https://medium.com/?source=post_page---top_nav_layout_nav-----------------------------------------) Get app [Write](https://medium.com/m/signin?operation=register&redirect=https%3A%2F%2Fmedium.com%2Fnew-story&source=---top_nav_layout_nav-----------------------new_post_topnav------------------) [Search](https://medium.com/search?source=post_page---top_nav_layout_nav-----------------------------------------) Sign up [Sign in](https://medium.com/m/signin?operation=login&redirect=https%3A%2F%2Fmedium.com%2F%40mmasdar%2Ffast-and-accurate-blood-cell-counting-with-yolo-v7-a-state-of-the-art-object-detection-algorithm-ad1d3169fe77&source=post_page---top_nav_layout_nav-----------------------global_nav------------------) ![Unknown user](https://miro.medium.com/v2/resize:fill:32:32/1*dmbNkD5D-u45r44go_cf0g.png) # Fast and Accurate Blood Cell Counting with YoLo v7: A State-of-the-Art Object Detection Algorithm [![Muhammad Masdar Mahasin](https://miro.medium.com/v2/resize:fill:32:32/0*9h36ngeUtfh3Xj3F.jpg)](https://medium.com/@mmasdar?source=post_page---byline--ad1d3169fe77---------------------------------------) [Muhammad Masdar Mahasin](https://medium.com/@mmasdar?source=post_page---byline--ad1d3169fe77---------------------------------------) Follow 3 min read · Apr 17, 2023 4 [Listen](https://medium.com/m/signin?actionUrl=https%3A%2F%2Fmedium.com%2Fplans%3Fdimension%3Dpost_audio_button%26postId%3Dad1d3169fe77&operation=register&redirect=https%3A%2F%2Fmedium.com%2F%40mmasdar%2Ffast-and-accurate-blood-cell-counting-with-yolo-v7-a-state-of-the-art-object-detection-algorithm-ad1d3169fe77&source=---header_actions--ad1d3169fe77---------------------post_audio_button------------------) Share ![](https://miro.medium.com/v2/resize:fit:651/1*l1zdJpdH9WutUbq7Sq-Y9g.jpeg) ### Introduction: The accurate counting and classification of blood cells is crucial in various clinical settings, such as diagnosing diseases and monitoring patient health. Traditional manual methods for blood cell counting can be time-consuming and prone to errors. However, with advancements in computer vision and deep learning, automated blood cell counters based on object detection algorithms have become more efficient and accurate. ### Motivation: The blood cell counter project is inspired by the research challenge in My Lab, Protection Radiation Laboratory, who are investigating blood quality using bioimpedance an [truncated — 6110 more characters] ``` ### backend/requirements.txt ``` fastapi==0.115.6 uvicorn==0.34.0 opencv-python-headless==4.11.0.86 numpy==2.2.1 python-multipart==0.0.20 websockets==15.0.1 python-dotenv==1.2.1 google-genai>=1.0.0 ``` ### frontend/package.json ``` { "name": "frontend", "private": true, "version": "0.0.0", "type": "module", "scripts": { "dev": "vite", "build": "vite build", "lint": "eslint .", "preview": "vite preview" }, "dependencies": { "react": "^19.2.6", "react-dom": "^19.2.6" }, "devDependencies": { "@eslint/js": "^10.0.1", "@types/react": "^19.2.14", "@types/react-dom": "^19.2.3", "@vitejs/plugin-react": "^6.0.1", "eslint": "^10.3.0", "eslint-plugin-react-hooks": "^7.1.1", "eslint-plugin-react-refresh": "^0.5.2", "globals": "^17.6.0", "vite": "^8.0.12" } } ``` ### frontend/src/main.jsx ```javascript import { StrictMode } from 'react' import { createRoot } from 'react-dom/client' import './index.css' import App from './App.jsx' createRoot(document.getElementById('root')).render( , ) ``` ### backend/main.py ```python import asyncio import base64 import json import os import subprocess import threading import time from datetime import datetime, timezone from dotenv import load_dotenv import cv2 import numpy as np from fastapi import FastAPI, HTTPException, Request, WebSocket, WebSocketDisconnect from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import Response, StreamingResponse from websockets.asyncio.client import connect as ws_connect from pathlib import Path from detector import Detector, decode_frame # Gemini (optional) try: from google import genai from google.genai import types as genai_types _GENAI_AVAILABLE = True except ImportError: _GENAI_AVAILABLE = False load_dotenv() app = FastAPI(title="Tempus - Hematology Cell Analysis") app.add_middleware( CORSMiddleware, allow_origins=[ "http://localhost:5173", "http://127.0.0.1:5173", "http://localhost:5174", "http://127.0.0.1:5174", ], allow_methods=["*"], allow_headers=["*"], ) detector = Detector() # Analytics cache for /alerts and /report endpoints _last_analytics = {} _last_analytics_lock = threading.Lock() PI_SSH_HOST = os.getenv("PI_SSH_HOST", "qnxuser@qnxpi27.local") PI_SSH_PASS = os.getenv("PI_SSH_PASS", "qnxuser") CAM_WIDTH = 640 CAM_HEIGHT = 480 CAM_FPS = 10 camera_frame = None camera_frame_lock = threading.Lock() camera_connected = False _cam_proc = None _receiver_thread = None _stop_camera = threading.Event() _camera_paused = threading.Event() # When set, frames are read but not exposed # Demo mode BACKEND_DIR = Path(__file__).parent DEMO_DIR = BACKEND_DIR / "demo_images" _demo_mode = False _demo_image_index = 0 _demo_images: list[Path] = sorted( p for p in DEMO_DIR.glob("*") if p.suffix.lower() in (".png", ".jpg", ".jpeg") ) if DEMO_DIR.exists() else [] def _kill_pi_camera_procs(): """Kill any existing camera processes on the Pi.""" cmd = [ "sshpass", "-p", PI_SSH_PASS, "ssh", "-o", "StrictHostKeyChecking=no", "-o", "ConnectTimeout=10", PI_SSH_HOST, "slay -f cam_stream 2>/dev/null; slay -f camera_bridge 2>/dev/null; " "slay -f camera_example3_viewfinder 2>/dev/null", ] try: subprocess.run(cmd, timeout=15, capture_output=True) time.sleep(2) except Exception: pass class FrameReader: """Reads length-prefixed JPEG frames from cam_stream.""" def __init__(self, stream): self._stream = stream def read_frame(self): """Read next frame: 4-byte big-endian length + JPEG data.""" hdr = self._read_exact(4) if not hdr: raise ConnectionError("Stream closed") size = int.from_bytes(hdr, "big") if size == 0 or size > 10_000_000: raise ConnectionError(f"Invalid frame size: {size}") data = self._read_exact(size) if not data: raise ConnectionError("Stream closed mid-frame") return data def _read_exact(self, n): buf = bytearray() while len(buf) < n: chunk = self._stream.read(n - len(buf)) if not chunk: return None buf.extend(chunk) return bytes(buf) def _center_crop_and_enhance(frame, crop_ratio=0.65): """Center-crop and sharpen frame to remove dark circular lens edges.""" h, w = frame.shape[:2] cx, cy = w // 2, h // 2 crop_w, crop_h = int(w * crop_ratio) // 2, int(h * crop_ratio) // 2 cropped = frame[cy - crop_h:cy + crop_h, cx - crop_w:cx + crop_w] # Unsharp mask for sharpening gaussian = cv2.GaussianBlur(cropped, (0, 0), 2.0) sharpened = cv2.addWeighted(cropped, 1.5, gaussian, -0.5, 0) return sharpened def _draw_annotations(frame, detections): """Draw YOLO bounding boxes, labels, confidence, and morphology tags on a frame copy.""" annotated = frame.copy() for det in detections: x, y, w, h = det["bbox"] color_bgr = det.get("color", [0, 255, 0]) # OpenCV uses BGR tuples color = tuple(color_bgr) conf = det.get("confidence", 0) label = det.get("label", "") det_type = det.get("type", "yolo") if det_type == "shape": # Dashed rectangle for shape detections dash_len = 8 # Top edge for i in range(x, x + w, dash_len * 2): cv2.line(annotated, (i, y), (min(i + dash_len, x + w), y), color, 2) # Bottom edge for i in range(x, x + w, dash_len * 2): cv2.line(annotated, (i, y + h), (min(i + dash_len, x + w), y + h), color, 2) # Left edge for i in range(y, y + h, dash_len * 2): cv2.line(annotated, (x, i), (x, min(i + dash_len, y + h)), color, 2) # Right edge for i in range(y, y + h, dash_len * 2): cv2.line(annotated, (x + w, i), (x + w, min(i + dash_len, y + h)), color, 2) else: # Solid rectangle for YOLO detections cv2.rectangle(annotated, (x, y), (x + w, y + h), color, 2) # Label + confidence tag tag = f"{label} {conf:.0%}" font = cv2.FONT_HERSHEY_SIMPLEX font_scale = 0.45 thickness = 1 (tw, th), baseline = cv2.getTextSize(tag, font, font_scale, thickness) # Background rectangle for label cv2.rectangle(annotated, (x, y - th - baseline - 4), (x + tw + 4, y), color, -1) # Text color: white or black depending on brightness brightness = sum(color_bgr) / 3 text_color = (0, 0, 0) if brightness > 140 else (255, 255, 255) cv2.putText(annotated, tag, (x + 2, y - baseline - 2), font, font_scale, text_color, thickness, cv2.LINE_AA) # Morphology sub-label for abnormal cells morph = det.get("morphology") if morph and morph != "Normal": morph_color = tuple(det.get("morph_color", [255, 255, 255])) morph_tag = f"[{morph}]" [truncated — 32468 more characters] ``` ### frontend/src/App.jsx ```javascript import { useState, useRef, useEffect, useCallback, useMemo } from "react"; import "./App.css"; import VoiceAgent from "./VoiceAgent"; const API_URL = "http://localhost:8000"; function useAnimatedValue(target, duration = 500) { const [val, setVal] = useState(target); const raf = useRef(null); useEffect(() => { const from = val; const t0 = performance.now(); const tick = (now) => { const t = Math.min((now - t0) / duration, 1); setVal(from + (target - from) * (1 - Math.pow(1 - t, 3))); if (t < 1) raf.current = requestAnimationFrame(tick); }; raf.current = requestAnimationFrame(tick); return () => cancelAnimationFrame(raf.current); }, [target, duration]); return val; } const fmt = (s) => `${String(Math.floor(s / 60)).padStart(2, "0")}:${String(s % 60).padStart(2, "0")}`; const uid = () => Date.now().toString(36) + Math.random().toString(36).slice(2, 8); // ── Header ── function HeaderBar({ backendConnected, cameraConnected, voiceActive, sessionElapsed, capturing, operationMode, setOperationMode, flaggedForReview, demoMode, toggleDemo }) { return (

Tempus

Hematology
Camera
Backend
Voice
{capturing &&
{fmt(sessionElapsed)}
} {flaggedForReview &&
Review Required
}
); } // ── Specimen Info ── function SpecimenInfo({ sessionId, sessionElapsed, hasCaptures, mode, captureCount, flagged }) { return (
Specimen
Accession No. {sessionId.slice(0, 10).toUpperCase()}
Analysis Mode {mode === "blood_cell" ? "CBC / Differential" : "General Cytology"}
Elapsed {fmt(sessionElapsed)}
Status {hasCaptures ? "Active" : "Idle"}
Captures {captureCount}
Review {flagged ? "Flagged" : "None"}
); } // ── Gauge ── function AtypicalGauge({ value, alertLevel }) { const r = 85; const rOuter = 95; const circ = 2 * Math.PI * r; const clamped = Math.min(Math.max(value, 0), 100); const offset = circ - (clamped / 100) * circ; const animated = useAnimatedValue(clamped); const labels = { normal: "WNL", warning: "Borderline", critical: "Critical" }; return (
{animated.toFixed(1)}% Dysmorphic
{labels[alertLevel]}
); } // ── Stats ── function LiveStatsGrid({ analytics, mode }) { const active = !!analytics; return (
Capture Metrics
#include #include #include #include #include #include #include #include #include #include #define PORT 8765 #define MAGIC 0x514E5846 /* "QNXF" */ static int client_fd = -1; static volatile int running = 1; static void sighandler(int sig) { (void)sig; running = 0; } static void vf_callback(camera_handle_t handle, camera_buffer_t *buf, void *arg) { (void)handle; (void)arg; if (client_fd < 0 || buf == NULL) return; uint32_t width = buf->framedesc.nv12.width; uint32_t height = buf->framedesc.nv12.height; uint32_t stride = buf->framedesc.nv12.stride; uint32_t uv_offset = buf->framedesc.nv12.uv_offset; uint32_t uv_stride = buf->framedesc.nv12.uv_stride; /* Calculate actual NV12 data size: Y plane + UV plane */ uint32_t size = uv_offset + uv_stride * (height / 2); /* Build header */ uint32_t header[8]; header[0] = htonl(MAGIC); header[1] = htonl(width); header[2] = htonl(height); header[3] = htonl(stride); header[4] = htonl(uv_offset); header[5] = htonl(uv_stride); header[6] = htonl(0); /* format: 0 = NV12 */ header[7] = htonl(size); /* Send header + frame data */ if (write(client_fd, header, 32) != 32) { close(client_fd); client_fd = -1; return; } uint32_t sent = 0; while (sent < size) { ssize_t n = write(client_fd, (uint8_t*)buf->framebuf + sent, size - sent); if (n <= 0) { close(client_fd); client_fd = -1; return; } sent += n; } } int main(int argc, char **argv) { int unit = 1; if (argc > 1) unit = atoi(argv[1]); signal(SIGINT, sighandler); signal(SIGTERM, sighandler); signal(SIGPIPE, SIG_IGN); /* Open camera */ camera_handle_t cam; camera_error_t err = camera_open((camera_unit_t)unit, CAMERA_MODE_RW, &cam); if (err != CAMERA_EOK) { fprintf(stderr, "camera_open failed: %d\n", err); return 1; } /* Configure viewfinder for NV12 callback */ camera_set_vf_property(cam, CAMERA_IMGPROP_FORMAT, CAMERA_FRAMETYPE_NV12); camera_set_vf_property(cam, CAMERA_IMGPROP_WIDTH, 1280); camera_set_vf_property(cam, CAMERA_IMGPROP_HEIGHT, 720); camera_set_vf_property(cam, CAMERA_IMGPROP_FRAMERATE, 15.0); /* Start viewfinder with callback */ err = camera_start_viewfinder(cam, vf_callback, NULL, NULL); if (err != CAMERA_EOK) { fprintf(stderr, "camera_start_viewfinder failed: %d\n", err); camera_close(cam); return 1; } /* TCP server */ int server_fd = socket(AF_INET, SOCK_STREAM, 0); int opt = 1; setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)); struct sockaddr_in addr = {0}; addr.sin_family = AF_INET; addr.sin_addr.s_addr = INADDR_ANY; addr.sin_port = htons(PORT); if (bind(server_fd, (struct sockaddr*)&addr, sizeof(addr)) < 0) { fprintf(stderr, "bind failed: %s\n", strerror(errno)); camera_stop_viewfinder(cam); camera_close(cam); return 1; } listen(server_fd, 2); printf("Camera bridge listening on port %d (NV12 1280x720@15fps)\n", PORT); fflush(stdout); while (running) { struct sockaddr_in caddr; socklen_t clen = sizeof(caddr); int fd = accept(server_fd, (struct sockaddr*)&caddr, &clen); if (fd < 0) { if (errno == EINTR) continue; break; } int nodelay = 1; setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &nodelay, sizeof(nodelay)); printf("Client connected\n"); fflush(stdout); client_fd = fd; /* Wait until client disconnects */ while (client_fd >= 0 && running) { usleep(100000); } printf("Client disconnected\n"); fflush(stdout); } camera_stop_viewfinder(cam); camera_close(cam); close(server_fd); return 0; } ``` ### cam_stream.c ```c /* * cam_stream - QNX Camera Module 3 JPEG streamer * Captures NV12 from camera, converts to I420, encodes JPEG with turbojpeg. * Each frame output: [4-byte big-endian length][JPEG data] * * Usage: cam_stream [-u unit] [-r fps] [-q quality] */ #include #include #include #include #include #include #include #include static volatile int running = 1; static tjhandle tj_handle = NULL; static int jpeg_quality = 70; static int first_frame = 1; static unsigned char *u_plane_buf = NULL; static unsigned char *v_plane_buf = NULL; static uint32_t uv_buf_size = 0; static void sighandler(int sig) { (void)sig; running = 0; } static void vf_callback(camera_handle_t handle, camera_buffer_t *buf, void *arg) { (void)handle; (void)arg; if (!running || buf == NULL || buf->framebuf == NULL || tj_handle == NULL) return; uint32_t width = buf->framedesc.nv12.width; uint32_t height = buf->framedesc.nv12.height; uint32_t stride = buf->framedesc.nv12.stride; uint32_t uv_offset = buf->framedesc.nv12.uv_offset; uint32_t uv_stride = buf->framedesc.nv12.uv_stride; if (width == 0 || height == 0) return; if (first_frame) { fprintf(stderr, "Camera: %ux%u stride=%u uv_offset=%u uv_stride=%u\n", width, height, stride, uv_offset, uv_stride); first_frame = 0; } /* Allocate U/V de-interleave buffers if needed */ uint32_t chroma_w = width / 2; uint32_t chroma_h = height / 2; uint32_t needed = chroma_w * chroma_h; if (needed > uv_buf_size) { free(u_plane_buf); free(v_plane_buf); u_plane_buf = (unsigned char *)malloc(needed); v_plane_buf = (unsigned char *)malloc(needed); if (!u_plane_buf || !v_plane_buf) return; uv_buf_size = needed; } /* De-interleave NV12 UV plane into separate U and V planes */ const unsigned char *nv12_uv = (const unsigned char *)buf->framebuf + uv_offset; for (uint32_t row = 0; row < chroma_h; row++) { const unsigned char *src = nv12_uv + row * uv_stride; unsigned char *u_dst = u_plane_buf + row * chroma_w; unsigned char *v_dst = v_plane_buf + row * chroma_w; for (uint32_t col = 0; col < chroma_w; col++) { u_dst[col] = src[col * 2]; v_dst[col] = src[col * 2 + 1]; } } /* Compress I420 to JPEG */ const unsigned char *y_plane = (const unsigned char *)buf->framebuf; const unsigned char *planes[3] = { y_plane, u_plane_buf, v_plane_buf }; int strides[3] = { (int)stride, (int)chroma_w, (int)chroma_w }; unsigned char *jpeg_buf = NULL; unsigned long jpeg_size = 0; int ret = tjCompressFromYUVPlanes( tj_handle, planes, (int)width, strides, (int)height, TJSAMP_420, &jpeg_buf, &jpeg_size, jpeg_quality, 0 ); if (ret != 0 || jpeg_buf == NULL) { if (jpeg_buf) tjFree(jpeg_buf); return; } /* Write 4-byte length header then JPEG data */ uint32_t len_net = htonl((uint32_t)jpeg_size); if (write(STDOUT_FILENO, &len_net, 4) != 4) { running = 0; tjFree(jpeg_buf); return; } size_t written = 0; while (written < jpeg_size && running) { ssize_t n = write(STDOUT_FILENO, jpeg_buf + written, jpeg_size - written); if (n <= 0) { running = 0; break; } written += n; } tjFree(jpeg_buf); } int main(int argc, char **argv) { int unit = 1; double fps = 15.0; for (int i = 1; i < argc; i++) { if (strcmp(argv[i], "-u") == 0 && i + 1 < argc) unit = atoi(argv[++i]); else if (strcmp(argv[i], "-r") == 0 && i + 1 < argc) fps = atof(argv[++i]); else if (strcmp(argv[i], "-q") == 0 && i + 1 < argc) jpeg_quality = atoi(argv[++i]); } signal(SIGINT, sighandler); signal(SIGTERM, sighandler); signal(SIGPIPE, sighandler); tj_handle = tjInitCompress(); if (!tj_handle) { fprintf(stderr, "tjInitCompress failed\n"); return 1; } camera_handle_t cam; camera_error_t err = camera_open((camera_unit_t)unit, CAMERA_MODE_RW, &cam); if (err != CAMERA_EOK) { fprintf(stderr, "camera_open failed: %d\n", err); tjDestroy(tj_handle); return 1; } camera_set_vf_property(cam, CAMERA_IMGPROP_FORMAT, CAMERA_FRAMETYPE_NV12); camera_set_vf_property(cam, CAMERA_IMGPROP_FRAMERATE, fps); err = camera_start_viewfinder(cam, vf_callback, NULL, NULL); if (err != CAMERA_EOK) { fprintf(stderr, "camera_start_viewfinder failed: %d\n", err); camera_close(cam); tjDestroy(tj_handle); return 1; } fprintf(stderr, "cam_stream: NV12->JPEG (q=%d) @ %.0f fps\n", jpeg_quality, fps); while (running) { usleep(100000); } camera_stop_viewfinder(cam); camera_close(cam); tjDestroy(tj_handle); free(u_plane_buf); free(v_plane_buf); return 0; } ``` ### pi_streamer.py ```python #!/usr/bin/env python3 """MJPEG TCP streamer for QNX Pi. Captures frames using GStreamer (videotestsrc or camera) and streams JPEG over a TCP socket on port 8765. The Mac backend connects and reads JPEG frames by finding FFD8/FFD9 boundaries. Usage on Pi: python3 pi_streamer.py [--test] --test Use test pattern instead of camera (for debugging) """ import socket import subprocess import sys import threading import time HOST = "0.0.0.0" PORT = 8765 WIDTH = 1280 HEIGHT = 720 FPS = 15 def gst_pipeline(use_test=False): """Return GStreamer pipeline command that outputs JPEG to stdout.""" if use_test: src = f"videotestsrc ! video/x-raw,width={WIDTH},height={HEIGHT},framerate={FPS}/1" else: # Try wrappercamerabinsrc for QNX camera src = f"wrappercamerabinsrc ! videoconvert ! videoscale ! video/x-raw,width={WIDTH},height={HEIGHT},framerate={FPS}/1" return [ "gst-launch-1.0", "-q", *src.split(" ! "), "!", "videoconvert", "!", "jpegenc", f"quality=80", "!", "fdsink", "fd=1", ] def gst_pipeline_str(use_test=False): """Return GStreamer pipeline as a single shell command string.""" if use_test: src = f"videotestsrc ! video/x-raw,width={WIDTH},height={HEIGHT},framerate={FPS}/1" else: src = f"wrappercamerabinsrc ! videoconvert ! videoscale ! video/x-raw,width={WIDTH},height={HEIGHT},framerate={FPS}/1" return f"gst-launch-1.0 -q {src} ! videoconvert ! jpegenc quality=80 ! fdsink fd=1" def start_capture(use_test=False): """Start GStreamer and return the subprocess.""" cmd = gst_pipeline_str(use_test) print(f"Starting: {cmd}") proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) return proc def handle_client(conn, addr, frame_lock, frame_holder, stop_event): """Send JPEG frames to connected client.""" print(f"[+] Client connected: {addr}") try: while not stop_event.is_set(): with frame_lock: jpeg = frame_holder[0] if jpeg is None: time.sleep(0.05) continue try: conn.sendall(jpeg) except (BrokenPipeError, ConnectionResetError, OSError): break time.sleep(1.0 / FPS) finally: print(f"[-] Client disconnected: {addr}") conn.close() def reader_loop(proc, frame_lock, frame_holder, stop_event): """Read JPEG frames from GStreamer stdout.""" buf = b"" while not stop_event.is_set() and proc.poll() is None: chunk = proc.stdout.read(16384) if not chunk: break buf += chunk # Extract complete JPEG frames while True: start = buf.find(b"\xff\xd8") if start == -1: buf = b"" break end = buf.find(b"\xff\xd9", start + 2) if end == -1: buf = buf[start:] break jpeg = buf[start:end + 2] buf = buf[end + 2:] with frame_lock: frame_holder[0] = jpeg stderr = proc.stderr.read().decode(errors="replace").strip() if stderr: print(f"GStreamer stderr: {stderr}") def main(): use_test = "--test" in sys.argv frame_lock = threading.Lock() frame_holder = [None] stop_event = threading.Event() # Start GStreamer capture proc = start_capture(use_test) time.sleep(1) if proc.poll() is not None: stderr = proc.stderr.read().decode(errors="replace") print(f"GStreamer failed to start: {stderr}") if not use_test: print("Retrying with test pattern...") proc = start_capture(use_test=True) time.sleep(1) if proc.poll() is not None: print("Test pattern also failed. Exiting.") sys.exit(1) # Start reader thread reader = threading.Thread(target=reader_loop, args=(proc, frame_lock, frame_holder, stop_event), daemon=True) reader.start() # Wait for first frame print("Waiting for first frame...") deadline = time.time() + 10 while time.time() < deadline: with frame_lock: if frame_holder[0] is not None: break time.sleep(0.1) else: print("WARNING: No frames received yet, but starting server anyway.") # TCP server server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) server.bind((HOST, PORT)) server.listen(4) print(f"MJPEG streamer ready on {HOST}:{PORT}") try: while True: conn, addr = server.accept() conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) t = threading.Thread(target=handle_client, args=(conn, addr, frame_lock, frame_holder, stop_event), daemon=True) t.start() except KeyboardInterrupt: print("\nShutting down.") finally: stop_event.set() proc.terminate() server.close() if __name__ == "__main__": main() ``` [8 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]