# Project export: Patronum This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: Patronum delivers AI-moderated group Cognitive Stimulation Therapy at home, increasing access to dementia care for economically or locationally disadvantaged patients. - Devpost: https://devpost.com/software/patronum - GitHub: https://github.com/RayHCai/patronum - Video: https://www.youtube.com/embed/XC-qGqeWGdo?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Ray Cai Flanagan (8 commits), pauldbd (4 commits) ## Devpost submission (written by the team) ### Inspiration https://www.reddit.com/r/ArtificialInteligence/comments/1bsf9mi/my_parents_are_getting_dementia_can_ai_help/ Dementia is a progressive neurological condition affecting over 55 million people worldwide. It impairs memory, language, reasoning, and the ability to perform daily activities. While there is currently no cure, treatments focus on slowing cognitive decline, maintaining function, and improving quality of life. One evidence-based approach to combating dementia is Cognitive Stimulation Therapy (CST): a structured program of themed activities designed to actively stimulate thinking, concentration, memory, and social interaction. CST sessions, typically conducted regularly, have been shown to be more effective in groups rather than individually (https://pubmed.ncbi.nlm.nih.gov/34942157/). The social component of CST encourages shared attention, conversation, and peer engagement, which amplifies its therapeutic impact. However, dementia patients in rural or underserved areas often lack access to local CST groups and may need to travel long distances to participate in sessions. We also wanted to raise awareness for the incredibly difficult work that many caregivers go through, drawing inspiration from the stories shared in communities like r/dementia, where family members openly describe the emotional exhaustion, financial strain, role reversal, and constant uncertainty that come with caring for a loved one with cognitive decline—often while balancing jobs, parenting, and their own mental health, with little formal support. We noticed positive support of AI assisted therapy for dementia patients by caregivers on the posted link and felt that a project like ours would be appropriate method for alleviating some of the stress that caregivers go through. ### What it does Patronum is an intelligent platform that enables dementia patients to participate in simulated group CST sessions with humanlike AI participants and a moderator. Designed as a supplement to in-person therapy, it allows patients to engage in more frequent sessions at their own convenience. It has 4 main features: Simulated Group Sessions: An AI moderator guides structured CST activities with up to 5 AI participants and the patient, preserving the social dynamics critical to CST's effectiveness. To put it simply, we emulate a group therapy session with AI agents where patients are asked mentally stimulating questions that relate to reminiscent and sensory experiences rather than factual recall. For example: What do you miss from your childhood? What brings you back to your favorite memories? Simulated Group Sessions: An AI moderator guides structured CST activities with up to 5 AI participants and the patient, preserving the social dynamics critical to CST's effectiveness. To put it simply, we emulate a group therapy session with AI agents where patients are asked mentally stimulating questions that relate to reminiscent and sensory experiences rather than factual recall. For example: What do you miss from your childhood? What brings you back to your favorite memories? Photo-Based Memory Stimulation: Sessions incorporate family-uploaded photos, prompting participants to describe, interpret, and react to visual cues to strengthen recall and communication skills. Photo-Based Memory Stimulation: Sessions incorporate family-uploaded photos, prompting participants to describe, interpret, and react to visual cues to strengthen recall and communication skills. Caregiver Dashboard: Caregivers can assign sessions, upload photos, and view an analytics dashboard with session summaries, engagement metrics, memory performance trends, and social interaction data. We provide research backed statistics that caregivers can refer to the information we provide to track the progress of their patients. Caregiver Dashboard: Caregivers can assign sessions, upload photos, and view an analytics dashboard with session summaries, engagement metrics, memory performance trends, and social interaction data. We provide research backed statistics that caregivers can refer to the information we provide to track the progress of their patients. Cognitive Games: After a session, the patient is presented with a cognitive game that is related to the session (i.e. questions like who asked certain questions). We employ Error-Less Learning in these games, ensuring that patients have immediate access to the correct answers to the questions after answering them. This is done to ensure that patients have a relatively guilt-free environment. Cognitive Games: After a session, the patient is presented with a cognitive game that is related to the session (i.e. questions like who asked certain questions). We employ Error-Less Learning in these games, ensuring that patients have immediate access to the correct answers to the questions after answering them. This is done to ensure that patients have a relatively guilt-free environment. Additional features Session summaries written in plain language so caregivers can quickly understand how the patient engaged, without reviewing full transcripts Automated alerts when engagement or performance drops significantly across sessions Adjustable session length and difficulty to better match disease stage and patient stamina ### How we built it Our tech stack combines modern tools for a robust, scalable solution: Frontend: Built with React for a responsive, intuitive interface. Complex state management handles real-time session data and large analytics datasets seamlessly. Backend: Handles session orchestration, user management, and data persistence with a focus on low-latency performance across concurrent AI participants. AI Processing: Powered by Claude, ElevenLabs, and HeyGen, multiple AI agents run in real time with optimized prompting to keep conversations natural and responsive. HeyGen allows for realistic appearing agents, while ElevenLabs powers the agents’ speech. Real-time Updates: WebSocket connections ensure live session updates and smooth caregiver dashboard experiences. ### Challenges we ran into Prompt Optimization; We struggled with keeping agents speaking in a natural way, making the conversation feel realistic. Safety: We needed quite a lot of testing to ensure that the AI wouldn’t go past the guardrails we set. Meaningful Metrics: We did lots of research into what insights would be actionable for the caretaker but also generatable by AI. Accomplishments we’re proud of Building a realistic group CST experience that preserves the social interaction central to its effectiveness Creating caregiver analytics that surface meaningful trends instead of raw data Demonstrating how generative AI can support evidence-based healthcare interventions rather than replace human care ### What we learned Advanced video processing techniques in the browser Real-time data handling with WebSocket connections to handle real-time updates effectively AI model optimization for edge cases Complex state management in React applications, especially when dealing with large datasets Integration of multiple third-party services The importance of user experience in security applications ### What's next Future enhancements we're planning: Multiple language support Integration with existing healthcare systems to facilitate session assignment by healthcare providers Wider variety of conversation topics and questions available Data encryption to protect patient privacy Implementation of multiple papers used to protect against AI hallucination and malfunctions Built With HeyGen, ElevenLabs, Anthropic SDK (Claude), React, Vite, TypeScript, Tailwind CSS, Node.js, TypeScript, Express, Postgres, Prisma, AWS S3, Docker ## README (from the GitHub repository) # Patronum - AI-Powered Cognitive Stimulation Therapy ## Inspiration Dementia affects over 55 million people worldwide, yet access to effective treatment remains deeply unequal. Cognitive Stimulation Therapy (CST) — a structured program of themed activities proven to improve memory, reasoning, and social engagement — is [most effective in group settings](https://pubmed.ncbi.nlm.nih.gov/34942157/). But for patients in rural or underserved areas, attending regular group sessions often means long, impractical travel. Meanwhile, family caregivers are stretched to their limits, reporting mental breakdowns, burnout, and zero time for themselves. We wanted to build something that bridges the access gap for patients while giving caregivers the visibility and breathing room they desperately need. We also wanted to raise awareness for the incredibly difficult work that many caregivers go through, drawing inspiration from the stories shared in communities like r/dementia, where family members openly describe the emotional exhaustion, financial strain, role reversal, and constant uncertainty that come with caring for a loved one with cognitive decline—often while balancing jobs, parenting, and their own mental health, with little formal support. We noticed positive support of AI assisted therapy for dementia patients by caregivers on r/dementia and felt that a project like ours would be appropriate method for eliviating some of the stress that caregivers go through. ## What it does Patronum is an intelligent platform that enables dementia patients to participate in simulated group CST sessions with humanlike AI participants and a moderator. Designed as a supplement to in-person therapy, it allows patients to engage in more frequent sessions at their own convenience. It has 4 main features: - **Simulated Group Sessions:** An AI moderator guides structured CST activities with up to 5 AI participants and the patient, preserving the social dynamics critical to CST's effectiveness. To put it simply, we emulate a group therapy session with AI agents where patients are asked mentally stimulating questions that relate to reminisence and sensory experiences rather than factual recall. For example: - What do you miss from your childhood? - What catalysts brings you back to your favorite memories? - **Photo-Based Memory Stimulation:** Sessions incorporate family-uploaded photos and web-retrieved images, prompting participants to describe, interpret, and react to visual cues to strengthen recall and communication skills. We place emphasis on a style of learning called Error-Less Learning where patients are limited from the guilt of failures. In Error-Less Learning, patients always have access to the correct labels to questions and don't face test-like scenarios that emphasize errors made. - **Caregiver Dashboard:** Caregivers can assign sessions, upload photos, and view an analytics dashboard with session summaries, engagement metrics, memory performance trends, and social interaction data. We provide research backed statistics and hope that caregivers can refer to the information we provide to track the progress of their patients. - **Automated Alerts:** The system flags significant drops in engagement or cognitive performance across sessions, surfacing concerns early ### Additional features - Session summaries written in plain language so caregivers can quickly understand patient engagement without reviewing full transcripts - Adjustable session length and difficulty to match disease stage and patient stamina - Trend tracking over time, replacing subjective observation with actionable data - Reliable moments of caregiver respite during patient sessions ## How we built it Our tech stack combines modern tools for a robust, scalable solution: - **Frontend:** Built with React for a responsive, intuitive interface. Complex state management handles real-time session data and large analytics datasets seamlessly. - **Backend:** Handles session orchestration, user management, and data persistence with a focus on low-latency performance across concurrent AI participants. - **AI Processing:** Powered by Claude, our AI moderator and participants are carefully tuned for tone, safety, and clinical appropriateness — supportive and engaging without being condescending or confusing. Multiple AI agents run in real time with optimized prompting to keep conversations natural and responsive. - **Real-time Updates:** WebSocket connections ensure live session updates and smooth caregiver dashboard experiences. ## Challenges we ran into - **Performance Optimization:** Running multiple AI participants in real time while maintaining low latency required careful orchestration and prompt optimization to keep conversations natural and responsive - **Tone and Safety:** Designing AI behavior that is supportive, patient, and clinically appropriate without being condescending or confusing required extensive iteration and testing - **Meaningful Metrics:** Translating raw conversational data into caregiver-friendly insights without overwhelming them was challenging, requiring a balance of clinical relevance with simplicity ## Accomplishments that we're proud of - Built a realistic group CST experience that preserves the social interaction central to its therapeutic effectiveness - Created caregiver analytics that surface meaningful trends instead of raw data - Demonstrated how generative AI can support evidence-based healthcare interventions rather than replace human care - Developed a system accessible through any modern browser with minimal setup ## What we learned - Advanced real-time AI orchestration across multiple concurrent agents - Real-time data handling with WebSocket connections for live session updates - AI prompt optimization for sensitive healthcare contexts and edge cases - Complex state management in React applications with large datasets - The critical importance of balancing clinical rigor with user-friendly design ## What's next for Patronum Future enhancements we're planning: ### 1. Expanded Accessibility - Multiple language support - Voice-first interaction modes for patients with limited dexterity - Mobile-optimized experience ### 2. Healthcare Integration - Integration with existing healthcare systems for provider-assigned sessions - Exportable reports for clinician review - Standardized outcome measures aligned with clinical CST frameworks ### 3. Enhanced Security & Privacy - End-to-end data encryption to protect patient information - HIPAA compliance tooling - Granular access controls for multi-caregiver households ### 4. Richer Sessions - Wider variety of conversation topics and activity types - Adaptive difficulty that responds to real-time patient engagement - Personalized session themes based on patient history and preferences Our vision is to make evidence-based cognitive stimulation accessible to every dementia patient, regardless of geography, while empowering caregivers with the tools and time they need to sustain their own wellbeing. ## Built With Claude Code ## Detected evidence (automated analysis) Indexed codebase: 124 recognized source files, 837 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - Express (technology) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - PostgreSQL (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - AWS (technology) — claimed on Devpost, not found in the code - Docker (technology) — claimed on Devpost, not found in the code - Node.js (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (120 of 139) ``` .gitignore client/.gitignore client/index.html client/package.json client/postcss.config.js client/src/App.tsx client/src/components/AbstractTreeSVG.tsx client/src/components/admin/Sidebar.tsx client/src/components/auth/AuthCard.tsx client/src/components/auth/AuthLayout.tsx client/src/components/auth/NeuralBackground.tsx client/src/components/GlassmorphismNav.tsx client/src/components/NeuralNetworkBackground.tsx client/src/components/ProtectedPatientRoute.tsx client/src/components/session/AgentCall.tsx client/src/components/session/AgentRing.tsx client/src/components/session/AnimatedBubble.tsx client/src/components/session/AvatarVideoPlayer.tsx client/src/components/session/BubbleVisualization.tsx client/src/components/session/ChatBubble.tsx client/src/components/session/CognitiveGame.tsx client/src/components/session/FloatingSubtitle.tsx client/src/components/session/GameChoiceScreen.tsx client/src/components/session/ImageMatchingOptions.tsx client/src/components/session/MemoryRecallOptions.tsx client/src/components/session/MicrophoneBar.tsx client/src/components/session/PatternRecognitionOptions.tsx client/src/components/session/SpeakerBubble.tsx client/src/components/session/Subtitle.tsx client/src/components/session/VideoAvatarGrid.tsx client/src/components/session/WordAssociationOptions.tsx client/src/components/SpeechGraph/index.ts client/src/components/SpeechGraph/SpeechGraphMetrics.tsx client/src/components/SpeechGraph/SpeechGraphVisualization.tsx client/src/components/ui/AgentAvatar.tsx client/src/components/ui/index.ts client/src/components/ui/LoadingScreen.tsx client/src/components/ui/PatientButton.tsx client/src/components/ui/PatientCard.tsx client/src/constants/config.ts client/src/contexts/PatientContext.tsx client/src/data/topics.ts client/src/hooks/useAudioFetcher.ts client/src/hooks/useAudioPlayback.ts client/src/hooks/useConversationFlow.ts client/src/hooks/useElevenLabsAgent.ts client/src/hooks/useHeygenAvatar.ts client/src/hooks/useMicrophone.ts client/src/hooks/useNewConversationFlow.ts client/src/hooks/useSpeakerDetermination.ts client/src/hooks/useTextGeneration.ts client/src/hooks/useVideoStreamManager.ts client/src/hooks/useWebSocket.ts client/src/main.tsx client/src/pages/admin/AdminLogin.tsx client/src/pages/admin/AdminSignup.tsx client/src/pages/admin/CreatePatient.tsx client/src/pages/admin/PatientList.tsx client/src/pages/admin/PatientProfile.tsx client/src/pages/admin/PatientSettings.tsx client/src/pages/ConversationSetup.tsx client/src/pages/Landing.tsx client/src/pages/NewSession.tsx client/src/pages/PatientAccess.tsx client/src/pages/PatientAuth.tsx client/src/pages/PatientHome.tsx client/src/pages/Quiz.tsx client/src/pages/Session.tsx client/src/pages/ThankYou.tsx client/src/services/avatarManager.ts client/src/services/participants.ts client/src/stores/authStore.ts client/src/stores/conversationStore.ts client/src/stores/patientStore.ts client/src/styles/globals.css client/src/types/cognitiveGame.ts client/src/types/index.ts client/src/utils/audioCache.ts client/src/utils/authUtils.ts client/src/utils/retryWithBackoff.ts client/tailwind.config.js client/tsconfig.json client/tsconfig.node.json client/vite.config.ts docker-compose.yml HEYGEN_AVATAR_FIX.md package.json README.md server/.gitignore server/package.json server/prisma/schema.prisma server/scripts/list-heygen-avatars.ts server/scripts/list-public-avatars.ts server/src/config.ts server/src/constants/config.ts server/src/index.ts server/src/prisma/client.ts server/src/routes/agents.ts server/src/routes/aiModerator.ts server/src/routes/aiPatients.ts server/src/routes/audio.ts server/src/routes/auth.ts server/src/routes/cognitive-game.ts server/src/routes/convai.ts server/src/routes/conversation.ts server/src/routes/heygen.ts server/src/routes/participants.ts server/src/routes/reinforcement.ts server/src/routes/sessions.ts server/src/services/agent.ts server/src/services/aiModerator.ts server/src/services/aiPatient.ts server/src/services/auth.ts server/src/services/claude.ts server/src/services/cognitiveGameFactory.ts server/src/services/elevenLabs.ts server/src/services/email.ts server/src/services/heygen.ts server/src/services/orchestrator.old.ts server/src/services/orchestrator.ts [19 more files omitted for size] ``` ### Dependencies - client/package.json: @heygen/streaming-avatar@^2.1.0, @hookform/resolvers@^3.3.4, @types/d3@^7.4.3, @types/react@^18.2.0, @types/react-dom@^18.2.0, @vitejs/plugin-react@^4.2.0, autoprefixer@^10.4.0, axios@^1.6.7, clsx@^2.1.0, d3@^7.8.5, date-fns@^3.3.1, framer-motion@^11.0.0, lucide-react@^0.344.0, postcss@^8.4.0, react@^18.3.0, react-dom@^18.3.0, react-hook-form@^7.50.1, react-hot-toast@^2.6.0, react-router-dom@^6.22.0, reactflow@^11.10.4, recharts@^2.12.0, tailwind-merge@^2.2.1, tailwindcss@^3.4.0, typescript@^5.4.0, vite@^5.1.0, wavesurfer.js@^7.7.0, zod@^3.22.4, zustand@^4.5.0 - package.json: bcryptjs@^3.0.3, concurrently@^8.2.2, typescript@^5.4.0 - server/package.json: @anthropic-ai/sdk@^0.30.0, @paralleldrive/cuid2@^3.3.0, @prisma/adapter-pg@^6.19.2, @prisma/client@^6.19.2, @types/bcrypt@^5.0.2, @types/bcryptjs@^2.4.6, @types/cors@^2.8.17, @types/express@^4.17.21, @types/fluent-ffmpeg@^2.1.24, @types/jsonwebtoken@^9.0.5, @types/multer@^1.4.11, @types/node@^20.11.0, @types/pg@^8.16.0, @types/ws@^8.5.10, aws-sdk@^2.1550.0, bcrypt@^5.1.1, cors@^2.8.5, dotenv@^16.4.0, express@^4.18.0, express-jwt@^8.4.1, ffmpeg-static@^5.2.0, fluent-ffmpeg@^2.1.2, jsonwebtoken@^9.0.2, multer@^1.4.5-lts.1, pg@^8.18.0, prisma@^6.19.2, resend@^6.9.2, tsx@^4.7.0, typescript@^5.4.0, ws@^8.16.0 - speech-graph-service/requirements.txt: fastapi@==0.104.1, networkx@==3.1, pydantic@==2.5.0, uvicorn[standard]@==0.24.0 ### Recent commits (newest first) - pus - Merge branch 'master' of https://github.com/RayHCai/patronum - heygen - readme update - added readme - push - games - stuff - shit - Update PatientProfile.tsx - end convo - env fix - env bs - conversations - heygen - init ## Key source files (fetched from GitHub, selected and truncated for size) ### HEYGEN_AVATAR_FIX.md ```markdown # HeyGen Avatar Display Fix ## Problem HeyGen avatars were being generated on the backend but not displaying on the frontend. The console showed: - `[AvatarVideoPlayer] Stream marked active for [name] but no valid HeyGen config - skipping initialization` - Avatars stuck at 0/2 ready, infinite loading ## Root Cause **Two issues identified:** 1. **Incomplete `heygenConfig` storage**: When creating agents in `orchestrator.ts` and `sessions.ts`, the `heygenConfig` was stored as: ```typescript heygenConfig: JSON.stringify({ appearance }) ``` This was **missing the critical `avatarId` field** that the frontend needs to initialize HeyGen avatars. 2. **Missing JSON parsing**: The `heygenConfig` is stored as a JSON string in the database, but wasn't being properly parsed before sending to the client. ## Solution ### 1. Fixed Agent Creation (orchestrator.ts & sessions.ts) Updated agent creation to store **complete** `heygenConfig` with all required fields: ```typescript heygenConfig: JSON.stringify({ avatarId: heygenAvatarId, // ✅ Now included! appearance, createdAt: new Date().toISOString(), // ✅ Now included! lastUsed: new Date().toISOString(), // ✅ Now included! }) ``` ### 2. Added JSON Parsing & Fallback Logic Added helper functions that: - Parse `heygenConfig` from JSON string to object - Handle legacy agents with incomplete configs by using the `heygenAvatarId` field as fallback - Create minimal valid configs for existing agents **Files Modified:** - [orchestrator.ts:8-38](server/src/services/orchestrator.ts#L8-L38) - Added `parseHeygenConfig()` helper - [sessions.ts:14-54](server/src/routes/sessions.ts#L14-L54) - Added `serializeAgentForClient()` helper ### 3. Applied to All Agent Creation Paths Fixed in 3 locations where agents are created: - ✅ `server/src/services/orchestrator.ts` (lines 127-139, 156-175) - ✅ `server/src/routes/sessions.ts` (lines 290-310, 316-334) - ✅ `server/src/services/agent.ts` (already correct - lines 157-162) ## Testing To test the fix: 1. **Restart the server** to apply the changes: ```bash # Stop the server if running, then restart cd server npm run dev ``` 2. **Option A - Test with new session:** - Create a new conversation session - Avatars should now initialize and display video 3. **Option B - Clear existing agents (if issues persist):** ```sql -- This will force regeneration of agents with correct config DELETE FROM Agent WHERE participantId = 'your-participant-id'; ``` ## Expected Behavior After Fix ✅ Backend generates HeyGen avatars with complete config including `avatarId` ✅ Frontend receives properly parsed `heygenConfig` object ✅ `AvatarVideoPlayer` sees valid config and initializes HeyGen stream ✅ Video avatars display and can speak ✅ Loading progress shows "2/2 avatars ready" ## Backward Compatibility The fix includes fallback logic for existing agents in the database: - If `heygenConfig` exists but missing `avatarId` → uses [truncated — 170 more characters] ``` ### docker-compose.yml ```yaml version: '3.8' services: speech-graph: build: ./speech-graph-service container_name: speech-graph-service ports: - "8001:8001" environment: - PYTHONUNBUFFERED=1 healthcheck: test: ["CMD", "curl", "-f", "http://localhost:8001/health"] interval: 30s timeout: 3s retries: 3 start_period: 5s restart: unless-stopped networks: - app-network networks: app-network: driver: bridge ``` ### package.json ``` { "name": "ai-cst-platform", "version": "1.0.0", "private": true, "description": "AI Group Conversational Stimulation Platform for Dementia", "workspaces": [ "server", "client" ], "scripts": { "dev": "concurrently \"npm run dev:server\" \"npm run dev:client\"", "dev:server": "cd server && npm run dev", "dev:client": "cd client && npm run dev", "build": "npm run build:server && npm run build:client", "build:server": "cd server && npm run build", "build:client": "cd client && npm run build" }, "devDependencies": { "concurrently": "^8.2.2", "typescript": "^5.4.0" }, "engines": { "node": ">=18.0.0" }, "dependencies": { "bcryptjs": "^3.0.3" } } ``` ### speech-graph-service/requirements.txt ``` fastapi==0.104.1 uvicorn[standard]==0.24.0 pydantic==2.5.0 networkx==3.1 ``` ### speech-graph-service/Dockerfile ``` FROM python:3.10-slim WORKDIR /app # Install dependencies COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt # Copy application code COPY app/ ./app/ # Expose port EXPOSE 8001 # Health check HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \ CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8001/health')" # Run the application CMD ["python", "-m", "app.main"] ``` ### client/package.json ``` { "name": "client", "version": "1.0.0", "private": true, "type": "module", "scripts": { "dev": "vite", "build": "tsc && vite build", "preview": "vite preview", "lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0" }, "dependencies": { "@heygen/streaming-avatar": "^2.1.0", "@hookform/resolvers": "^3.3.4", "axios": "^1.6.7", "clsx": "^2.1.0", "d3": "^7.8.5", "date-fns": "^3.3.1", "framer-motion": "^11.0.0", "lucide-react": "^0.344.0", "react": "^18.3.0", "react-dom": "^18.3.0", "react-hook-form": "^7.50.1", "react-hot-toast": "^2.6.0", "react-router-dom": "^6.22.0", "reactflow": "^11.10.4", "recharts": "^2.12.0", "tailwind-merge": "^2.2.1", "wavesurfer.js": "^7.7.0", "zod": "^3.22.4", "zustand": "^4.5.0" }, "devDependencies": { "@types/d3": "^7.4.3", "@types/react": "^18.2.0", "@types/react-dom": "^18.2.0", "@vitejs/plugin-react": "^4.2.0", "autoprefixer": "^10.4.0", "postcss": "^8.4.0", "tailwindcss": "^3.4.0", "typescript": "^5.4.0", "vite": "^5.1.0" } } ``` ### server/package.json ``` { "name": "server", "version": "1.0.0", "private": true, "description": "Backend server for AI CST Platform", "main": "dist/index.js", "scripts": { "dev": "tsx watch src/index.ts", "build": "tsc", "start": "node dist/index.js", "prisma:generate": "prisma generate", "prisma:migrate": "prisma migrate dev", "prisma:studio": "prisma studio" }, "dependencies": { "@anthropic-ai/sdk": "^0.30.0", "@paralleldrive/cuid2": "^3.3.0", "aws-sdk": "^2.1550.0", "bcrypt": "^5.1.1", "cors": "^2.8.5", "dotenv": "^16.4.0", "express": "^4.18.0", "express-jwt": "^8.4.1", "ffmpeg-static": "^5.2.0", "fluent-ffmpeg": "^2.1.2", "jsonwebtoken": "^9.0.2", "multer": "^1.4.5-lts.1", "pg": "^8.18.0", "resend": "^6.9.2", "ws": "^8.16.0" }, "devDependencies": { "@prisma/adapter-pg": "^6.19.2", "@prisma/client": "^6.19.2", "@types/bcrypt": "^5.0.2", "@types/bcryptjs": "^2.4.6", "@types/cors": "^2.8.17", "@types/express": "^4.17.21", "@types/fluent-ffmpeg": "^2.1.24", "@types/jsonwebtoken": "^9.0.5", "@types/multer": "^1.4.11", "@types/node": "^20.11.0", "@types/pg": "^8.16.0", "@types/ws": "^8.5.10", "prisma": "^6.19.2", "tsx": "^4.7.0", "typescript": "^5.4.0" } } ``` ### client/src/main.tsx ```typescript import React from 'react' import ReactDOM from 'react-dom/client' import App from './App.tsx' import './styles/globals.css' ReactDOM.createRoot(document.getElementById('root')!).render( , ) ``` ### client/src/App.tsx ```typescript import { BrowserRouter as Router, Routes, Route } from 'react-router-dom' import { Toaster } from 'react-hot-toast' import Landing from './pages/Landing' import PatientHome from './pages/PatientHome' import ConversationSetup from './pages/ConversationSetup' import Session from './pages/Session' import ThankYou from './pages/ThankYou' import Quiz from './pages/Quiz' import PatientList from './pages/admin/PatientList' import PatientProfile from './pages/admin/PatientProfile' import PatientSettings from './pages/admin/PatientSettings' import CreatePatient from './pages/admin/CreatePatient' import AdminLogin from './pages/admin/AdminLogin' import AdminSignup from './pages/admin/AdminSignup' import PatientAuth from './pages/PatientAuth' import PatientAccess from './pages/PatientAccess' import ProtectedPatientRoute from './components/ProtectedPatientRoute' function App() { return ( {/* Public routes */} } /> {/* Auth routes */} } /> } /> } /> } /> {/* Protected Patient routes - require participant ID from URL */} }> } /> } /> } /> } /> } /> {/* Legacy routes - redirect to home for now */} } /> } /> } /> } /> } /> {/* Admin routes */} } /> } /> } /> } /> ) } export default App ``` ### server/src/index.ts ```typescript // server/src/index.ts import express from 'express'; import cors from 'cors'; import { createServer } from 'http'; import { WebSocketServer } from 'ws'; import { config } from './config'; import { prisma } from './prisma/client'; import authRoutes from './routes/auth'; import participantRoutes from './routes/participants'; import agentRoutes from './routes/agents'; import reinforcementRoutes from './routes/reinforcement'; import audioRoutes from './routes/audio'; import conversationRoutes from './routes/conversation'; import aiPatientRoutes from './routes/aiPatients'; import aiModeratorRoutes from './routes/aiModerator'; import cognitiveGameRoutes from './routes/cognitive-game'; import heygenRoutes from './routes/heygen'; import sessionsRoutes from './routes/sessions'; import { ConversationWebSocketHandler } from './websocket/handler'; import { AppError } from './types'; const app = express(); // Middleware app.use(cors({ origin: config.CLIENT_URL, credentials: true, })); app.use(express.json()); app.use(express.urlencoded({ extended: true })); // Health check route app.get('/health', (req, res) => { res.json({ status: 'ok', timestamp: new Date().toISOString() }); }); // API routes app.get('/api', (req, res) => { res.json({ message: 'AI CST Platform API' }); }); app.use('/api/auth', authRoutes); app.use('/api/participants', participantRoutes); app.use('/api/agents', agentRoutes); app.use('/api/reinforcement', reinforcementRoutes); app.use('/api/audio', audioRoutes); app.use('/api/conversation', conversationRoutes); app.use('/api/ai-patients', aiPatientRoutes); app.use('/api/ai-moderator', aiModeratorRoutes); app.use('/api/cognitive-game', cognitiveGameRoutes); app.use('/api/heygen', heygenRoutes); app.use('/api/sessions', sessionsRoutes); // 404 handler app.use((req, res) => { res.status(404).json({ success: false, error: 'Route not found', }); }); // Error handling middleware app.use((err: any, req: express.Request, res: express.Response, next: express.NextFunction) => { console.error('Error:', err); // Handle AppError instances if (err instanceof AppError) { return res.status(err.statusCode).json({ success: false, error: err.message, ...(config.NODE_ENV === 'development' && { stack: err.stack }), }); } // Handle other errors res.status(err.status || 500).json({ success: false, error: err.message || 'Internal server error', ...(config.NODE_ENV === 'development' && { stack: err.stack }), }); }); // Create HTTP server const server = createServer(app); // Create WebSocket server const wss = new WebSocketServer({ server, path: '/ws' }); const wsHandler = new ConversationWebSocketHandler(wss); // Handle WebSocket connections wss.on('connection', (ws, request) => { wsHandler.handleConnection(ws as any, request); }); // Start server const PORT = config.PORT; async function startServer() { try { // Test database connection await prisma.$connect(); console.log('✅ Database connected successfully'); server.listen(PORT, () => { console.log(`🚀 Server running on http://localhost:${PORT}`); console.log(`🔌 WebSocket server running on ws://localhost:${PORT}/ws`); console.log(`📝 Environment: ${config.NODE_ENV}`); console.log(`🌐 CORS enabled for: ${config.CLIENT_URL}`); }); } catch (error) { console.error('❌ Failed to start server:', error); process.exit(1); } } // Graceful shutdown process.on('SIGINT', async () => { console.log('\n🛑 Shutting down gracefully...'); await prisma.$disconnect(); process.exit(0); }); process.on('SIGTERM', async () => { console.log('\n🛑 Shutting down gracefully...'); await prisma.$disconnect(); process.exit(0); }); startServer(); ``` [117 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]