Project Info
Inspiration
https://www.reddit.com/r/ArtificialInteligence/comments/1bsf9mi/my_parents_are_getting_dementia_can_ai_help/ Dementia is a progressive neurological condition affecting over 55 million people worldwide. It impairs memory, language, reasoning, and the ability to perform daily activities. While there is currently no cure, treatments focus on slowing cognitive decline, maintaining function, and improving quality of life. One evidence-based approach to combating dementia is Cognitive Stimulation Therapy (CST): a structured program of themed activities designed to actively stimulate thinking, concentration, memory, and social interaction. CST sessions, typically conducted regularly, have been shown to be more effective in groups rather than individually (https://pubmed.ncbi.nlm.nih.gov/34942157/). The social component of CST encourages shared attention, conversation, and peer engagement, which amplifies its therapeutic impact. However, dementia patients in rural or underserved areas often lack access to local CST groups and may need to travel long distances to participate in sessions. We also wanted to raise awareness for the incredibly difficult work that many caregivers go through, drawing inspiration from the stories shared in communities like r/dementia, where family members openly describe the emotional exhaustion, financial strain, role reversal, and constant uncertainty that come with caring for a loved one with cognitive decline—often while balancing jobs, parenting, and their own mental health, with little formal support. We noticed positive support of AI assisted therapy for dementia patients by caregivers on the posted link and felt that a project like ours would be appropriate method for alleviating some of the stress that caregivers go through.
What it does
Patronum is an intelligent platform that enables dementia patients to participate in simulated group CST sessions with humanlike AI participants and a moderator. Designed as a supplement to in-person therapy, it allows patients to engage in more frequent sessions at their own convenience. It has 4 main features: Simulated Group Sessions: An AI moderator guides structured CST activities with up to 5 AI participants and the patient, preserving the social dynamics critical to CST's effectiveness. To put it simply, we emulate a group therapy session with AI agents where patients are asked mentally stimulating questions that relate to reminiscent and sensory experiences rather than factual recall. For example: What do you miss from your childhood? What brings you back to your favorite memories? Simulated Group Sessions: An AI moderator guides structured CST activities with up to 5 AI participants and the patient, preserving the social dynamics critical to CST's effectiveness. To put it simply, we emulate a group therapy session with AI agents where patients are asked mentally stimulating questions that relate to reminiscent and sensory experiences rather than factual recall. For example: What do you miss from your childhood? What brings you back to your favorite memories? Photo-Based Memory Stimulation: Sessions incorporate family-uploaded photos, prompting participants to describe, interpret, and react to visual cues to strengthen recall and communication skills. Photo-Based Memory Stimulation: Sessions incorporate family-uploaded photos, prompting participants to describe, interpret, and react to visual cues to strengthen recall and communication skills. Caregiver Dashboard: Caregivers can assign sessions, upload photos, and view an analytics dashboard with session summaries, engagement metrics, memory performance trends, and social interaction data. We provide research backed statistics that caregivers can refer to the information we provide to track the progress of their patients. Caregiver Dashboard: Caregivers can assign sessions, upload photos, and view an analytics dashboard with session summaries, engagement metrics, memory performance trends, and social interaction data. We provide research backed statistics that caregivers can refer to the information we provide to track the progress of their patients. Cognitive Games: After a session, the patient is presented with a cognitive game that is related to the session (i.e. questions like who asked certain questions). We employ Error-Less Learning in these games, ensuring that patients have immediate access to the correct answers to the questions after answering them. This is done to ensure that patients have a relatively guilt-free environment. Cognitive Games: After a session, the patient is presented with a cognitive game that is related to the session (i.e. questions like who asked certain questions). We employ Error-Less Learning in these games, ensuring that patients have immediate access to the correct answers to the questions after answering them. This is done to ensure that patients have a relatively guilt-free environment. Additional features Session summaries written in plain language so caregivers can quickly understand how the patient engaged, without reviewing full transcripts Automated alerts when engagement or performance drops significantly across sessions Adjustable session length and difficulty to better match disease stage and patient stamina
How we built it
Our tech stack combines modern tools for a robust, scalable solution: Frontend: Built with React for a responsive, intuitive interface. Complex state management handles real-time session data and large analytics datasets seamlessly. Backend: Handles session orchestration, user management, and data persistence with a focus on low-latency performance across concurrent AI participants. AI Processing: Powered by Claude, ElevenLabs, and HeyGen, multiple AI agents run in real time with optimized prompting to keep conversations natural and responsive. HeyGen allows for realistic appearing agents, while ElevenLabs powers the agents’ speech. Real-time Updates: WebSocket connections ensure live session updates and smooth caregiver dashboard experiences.
Challenges we ran into
Prompt Optimization; We struggled with keeping agents speaking in a natural way, making the conversation feel realistic. Safety: We needed quite a lot of testing to ensure that the AI wouldn’t go past the guardrails we set. Meaningful Metrics: We did lots of research into what insights would be actionable for the caretaker but also generatable by AI. Accomplishments we’re proud of Building a realistic group CST experience that preserves the social interaction central to its effectiveness Creating caregiver analytics that surface meaningful trends instead of raw data Demonstrating how generative AI can support evidence-based healthcare interventions rather than replace human care
What we learned
Advanced video processing techniques in the browser Real-time data handling with WebSocket connections to handle real-time updates effectively AI model optimization for edge cases Complex state management in React applications, especially when dealing with large datasets Integration of multiple third-party services The importance of user experience in security applications
What's next
Future enhancements we're planning: Multiple language support Integration with existing healthcare systems to facilitate session assignment by healthcare providers Wider variety of conversation topics and questions available Data encryption to protect patient privacy Implementation of multiple papers used to protect against AI hallucination and malfunctions Built With HeyGen, ElevenLabs, Anthropic SDK (Claude), React, Vite, TypeScript, Tailwind CSS, Node.js, TypeScript, Express, Postgres, Prisma, AWS S3, Docker
Patronum - AI-Powered Cognitive Stimulation Therapy
Inspiration
Dementia affects over 55 million people worldwide, yet access to effective treatment remains deeply unequal. Cognitive Stimulation Therapy (CST) — a structured program of themed activities proven to improve memory, reasoning, and social engagement — is most effective in group settings. But for patients in rural or underserved areas, attending regular group sessions often means long, impractical travel. Meanwhile, family caregivers are stretched to their limits, reporting mental breakdowns, burnout, and zero time for themselves. We wanted to build something that bridges the access gap for patients while giving caregivers the visibility and breathing room they desperately need.
We also wanted to raise awareness for the incredibly difficult work that many caregivers go through, drawing inspiration from the stories shared in communities like r/dementia, where family members openly describe the emotional exhaustion, financial strain, role reversal, and constant uncertainty that come with caring for a loved one with cognitive decline—often while balancing jobs, parenting, and their own mental health, with little formal support. We noticed positive support of AI assisted therapy for dementia patients by caregivers on r/dementia and felt that a project like ours would be appropriate method for eliviating some of the stress that caregivers go through.
What it does
Patronum is an intelligent platform that enables dementia patients to participate in simulated group CST sessions with humanlike AI participants and a moderator. Designed as a supplement to in-person therapy, it allows patients to engage in more frequent sessions at their own convenience. It has 4 main features:
-
Simulated Group Sessions: An AI moderator guides structured CST activities with up to 5 AI participants and the patient, preserving the social dynamics critical to CST's effectiveness. To put it simply, we emulate a group therapy session with AI agents where patients are asked mentally stimulating questions that relate to reminisence and sensory experiences rather than factual recall. For example:
- What do you miss from your childhood?
- What catalysts brings you back to your favorite memories?
-
Photo-Based Memory Stimulation: Sessions incorporate family-uploaded photos and web-retrieved images, prompting participants to describe, interpret, and react to visual cues to strengthen recall and communication skills. We place emphasis on a style of learning called Error-Less Learning where patients are limited from the guilt of failures. In Error-Less Learning, patients always have access to the correct labels to questions and don't face test-like scenarios that emphasize errors made.
-
Caregiver Dashboard: Caregivers can assign sessions, upload photos, and view an analytics dashboard with session summaries, engagement metrics, memory performance trends, and social interaction data. We provide research backed statistics and hope that caregivers can refer to the information we provide to track the progress of their patients.
-
Automated Alerts: The system flags significant drops in engagement or cognitive performance across sessions, surfacing concerns early
Additional features
- Session summaries written in plain language so caregivers can quickly understand patient engagement without reviewing full transcripts
- Adjustable session length and difficulty to match disease stage and patient stamina
- Trend tracking over time, replacing subjective observation with actionable data
- Reliable moments of caregiver respite during patient sessions
How we built it
Our tech stack combines modern tools for a robust, scalable solution:
- Frontend: Built with React for a responsive, intuitive interface. Complex state management handles real-time session data and large analytics datasets seamlessly.
- Backend: Handles session orchestration, user management, and data persistence with a focus on low-latency performance across concurrent AI participants.
- AI Processing: Powered by Claude, our AI moderator and participants are carefully tuned for tone, safety, and clinical appropriateness — supportive and engaging without being condescending or confusing. Multiple AI agents run in real time with optimized prompting to keep conversations natural and responsive.
- Real-time Updates: WebSocket connections ensure live session updates and smooth caregiver dashboard experiences.
Challenges we ran into
- Performance Optimization: Running multiple AI participants in real time while maintaining low latency required careful orchestration and prompt optimization to keep conversations natural and responsive
- Tone and Safety: Designing AI behavior that is supportive, patient, and clinically appropriate without being condescending or confusing required extensive iteration and testing
- Meaningful Metrics: Translating raw conversational data into caregiver-friendly insights without overwhelming them was challenging, requiring a balance of clinical relevance with simplicity
Accomplishments that we're proud of
- Built a realistic group CST experience that preserves the social interaction central to its therapeutic effectiveness
- Created caregiver analytics that surface meaningful trends instead of raw data
- Demonstrated how generative AI can support evidence-based healthcare interventions rather than replace human care
- Developed a system accessible through any modern browser with minimal setup
What we learned
- Advanced real-time AI orchestration across multiple concurrent agents
- Real-time data handling with WebSocket connections for live session updates
- AI prompt optimization for sensitive healthcare contexts and edge cases
- Complex state management in React applications with large datasets
- The critical importance of balancing clinical rigor with user-friendly design
What's next for Patronum
Future enhancements we're planning:
1. Expanded Accessibility
- Multiple language support
- Voice-first interaction modes for patients with limited dexterity
- Mobile-optimized experience
2. Healthcare Integration
- Integration with existing healthcare systems for provider-assigned sessions
- Exportable reports for clinician review
- Standardized outcome measures aligned with clinical CST frameworks
3. Enhanced Security & Privacy
- End-to-end data encryption to protect patient information
- HIPAA compliance tooling
- Granular access controls for multi-caregiver households
4. Richer Sessions
- Wider variety of conversation topics and activity types
- Adaptive difficulty that responds to real-time patient engagement
- Personalized session themes based on patient history and preferences
Our vision is to make evidence-based cognitive stimulation accessible to every dementia patient, regardless of geography, while empowering caregivers with the tools and time they need to sustain their own wellbeing.
Built With
Claude Code
Analysis
View
Metric
- 8
- 4
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- AnthropicIn code
- CSSIn code
- ExpressIn code
- FastAPIIn code
- HTMLIn code
- JavaScriptIn code
- PostgreSQLIn code
- PythonIn code
- ReactIn code
- Tailwind CSSIn code
- TypeScriptIn code
- AWSClaimed
- DockerClaimed
- Node.jsClaimed
11 of 14 appear in the indexed code. 3 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
837 KB
Source files
124
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
RayHCai/patronum
141 files · 1.2 MB · @ bf51204
Structure
Interface
53 files · 38%Screens, components and styles rendered to the user.
API & routing
39 files · 28%Request entry points: routes, handlers and controllers.
Application logic
28 files · 20%Domain rules, services and shared utilities.
Data & schema
2 files · 1%Schema definitions, migrations and data access.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- TypeScript96%
- Python2%
- Markdown1%
- CSS1%
- JavaScript0%
- YAML0%
- Other (1)0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
server/package.json
npm · 30- @anthropic-ai/sdk
- @paralleldrive/cuid2
- aws-sdk
- bcrypt
- cors
- dotenv
- express
- express-jwt
- ffmpeg-static
- fluent-ffmpeg
- jsonwebtoken
- multer
- pg
- resend
- ws
- +15 more
client/package.json
npm · 28- @heygen/streaming-avatar
- @hookform/resolvers
- axios
- clsx
- d3
- date-fns
- framer-motion
- lucide-react
- react
- react-dom
- react-hook-form
- react-hot-toast
- react-router-dom
- reactflow
- recharts
- tailwind-merge
- wavesurfer.js
- zod
- +10 more
speech-graph-service/requirements.txt
pypi · 4- fastapi
- networkx
- pydantic
- uvicorn[standard]
package.json
npm · 3- bcryptjs
- +2 more
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.