Project Info
Inspiration
“Why waste time say lot word when few word do trick.” — Kevin Malone, The Office. Kevin might’ve been joking, but for many people with speech impediments or communication challenges, saying fewer words can actually make communication smoother, faster, and more accessible. We wanted to build something that not only helps people communicate, but also helps them express their personality and emotion. Because real communication isn’t just about words — it’s about who you are.
What it does
Few Words Do Trick is an assistive communication platform designed for people with speech impediments or expressive communication difficulties. Unlike traditional AAC (Augmentative and Alternative Communication) tools that focus purely on transmitting speech, our system adds an emotional and personalized layer using real-time EEG emotion detection and MBTI-based personality modeling. From previous research, giving LLMs a persona using the MBTI framework boosts their conversational intelligence by 17–22%. Thus, our system integrates emotional signals from the user’s EEG headset with their personality profile to generate responses that are not only faster and clearer but also more natural and authentic to who they are. This creates a communication experience that feels genuinely human — reflecting tone, mood, and individuality — rather than robotic or generic. By combining neuroscience, machine learning, and personality theory, Few Words Do Trick bridges the gap between accessibility and emotional expression, helping users communicate efficiently and meaningfully in real time.
How we built it
Our system runs on three main layers: Signal and Emotion Processing, Intelligent Backend, and Frontend Experience. The Signal and Emotion Processing Layer integrates the EEG headset, applies Fourier Transforms and temporal smoothing, and performs emotion classification using power spectrum density analysis and a Random Forest Classifier model. The Intelligent Backend Layer handles speech-to-text and sentence generation using Lava and OpenAI’s GPT-5, as well as text-to-speech synthesis with ElevenLabs (more specifically, Whisper model) for customizable, emotion-aware voices. It’s built with FastAPI and Pydantic for validation, with Vite ensuring a smooth connection between the backend and frontend. The Frontend Experience Layer is built with React and NGROK tunneling. It features a MBTI personality quiz, real-time EEG and voice visualization, and a voice customization dashboard using the ElevenLabs API. The UI is designed to be simple, intuitive, and a little fun — keeping accessibility at the center.
Challenges we ran into
We faced several challenges throughout development. Microphone and EEG data access proved difficult without deployment, and collecting consistent EEG signals for model training required plenty of creative “method acting” to simulate emotional states. Integrating detected emotions into the real-time speech output pipeline was complex, and setting up a server to merge MBTI personality data with generated responses added another layer of difficulty. On top of that, we had to design a user interface that felt approachable, expressive, and even enjoyable to use.
Accomplishments we're proud of
We’re proud to have achieved 90% confidence in our emotion classification using EEG data, as well as successfully integrating multiple APIs across the frontend and backend. We built a fully functional real-time emotion-to-speech pipeline and developed personalized, expressive voice outputs that feel human and authentic. Most importantly, we built something that makes communication more natural and personal — a system that doesn’t just speak for you, but speaks like you.
What we learned
We learned how to process and classify EEG signals in real time, integrate emotional intelligence into speech systems, and design with empathy in mind. We also realized how vital personalization is in communication — even when powered by AI. And of course, we learned that Kevin Malone’s wisdom can be surprisingly relevant at a hackathon.
What's next
Looking ahead, we plan to expand Few Words Do Trick into a tool for everyday use by integrating portable EEG hardware and refining our emotion models with larger datasets. We also hope to add multilingual and cultural context support and eventually release it as an open-source assistive communication platform. Our goal is to bridge technology and empathy to help everyone express themselves — because sometimes, the fewest words make the biggest difference.
AI Conversation Assistant
A real-time conversation assistant that uses AI to generate contextual responses based on keywords, conversation history, and emotional state from EEG data.
Features
- Real-time Audio Recording: Browser-based microphone recording with Web Audio API
- Speech-to-Text: OpenAI Whisper integration via Lava Payments
- AI Response Generation: GPT-4o-mini for generating contextual responses
- Text-to-Speech: ElevenLabs integration with sentiment-based voice modulation
- Emotional State Integration: EEG data visualization and processing
- Multi-sentiment Responses: Generate calm, neutral, and excited response options
Tech Stack
- Frontend: React + Vite
- Backend: FastAPI (Python)
- AI Services: OpenAI (GPT-3.5, Whisper) via Lava Payments
- TTS: ElevenLabs
- Audio: Web Audio API
Setup Instructions
1. Backend Setup
cd backend
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# Edit .env with your API keys
python main.py
2. Frontend Setup
cd frontend
npm install
npm run dev
3. Environment Variables
Create .env file in backend directory:
LAVA_BASE_URL=your_lava_base_url
LAVA_FORWARD_TOKEN=your_lava_forward_token
ELEVENLABS_VOICE_ID=your_voice_id
Usage
- Start Recording: Click the microphone button to start recording
- Enter Keywords: Type keywords in the input field
- Generate Responses: Click "Generate" to get AI responses
- Select Response: Choose from calm, neutral, or excited options
- Listen: The selected response will be spoken with appropriate sentiment
API Endpoints
POST /api/speech-to-text- Convert audio to textPOST /api/generate-responses- Generate AI responsesPOST /api/text-to-speech- Convert text to speech with sentimentGET /api/health- Health check
Development
- Backend runs on
http://localhost:8000 - Frontend runs on
http://localhost:3000 - CORS is configured for development
- Use ngrok for microphone access in production
Hackathon Notes
- 24-hour development timeline
- Lava Payments integration for OpenAI API calls
- EEG emotional state simulation
- Real-time conversation flow
- Sentiment-based response generation
Analysis
View
Metric
- 10
- 9
- 2
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- FastAPIIn code
- FirebaseIn code
- HTMLIn code
- JavaScriptIn code
- LangChainIn code
- PythonIn code
- ReactIn code
- OpenAIClaimed
8 of 9 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
9.6 MB
Source files
49
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
ianalin123/few-word-do-trick
89 files · 33.0 MB · @ e6db5a0
Structure
Interface
23 files · 26%Screens, components and styles rendered to the user.
Application logic
44 files · 49%Domain rules, services and shared utilities.
+1 moreData & schema
2 files · 2%Schema definitions, migrations and data access.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- HTML97%
- Python2%
- JavaScript1%
- CSS1%
- Markdown0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
backend/requirements.txt
pypi · 19- fastapi
- firebase-admin
- httpx
- joblib
- langchain
- langchain-core
- langchain-openai
- langgraph
- librosa
- muselsl
- numpy
- pydantic
- pylsl
- python-dotenv
- python-multipart
- scikit-learn
- scipy
- uvicorn
- +1 more
frontend/package.json
npm · 13- axios
- firebase
- react
- react-dom
- react-router-dom
- +8 more
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
Feature verification
AI response generation via GPT (contextual, keyword-based)Verified
Speech-to-text and sentence generation using GPT-5 / GPT-4o-mini, generating contextual responses from keywords and history
Claimed on Devpostmedium confidencebackend/agents/conversation_agent.py:148— _call_gpt posts to gpt-4o-mini (not GPT-5 as devpost claims) through a Lava proxy; generates 4 response variants using conversation history and tool callsbackend/main.py:259— /api/generate-responses endpoint invokes the LangGraph conversation_graph and returns all_responses
Browser-based real-time audio recording (Web Audio API)Verified
Real-time Audio Recording: browser-based microphone recording with Web Audio API
Claimed on readmehigh confidencefrontend/src/components/AudioRecorder.jsx:36— Uses getUserMedia + MediaRecorder, plus AudioContext/AnalyserNode for live audio-level visualization
EEG emotion classification with Random Forest and PSD/Fourier featuresVerified
Emotion classification using power spectrum density analysis and a Random Forest Classifier model, achieving 90% confidence
Claimed on Devposthigh confidencebackend/pretraining/train_binary_classifier.py:193— Trains a RandomForestClassifier (n_estimators=200) with calibration on extracted EEG featuresbackend/live_stream.py:90— compute_band_power uses signal.welch (Welch's PSD method) per frequency bandbackend/emotion_model_binary.joblib— Serialized trained model artifact is present and loaded by main.py
FastAPI + Pydantic backendVerified
It's built with FastAPI and Pydantic for validation
Claimed on Devposthigh confidencebackend/main.py:1— FastAPI app with Pydantic BaseModel request schemas throughout
MBTI personality quiz UIVerified
It features a MBTI personality quiz
Claimed on Devposthigh confidencefrontend/src/components/FeedbackModal.jsx:81— 5-question quiz form that computes a personality type/description, described in-UI as based on MBTI
MBTI-based personality modeling injected into responsesVerified
Integrates MBTI personality profile with generated responses to make them more natural/authentic
Claimed on Devposthigh confidencebackend/mbti.py:1— get_mbti_communication_style returns per-type vocabulary/style/few-shot examples for all 16 MBTI typesbackend/agents/conversation_agent.py:224— personality_type/description injected into the GPT system prompt used to generate response variantsfrontend/src/components/FeedbackModal.jsx:127— Frontend quiz computes an MBTI-like personality type and passes it up via onPersonalityResult
Multi-sentiment response options (calm/neutral/excited or low/medium/high/contradictory)Verified
Generate calm, neutral, and excited response options for the user to choose from
Claimed on readmehigh confidencebackend/agents/conversation_agent.py:260— Prompts GPT for exactly 4 variants: low, medium, high, contradictoryfrontend/src/components/ResponseSelector.jsx:1— UI component for selecting among the generated response energy levels
Persistent conversation history (Firestore)Verified
conversation history used to generate contextual responses (README: 'conversation history')
Claimed on readmehigh confidencebackend/services/firestore_service.py:1— create_conversation/append_message/get_recent_messages persist and retrieve messages in Firestorefrontend/src/App.jsx:86— Subscribes to Firestore conversation messages via onSnapshot for live UI updates
Real-time EEG and voice visualization in UIVerified
Real-time EEG and voice visualization
Claimed on Devpostmedium confidencefrontend/src/components/EmotionalStateDisplay.jsx:3— Displays emotional state (happy/neutral/sad) driven by live WebSocket EEG updatesfrontend/src/components/AudioRecorder.jsx:107— Live audio-level bar visualization during recording via AnalyserNode
Real-time EEG emotion detectionVerified
Real-time EEG emotion detection integrated with a Muse headset, feeding emotional state into responses
Claimed on Devposthigh confidencebackend/live_stream.py:10— Connects to Muse headset via pylsl, extracts band power/DE/asymmetry features and streams them to the backend WebSocket for predictionbackend/main.py:409— /ws endpoint receives 52-feature vectors and runs predict_emotion(), broadcasting results to frontend clients
Real-time speech-to-text via Whisper (through Lava Payments)Verified
Speech-to-text using OpenAI Whisper integration via Lava Payments
Claimed on readmehigh confidencebackend/main.py:208— /api/speech-to-text forwards audio to OpenAI whisper-1 via a Lava proxy URL and returns transcribed text
Text-to-speech with emotion/energy-based voice modulation (ElevenLabs)Verified
Text-to-speech synthesis with ElevenLabs for customizable, emotion-aware voices
Claimed on Devposthigh confidencebackend/main.py:318— /api/text-to-speech applies energy-based and emotion-based offsets to stability/style, then calls ElevenLabs TTS APIbackend/main.py:293— apply_energy_range implements low/medium/high/contradictory stability ranges
Vite-based frontend/backend connectivityVerified
Vite ensuring a smooth connection between the backend and frontend
Claimed on Devpostmedium confidencefrontend/vite.config.js:1— Vite dev config present, including allowedHosts for an ngrok tunnel domain
Voice customization dashboard using ElevenLabs APIVerified
Voice customization dashboard using the ElevenLabs API, listing and previewing voices, saving settings
Claimed on Devposthigh confidencefrontend/src/components/VoiceDashboard.jsx:38— Fetches voices from ElevenLabs API, supports preview, per-voice settings sliders, save/reset, and calls backend voice-settings endpoints
NGROK tunneling for frontendCode-supported
The Frontend Experience Layer is built with React and NGROK tunneling
Claimed on Devpostmedium confidencefrontend/vite.config.js:8— allowedHosts whitelist includes an ngrok-free.dev domain, implying ngrok was used, but no ngrok setup/automation script exists in the repo
Speaker diarization for conversation historyCode-supported
implied by 'conversation history' context used in response generation (not explicitly named in Devpost/README but core to how EEG+MBTI context is merged)
Claimed on readmemedium confidencebackend/services/speaker_service.py:46— identify_speaker computes MFCC fingerprints and cosine similarity to assign/track speaker_1, speaker_2, etc. per conversation; used by speech-to-text to label persisted messages
Portable EEG hardware / larger dataset roadmapClaimed only
Plans to integrate portable EEG hardware and refine emotion models with larger datasets (future work)
Claimed on Devposthigh confidence17-22% conversational intelligence boost from MBTI persona (research claim)Blocked
From previous research, giving LLMs a persona using the MBTI framework boosts their conversational intelligence by 17-22%
Claimed on Devpostlow confidence
An AI agent derived these features from the project’s Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.