# Project export: BlinkAI
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: TreeHacks 2025
- Tagline: Blink AI gives a voice to those who cannot type or speak. It translates blinks into Morse code, converts them into text, and sends prompts to AI—unlocking seamless, hands-free communication instantly.
- Devpost: https://devpost.com/software/blinkai
- GitHub: https://github.com/FuzionEcho/BlinkAI
- Video: https://www.youtube.com/embed/wwaJdA6XVPk?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (Best Beginner Hack (Keychron K8 Pro QMK/VIA Wireless Mechanical Keyboard per team member))
- Team: 1 GitHub contributor(s) — FuzionEcho (11 commits)
## Devpost submission (written by the team)
### Inspiration
For millions of people living with Parkinson’s, ALS, and other mobility challenges, something as simple as sending a message can feel overwhelming. Most digital assistants rely on voice or typing, but what if those barriers didn’t exist at all? That’s why we created Blink AI—not just another assistant, but an intelligent system powered by a groundbreaking translation algorithm that converts blinks into text, opening a world of possibilities for those who need it most. Because communication should never be limited by physical ability. Everyone deserves a way to express their thoughts, ask questions, and stay connected—and now, they can do it with just a blink.
### What it does
Blink AI detects intentional blinks, measures their duration and sequence, translates them into Morse code, converts that Morse code into text, and then sends the final text to an AI assistant to generate a response. How It Works: Blink Detection: The Blink AI algorithm tracks intentional blinks, filtering out natural blinking to avoid false inputs. Blink Detection: The Blink AI algorithm tracks intentional blinks, filtering out natural blinking to avoid false inputs. Blink Interpretation: It analyzes the duration and frequency of each blink, translating them into dots and dashes in Morse code. Blink Interpretation: It analyzes the duration and frequency of each blink, translating them into dots and dashes in Morse code. Morse Code Conversion: The Morse code sequence is instantly decoded into readable text, forming a structured message. Morse Code Conversion: The Morse code sequence is instantly decoded into readable text, forming a structured message. AI Processing: The translated text is sent to an external AI assistant, which processes it like a standard input and generates a response. AI Processing: The translated text is sent to an external AI assistant, which processes it like a standard input and generates a response. Response Delivery: The AI assistant’s response is sent back to the user, enabling a smooth, natural exchange of information. Response Delivery: The AI assistant’s response is sent back to the user, enabling a smooth, natural exchange of information. What It Unlocks: Ask Questions & Get Answers – Interact with AI assistants and receive real-time responses. Send Messages with a Blink – Communicate effortlessly without touch or voice. Control Smart Devices – Operate technology hands-free with precise blink commands. Express Thoughts Clearly – A powerful tool for individuals with limited mobility, turning intention into action. Blink AI doesn’t just detect blinks—it translates them into a language AI can understand, creating a seamless bridge between human expression and technology. No voice. No keyboard. Just a blink.
### Challenges we ran into
Implementing the Morse Code Algorithm Accurately – Converting blinks into Morse code required precision. The algorithm had to reliably distinguish between short and long blinks, recognize intentional sequences, and filter out involuntary blinks. Fine-tuning timing and detection thresholds was crucial to prevent errors and misinterpretations. Implementing the Morse Code Algorithm Accurately – Converting blinks into Morse code required precision. The algorithm had to reliably distinguish between short and long blinks, recognize intentional sequences, and filter out involuntary blinks. Fine-tuning timing and detection thresholds was crucial to prevent errors and misinterpretations. Integrating the AI Assistant – Once Morse code was translated into text, we needed to ensure the AI assistant could process the input accurately and return relevant responses. This required optimizing how text was formatted, transmitted, and made compatible with various AI models to provide fast, meaningful, and user-friendly interactions. Integrating the AI Assistant – Once Morse code was translated into text, we needed to ensure the AI assistant could process the input accurately and return relevant responses. This required optimizing how text was formatted, transmitted, and made compatible with various AI models to provide fast, meaningful, and user-friendly interactions. Balancing Speed and Accuracy– The system needed to be both fast and precise. If it processed blinks too quickly, it risked misreading input; too slowly, and it would feel unresponsive. Achieving the right balance required extensive testing, adjustments, and optimizations. Balancing Speed and Accuracy– The system needed to be both fast and precise. If it processed blinks too quickly, it risked misreading input; too slowly, and it would feel unresponsive. Achieving the right balance required extensive testing, adjustments, and optimizations.
### Accomplishments we're proud of
We developed a fully functional prototype that accurately translates blinks into real-time AI-powered communication. We developed a fully functional prototype that accurately translates blinks into real-time AI-powered communication. We created one of the first Morse-code-based blink detection systems, turning blinks into a seamless way to interact with AI. We created one of the first Morse-code-based blink detection systems, turning blinks into a seamless way to interact with AI. We empowered individuals with a new way to communicate, proving that technology can be for everyone. We empowered individuals with a new way to communicate, proving that technology can be for everyone.
### What we learned
Technology should work for people—not the other way around. If it’s not intuitive, it won’t be used. Technology should work for people—not the other way around. If it’s not intuitive, it won’t be used. Small improvements make a big difference. Even a 0.1-second boost in response time completely changed the user experience. Small improvements make a big difference. Even a 0.1-second boost in response time completely changed the user experience. Good technology isn’t about making things complicated—it’s about making life easier. The best solutions take away barriers, not add them.
### What's next
Good technology isn’t about making things complicated—it’s about making life easier. The best solutions take away barriers, not add them. What's next for BlinkAI Making Blink Detection Even Smarter – Improving accuracy so it recognizes blinks more naturally, refines Morse code translation, and responds even faster. Making Blink Detection Even Smarter – Improving accuracy so it recognizes blinks more naturally, refines Morse code translation, and responds even faster. Helping Blink AI Learn From Users – Fine-tuning the system so it adapts to each person’s unique blinking style, making interactions feel effortless. Helping Blink AI Learn From Users – Fine-tuning the system so it adapts to each person’s unique blinking style, making interactions feel effortless. Expanding AI Compatibility – Ensuring Blink AI works smoothly with different AI assistants and platforms, giving users more choices. Expanding AI Compatibility – Ensuring Blink AI works smoothly with different AI assistants and platforms, giving users more choices. Bringing Blink AI to More People – Optimizing it for more devices so anyone who needs it can use it, no matter their setup. Bringing Blink AI to More People – Optimizing it for more devices so anyone who needs it can use it, no matter their setup. Real-World Testing & Feedback – Working directly with users, accessibility experts, and healthcare providers to keep improving Blink AI where it matters most. Real-World Testing & Feedback – Working directly with users, accessibility experts, and healthcare providers to keep improving Blink AI where it matters most.
## README (from the GitHub repository)
No README available.
## Detected evidence (automated analysis)
Indexed codebase: 4 recognized source files, 37 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — claimed on Devpost, not found in the code
## Codebase structure (from repository index)
### Files (6 of 6)
```
morse.py
README.md
requirements.txt
static/css/style.css
templates/index.html
vercel.json
```
### Dependencies
- requirements.txt: bidict@==0.23.1, blinker@==1.9.0, click@==8.1.8, cvzone@==1.6.1, Flask@==3.1.0, Flask-SocketIO@==5.5.1, h11@==0.14.0, imutils@==0.5.4, itsdangerous@==2.2.0, Jinja2@==3.1.5, MarkupSafe@==3.0.2, opencv-contrib-python@==4.11.0.86, opencv-python@==4.11.0.86, opencv-python-headless@==4.11.0.86, python-engineio@==4.11.2, python-socketio@==5.12.1, scipy@==1.15.1, simple-websocket@==1.1.0, Werkzeug@==3.1.3, wsproto@==1.2.0
### Recent commits (newest first)
- Update morse.py
- Update README.md
- Create README.md
- Delete static/style.css
- Add files via upload
- Update requirements.txt
- Update morse.py
- Delete success.log
- Delete shape_predictor_68_face_landmarks.dat.zip
- Add files via upload
- Delete .DS_Store
- Delete shape_predictor_68_face_landmarks.dat
- second commit
- Initial commit
## Key source files (fetched from GitHub, selected and truncated for size)
### requirements.txt
```
bidict==0.23.1
blinker==1.9.0
click==8.1.8
cvzone==1.6.1
Flask==3.1.0
Flask-SocketIO==5.5.1
h11==0.14.0
imutils==0.5.4
itsdangerous==2.2.0
Jinja2==3.1.5
MarkupSafe==3.0.2
opencv-contrib-python==4.11.0.86
opencv-python==4.11.0.86
opencv-python-headless==4.11.0.86
python-engineio==4.11.2
python-socketio==5.12.1
scipy==1.15.1
simple-websocket==1.1.0
Werkzeug==3.1.3
wsproto==1.2.0
```
### morse.py
```python
from time import perf_counter
#import numpy as np
import cv2
import cvzone
from cvzone.FaceMeshModule import FaceMeshDetector
from cvzone.PlotModule import LivePlot
#import time
from flask import Flask, render_template, Response, jsonify
import requests
# ------------------------------
# Mistral AI Configuration
# ------------------------------
MISTRAL_API_URL = "https://api.mistral.ai/v1/chat/completions"
MISTRAL_API_KEY = "Your API KEY"
# ------------------------------
# Global Morse Decoder State
# ------------------------------
# Binary tree representation of Morse code.
# The index (trace-1) corresponds to a letter based on the sequence of dots (0) and dashes (1).
letters = ['','E','T','I','A','N','M','S','U','R','W','D','K','G','O','H','V','F','','L','','P','J','B','X','C','Y','Z','Q','','']
def choose_letter(dots):
if len(dots) > 4:
return 'TOO BIG'
trace = 1
for dot in dots:
if dot == 0:
trace = 2 * trace
else:
trace = (2 * trace) + 1
return letters[int(trace - 1)]
# Global variables to keep track of blink detection and Morse decoding.
start_flag = False # When True, start processing blinks
is_paused = False # When True, pause processing while maintaining state
is_processing = False # For AI processing state
letter = [] # List to hold current letter's Morse signals (0 for dot, 1 for dash)
word = '' # Decoded word (all letters concatenated)
morse = '' # String representation of the current Morse sequence
conversation_history = [] # To store chat history
startTime = 0
notBlinking = 0
blinking = 0
counter = 0
# For smoothing the ratio calculation.
dList = [] # Vertical distances (for averaging)
horList = [] # Horizontal distances (for averaging)
r_list = [] # Ratio values
avg_d = [] # Running average of ratio
# ------------------------------
# Initialize Video and FaceMesh Detector
# ------------------------------
def initialize_camera():
"""Try different camera indices until finding a working one."""
for index in range(10): # Try first 10 camera indices
cap = cv2.VideoCapture(index)
if cap.isOpened():
ret, frame = cap.read()
if ret:
print(f"Successfully opened camera at index {index}")
# Set camera properties
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
cap.set(cv2.CAP_PROP_FPS, 30)
return cap
cap.release()
raise RuntimeError("No working camera found")
try:
video = initialize_camera()
except RuntimeError as e:
print(f"Error: {e}")
exit(1)
detector = FaceMeshDetector(maxFaces=1)
plotY = LivePlot(640, 360, [20, 50]) # Optional live plot
idList = [22, 23, 24, 26, 110, 157, 158, 159, 160, 161, 130, 243] # For drawing landmarks
# ------------------------------
# Flask Application Setup
# ------------------------------
app = Flask(__name__)
@app.route('/')
def index():
return render_template('index.html')
# Endpoint to start blink processing.
@app.route('/start', methods=['POST'])
def start_detection():
global start_flag
start_flag = True
return jsonify({'status': 'started'})
# Endpoint to reset the current Morse message.
@app.route('/reset', methods=['POST'])
def reset_detection():
global letter, word, morse, start_flag, is_paused, conversation_history
letter = []
word = ''
morse = ''
start_flag = False
is_paused = False
conversation_history = []
return jsonify({'status': 'reset'})
# Endpoint to toggle pause state
@app.route('/toggle_pause', methods=['POST'])
def toggle_pause():
global is_paused
is_paused = not is_paused
return jsonify({'status': 'paused' if is_paused else 'resumed'})
# Updated endpoint to get current message state including conversation
@app.route('/get_current_message')
def get_current_message():
global word, morse, conversation_history
return jsonify({
'word': word,
'morse': morse,
'conversation': conversation_history
})
# New endpoint for AI processing
@app.route('/process_message', methods=['POST'])
def process_message():
global word, is_processing, conversation_history
if not word or is_processing:
return jsonify({
'status': 'error',
'message': 'No message to process or already processing'
})
try:
is_processing = True
# Add user message to conversation history
conversation_history.append({
'role': 'user',
'content': word
})
# Prepare the request for Mistral AI
headers = {
'Authorization': f'Bearer {MISTRAL_API_KEY}',
'Content-Type': 'application/json'
}
data = {
'model': 'mistral-large-latest',
'messages': conversation_history,
'max_tokens': 1024,
'temperature': 0.7
}
# Get AI response
response = requests.post(MISTRAL_API_URL, headers=headers, json=data)
response.raise_for_status()
ai_response = response.json()['choices'][0]['message']['content']
# Add AI response to conversation history
conversation_history.append({
'role': 'assistant',
'content': ai_response
})
# Reset the current word after processing
word = ''
return jsonify({
'status': 'success',
'response': ai_response
})
except requests.exceptions.RequestException as e:
return jsonify({
'status': 'error',
'message': f'API Error: {str(e)}'
})
except Exception as e:
return jsonify({
'status': 'error',
'message': str(e)
})
finally:
is_processing = Fal
[truncated — 4148 more characters]
```
### static/css/style.css
```css
* {
margin: 0;
padding: 0;
}
div.box {
background: #FBFCFC;
height: 100vh;
width: 100%;
font-family:'Montserrat', sans-serif;
flex-direction: column;
}
div.div1 {
background: #212F3C;
float: left;
height: 100%;
width: 50%;
}
div.div2 {
height: 100%;
}
.inside {
position: absolute;
bottom: 0;
}
.right{
font-family: "Avant Garde", Avantgarde, "Century Gothic", CenturyGothic, "AppleGothic", sans-serif;
margin-bottom:1%;
left: 60%;
}
div.clear {
clear: both;
height: 1px;
overflow: hidden;
font-size: 0pt;
margin-top: -1px;
}
img {
max-width: 100%;
max-height: 100%;
background-position: cover;
}
.line{
font-size: 35px;
font-weight: bold;
color: #001f3f;
width: 145px;
height: auto;
border-bottom: 4px solid #0074D9;
position: absolute;
/* float: right; */
/* margin: 72px; */
left:70%;
padding: 4px;
top: 6%;
}
#down{
margin: 65px;
}
.menu{
position: absolute;
margin-left: 55%;
margin-top: 15%;
}
.menu a {
position: relative;
display: inline-block;
color: #007bff !important;
text-transform: uppercase;
font-weight: 700;
margin: 0 33px;
outline: 0;
transition: all 0.2s ease-in-out;
text-decoration: none;
}
.menu a:first-of-type {
margin-left: 0px;
}
.menu a:after {
position: absolute;
top: 100%;
left: 0;
width: 100%;
height: 1px;
background: #007bff;
content: "";
opacity: 0;
-webkit-transition: height 0.3s, opacity 0.3s, -webkit-transform 0.3s;
transition: height 0.3s, opacity 0.3s, -webkit-transform 0.3s;
transition: height 0.3s, opacity 0.3s, transform 0.3s;
transition: height 0.3s, opacity 0.3s, transform 0.3s, -webkit-transform 0.3s;
-webkit-transform: translateY(-5px);
transform: translateY(-5px);
}
.menu a:focus:after,
.menu a:hover:after {
height: 2px;
border-radius: 3px;
opacity: 1;
-webkit-transform: translateY(0);
transform: translateY(0);
}
.info{
margin-bottom:3%;
margin-left: 10%;
}
.info{
font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;
font-size: 12px;
float: left;
padding: 1px 5px;
white-space: nowrap;
border: 1px solid #eaeaea;
background-color: #f8f8f8;
border-radius: 3px; }
.letter{
font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;
font-size: 1.2em;
position: absolute;
left: 70%;
top: 50%;
}
#morse{
margin-bottom:15px;
}
.kbd{
display: inline-block;
padding: 3px 5px;
font: 20px "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;
line-height: 10px;
color: #444d56;
vertical-align: middle;
background-color: #fafbfc;
border: solid 1px #d1d5da;
border-bottom-color: #c6cbd1;
border-radius: 3px;
box-shadow: inset 0 -1px 0 #c6cbd1;
}
@media only screen and (max-width: 944px) {
.menu {
margin-top: 20%;
}
}
@media only screen and (max-width: 768px) {
.menu {
margin-top: 25%;
}
}
@media only screen and (max-width: 404px) {
.menu {
margin-top: 43%;
}
}
```
### templates/index.html
```html
Blink AI