# Project export: Text to Dot
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: Cal Hacks 11.0
- Tagline: Real time translation from text to braille.
- Devpost: https://devpost.com/software/text-to-dot
- GitHub: https://github.com/psycho-baller/text2dot
- Result: winner (Deepgram: Deepgram Voice Agent Quest)
- Team: 2 GitHub contributor(s) — Rami Maalouf (14 commits), ethanjags (5 commits)
## Devpost submission (written by the team)
### Inspiration
Our team focuses primarily on implementing innovative technologies to spark positive change. In line with these ideals, we decided to explore the potential impact of a text reading software that not only provided an additional layer of accessibility for blind and deafblind individuals. Through these efforts, we discovered that despite previous attempts to solve similar problems, solutions were often extremely expensive and incongruous. According to many visually challenged advocates these technologies often lacked real-world application and were specific to online texts or readings which limited their opportunities in everyday tasks like reading books or scanning over menus. Upon further research into afflicted groups, we discovered that there was additionally a large population of people who were both deaf and blind which stopped them from utilizing any forms of auditory input as an alternative, significantly obstructing their means of communication. Employing a very human centered design rooted in various personal accounts and professional testimony, we were able to develop a universal design that provides the visually and dual sensory impaired to experience the world from a new lens. By creating a handheld text to braille and speech generator, we are able to revolutionize the prospects of interpersonal communication for these individuals.
### What it does
This solution utilizes a singular piece with two modules, a video camera to decipher text, and a set of solenoids that imitates a standard Grade 2 Braille Grid. This portable accessory is intended to be utilized by a visually impaired or blind deaf individual when they’re attempting to analyze a physical text. This three finger supplement, equipped with a live action camera and sensitive solenoid components, is capable of utilizing a live camera feed to discern the diction of a physical text. Moving forward, the scanned text is moved to an A.I application to clean up the text for either auditory or sensory output in the form of TTS and braille. The text is then adapted into an audio format through a web application or to the classic 6 cells present in the Braille dictionary. Users are given a brief moment to make sense of each braille letter before the system automatically iterates through the remainder of the text. This technology effectively provides users with an alternative method to receive information that isn’t ordinarily accessible to them, granting a more authentic and amplified understanding of the world around them. In this unique application of these technologies, those who are hard of seeing and/or dual sensory impaired receive a more genuine appreciation for texts.
### How we built it
As our project required two extremely different pieces, we decided to split up our resources in hopes of tackling both problems at the same time. Regardless, we needed to set firm goals and plan out our required resources or timeline which helped us stay on schedule and formulate a final product that fully utilized our expertise. In terms of hardware, we were somewhat limited for the first half of the hackathon as we had to purchase many of our materials and were unable to complete much of this work till later. We started by identifying a potential circuit design and creating a rigid structure to house our components. From there we simply spent a large amount of time actually implementing our theoretical circuit and applying it to our housing model in addition to cleaning the whole product up. For software, we mostly had problems with connecting each of the pieces after building them out. We first created an algorithm that could take a camera feed and produce a coherent string of text. This would then be run through an AI text to speech generator that could decipher any gibberish. Finally, these texts would be sent through to either be read out loud or be compared against a dictionary to create a binary code that would dictate the on/off states off our solenoids. Finally, we prototyped our product and tested it to see what we could improve in our final implementation to both increase efficiency and decrease latency.
### Challenges we ran into
This project was extremely technical and ambitious which meant that it was plagued with difficulties. As a large portion of the project relied on its hardware and implementing complementary materials to formulate a cohesive product, there were countless problems throughout the building phase. We often had incompatible parts whether it be cables, Voltage output/input, or even sizing and scaling issues, we were constantly scrambling to alter, scavenge, and adapt materials for uncommon use cases. Even our main board didn’t produce enough power, leading to an unusual usage of a dehumidifier charger and balled up aluminum foil as a makeshift power bank. All of these mechanical complexities followed by a difficult software end of the project led to an innovative and reworked solution that maintained applicative efficiency. These modifications even continued just hours before the submission deadline when we revamped the entire physical end of our project to make use of newly acquired materials using a more efficient modeling technique. These last second improvements gave our product a more polished and adept edge, making a more impactful and satisfying design. Software wise we also strove to uncover the underappreciated features from our various APIs and tools which often didn’t coincide with our team’s strengths. As we had to simultaneously build out an effective product while troubleshooting our software side, we often ran into incompetencies and struggles. Regardless, we were able to overcome these adversities and produce an impressive result.
### Accomplishments we're proud of
We are proud that we were able to overcome the various difficulties that arose throughout our process and to still showcase the level of success that we did even given such a short timeframe. Our team came in with some members having never done a hackathon before and we made extremely ambitious goals that we were unsure we could uphold. However, we were able to effectively work as a team to develop a final product that clearly represents our initial intentions for the project.
### What we learned
As a result of the many cutting-edge sponsors and new technological constraints, our whole team was able to draw from new more effective tools to increase efficiency and quality of our product. Through our careful planning and consistent collaboration, we experienced the future of software and progressed in our intricate technical knowledge within our fields and across specializations. and Because of the cross discipline nature of this project. Additionally, we became more flexible with what materials we needed to build out our hardware applications and especially utilized new TTS technologies to amplify the impact of our projects. In the future, we intend to continue to develop these crucial skills that we obtained at Cal Hacks 11.0, working towards a more accessible future.
### What's next
We would like to work on integrating a more refined design to the hardware component of our project. Unforeseen circumstances with the solenoid led to our final design needing to be adjusted beyond the design of the original model, which could be rectified in future iterations.
## README (from the GitHub repository)
Text2Dot - CalHacks 11 hackathon winner
[Devpost submission](https://devpost.com/software/text-to-dot)
## Inspiration
Our team focuses primarily on implementing innovative technologies to spark positive change. In line with these ideals, we decided to explore the potential impact of a text reading software that not only provided an additional layer of accessibility for blind and deafblind individuals. Through these efforts, we discovered that despite previous attempts to solve similar problems, solutions were often extremely expensive and incongruous. According to many visually challenged advocates these technologies often lacked real-world application and were specific to online texts or readings which limited their opportunities in everyday tasks like reading books or scanning over menus. Upon further research into afflicted groups, we discovered that there was additionally a large population of people who were both deaf and blind which stopped them from utilizing any forms of auditory input as an alternative, significantly obstructing their means of communication. Employing a very human centered design rooted in various personal accounts and professional testimony, we were able to develop a universal design that provides the visually and dual sensory impaired to experience the world from a new lens. By creating a handheld text to braille and speech generator, we are able to revolutionize the prospects of interpersonal communication for these individuals.
## What it does
This solution utilizes a singular piece with two modules, a video camera to decipher text, and a set of solenoids that imitates a standard Grade 2 Braille Grid. This portable accessory is intended to be utilized by a visually impaired or blind deaf individual when they’re attempting to analyze a physical text. This three finger supplement, equipped with a live action camera and sensitive solenoid components, is capable of utilizing a live camera feed to discern the diction of a physical text. Moving forward, the scanned text is moved to an A.I application to clean up the text for either auditory or sensory output in the form of TTS and braille. The text is then adapted into an audio format through a web application or to the classic 6 cells present in the Braille dictionary. Users are given a brief moment to make sense of each braille letter before the system automatically iterates through the remainder of the text. This technology effectively provides users with an alternative method to receive information that isn’t ordinarily accessible to them, granting a more authentic and amplified understanding of the world around them. In this unique application of these technologies, those who are hard of seeing and/or dual sensory impaired receive a more genuine appreciation for texts.
## How we built it
As our project required two extremely different pieces, we decided to split up our resources in hopes of tackling both problems at the same time. Regardless, we needed to set firm goals and plan out our required resources or timeline which helped us stay on schedule and formulate a final product that fully utilized our expertise.
In terms of hardware, we were somewhat limited for the first half of the hackathon as we had to purchase many of our materials and were unable to complete much of this work till later. We started by identifying a potential circuit design and creating a rigid structure to house our components. From there we simply spent a large amount of time actually implementing our theoretical circuit and applying it to our housing model in addition to cleaning the whole product up.
For software, we mostly had problems with connecting each of the pieces after building them out. We first created an algorithm that could take a camera feed and produce a coherent string of text. This would then be run through an AI text to speech generator that could decipher any gibberish. Finally, these texts would be sent through to either be read out loud or be compared against a dictionary to create a binary code that would dictate the on/off states off our solenoids.
Finally, we prototyped our product and tested it to see what we could improve in our final implementation to both increase efficiency and decrease latency.
## Challenges we ran into
This project was extremely technical and ambitious which meant that it was plagued with difficulties. As a large portion of the project relied on its hardware and implementing complementary materials to formulate a cohesive product, there were countless problems throughout the building phase. We often had incompatible parts whether it be cables, Voltage output/input, or even sizing and scaling issues, we were constantly scrambling to alter, scavenge, and adapt materials for uncommon use cases. Even our main board didn’t produce enough power, leading to an unusual usage of a dehumidifier charger and balled up aluminum foil as a makeshift power bank. All of these mechanical complexities followed by a difficult software end of the project led to an innovative and reworked solution that maintained applicative efficiency. These modifications even continued just hours before the submission deadline when we revamped the entire physical end of our project to make use of newly acquired materials using a more efficient modeling technique. These last second improvements gave our product a more polished and adept edge, making a more impactful and satisfying design.
Software wise we also strove to uncover the underappreciated features from our various APIs and tools which often didn’t coincide with our team’s strengths. As we had to simultaneously build out an effective product while troubleshooting our software side, we often ran into incompetencies and struggles. Regardless, we were able to overcome these adversities and produce an impressive result.
## Accomplishments that we're proud of
We are proud that we were able to overcome the various difficulties that arose throughout our process and to still showcase the level of success that we did even given such a short timeframe. Our team came in with some members having never done a hackathon before and we made extremely ambitious goals that we were unsure we could uphold. However, we were able to effectively work as a team to develop a final product that clearly represents our initial intentions for the project.
## What we learned
As a result of the many cutting-edge sponsors and new technological constraints, our whole team was able to draw from new more effective tools to increase efficiency and quality of our product. Through our careful planning and consistent collaboration, we experienced the future of software and progressed in our intricate technical knowledge within our fields and across specializations. and Because of the cross discipline nature of this project. Additionally, we became more flexible with what materials we needed to build out our hardware applications and especially utilized new TTS technologies to amplify the impact of our projects. In the future, we intend to continue to develop these crucial skills that we obtained at Cal Hacks 11.0, working towards a more accessible future.
## What's next for Text to Dot
We would like to work on integrating a more refined design to the hardware component of our project. Unforeseen circumstances with the solenoid led to our final design needing to be adjusted beyond the design of the original model, which could be rectified in future iterations.
## Detected evidence (automated analysis)
Indexed codebase: 25 recognized source files, 451 KB.
- C (language) — detected in the code
- C++ (language) — detected in the code
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- Next.js (technology) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- PyTorch (technology) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
## Codebase structure (from repository index)
### Files (41 of 41)
```
.env.example
.eslintrc.json
.gitignore
app.py
app/api/extract_image/route.ts
app/globals.css
app/layout.tsx
app/page.tsx
backend/cartesia.ipynb
backend/llm.ipynb
backend/main.ipynb
backend/ocr.py
bun.lockb
CameraWebServer_for_esp-arduino_3.0.x/app_httpd.cpp
CameraWebServer_for_esp-arduino_3.0.x/camera_index.h
CameraWebServer_for_esp-arduino_3.0.x/camera_pins.h
CameraWebServer_for_esp-arduino_3.0.x/CameraWebServer_for_esp-arduino_3.0.x.ino
CameraWebServer_for_esp-arduino_3.0.x/README.md
components.json
components/ConnectSocket.tsx
components/ui/button.tsx
components/VapiUI.tsx
hooks/useVapi.ts
hooks/useWebSockets.ts
hooks/useWebSocketWithDeepgram.ts
lib/utils.ts
next.config.mjs
package.json
postcss.config.mjs
README.md
requirements.txt
tailwind.config.ts
tsconfig.json
Xiao_Sense_CameraWebServer_Audio/app_httpd.cpp
Xiao_Sense_CameraWebServer_Audio/audio_server.h
Xiao_Sense_CameraWebServer_Audio/camera_index.h
Xiao_Sense_CameraWebServer_Audio/camera_pins.h
Xiao_Sense_CameraWebServer_Audio/ESP32CAM_stream.html
Xiao_Sense_CameraWebServer_Audio/partitions.csv
Xiao_Sense_CameraWebServer_Audio/README.md
Xiao_Sense_CameraWebServer_Audio/Xiao_Sense_CameraWebServer_Audio.ino
```
### Dependencies
- package.json: @biomejs/biome@^1.9.4, @deepgram/sdk@^3.8.1, @radix-ui/react-slot@^1.1.0, @types/node@^20, @types/react@^18, @types/react-dom@^18, @vapi-ai/web@^2.1.4, class-variance-authority@^0.7.0, clsx@^2.1.1, eslint@^8, eslint-config-next@14.2.15, lucide-react@^0.453.0, next@14.2.15, postcss@^8, react@^18, react-dom@^18, tailwind-merge@^2.5.4, tailwindcss@^3.4.1, tailwindcss-animate@^1.0.7, three@^0.169.0, typescript@^5
- requirements.txt: aenum@==3.1.15, aiofiles@==24.1.0, aiohappyeyeballs@==2.4.3, aiohttp@==3.10.10, aiosignal@==1.3.1, annotated-types@==0.7.0, anyio@==4.6.2.post1, attrs@==24.2.0, blinker@==1.8.2, certifi@==2024.8.30, charset-normalizer@==3.4.0, click@==8.1.7, dataclasses-json@==0.6.7, deepgram-sdk@==3.7.4, deprecation@==2.1.0, distro@==1.9.0, easyocr@==1.7.2, filelock@==3.16.1, Flask@==3.0.3, frozenlist@==1.4.1, fsspec@==2024.9.0, gevent@==24.10.3, greenlet@==3.1.1, h11@==0.14.0, httpcore@==1.0.6, httpx@==0.27.2, idna@==3.10, imageio@==2.36.0, itsdangerous@==2.2.0, Jinja2@==3.1.4, jiter@==0.6.1, lazy_loader@==0.4, MarkupSafe@==3.0.2, marshmallow@==3.23.0, mpmath@==1.3.0, multidict@==6.1.0, mypy-extensions@==1.0.0, networkx@==3.4.1, ninja@==1.11.1.1, numpy@==2.1.2, openai@==1.52.0, opencv-python@==4.10.0.84, opencv-python-headless@==4.10.0.84, packaging@==24.1, pillow@==11.0.0, propcache@==0.2.0, pyclipper@==1.3.0.post6, pydantic@==2.9.2, pydantic_core@==2.23.4, python-bidi@==0.6.3, python-dotenv@==1.0.1, PyYAML@==6.0.2, requests@==2.32.3, scikit-image@==0.24.0, scipy@==1.14.1, setuptools@==75.2.0, shapely@==2.0.6, sniffio@==1.3.1, sympy@==1.13.1, tifffile@==2024.9.20, torch@==2.5.0, torchvision@==0.20.0, tqdm@==4.66.5, typing_extensions@==4.12.2, typing-inspect@==0.9.0, urllib3@==2.2.3, websocket@==0.2.1, websockets@==13.1, Werkzeug@==3.0.4, yarl@==1.15.5, zope.event@==5.0, zope.interface@==7.1.0
### Recent commits (newest first)
- Update README.md
- better docs
- typo
- added the rest of the codebase
- added "device sucessfully connected" audio and immediately disconnect audio when you click that button
- only send over words that were not already shared
- fix backend
- Merge branch 'main' of github.com:psycho-baller/text2dot
- fix ocr
- added reqirements
- BAckend server uses 2 websocket servers to interact with camera and client
- Merge branch 'main' of github.com:psycho-baller/text2dot
- added jupyter files
- cleanup
- deepgram actually works now
- this is a mess
- Added initial trasncription jupyter notebook
- Added API route for hyperbolic
- adding .local to the env
- Added env.example
## Key source files (fetched from GitHub, selected and truncated for size)
### package.json
```
{
"name": "text_to_dot",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"dependencies": {
"@biomejs/biome": "^1.9.4",
"@deepgram/sdk": "^3.8.1",
"@radix-ui/react-slot": "^1.1.0",
"@vapi-ai/web": "^2.1.4",
"class-variance-authority": "^0.7.0",
"clsx": "^2.1.1",
"lucide-react": "^0.453.0",
"next": "14.2.15",
"react": "^18",
"react-dom": "^18",
"tailwind-merge": "^2.5.4",
"tailwindcss-animate": "^1.0.7",
"three": "^0.169.0"
},
"devDependencies": {
"@types/node": "^20",
"@types/react": "^18",
"@types/react-dom": "^18",
"eslint": "^8",
"eslint-config-next": "14.2.15",
"postcss": "^8",
"tailwindcss": "^3.4.1",
"typescript": "^5"
}
}
```
### requirements.txt
```
aenum==3.1.15
aiofiles==24.1.0
aiohappyeyeballs==2.4.3
aiohttp==3.10.10
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.6.2.post1
attrs==24.2.0
blinker==1.8.2
certifi==2024.8.30
charset-normalizer==3.4.0
click==8.1.7
dataclasses-json==0.6.7
deepgram-sdk==3.7.4
deprecation==2.1.0
distro==1.9.0
easyocr==1.7.2
filelock==3.16.1
Flask==3.0.3
frozenlist==1.4.1
fsspec==2024.9.0
gevent==24.10.3
greenlet==3.1.1
h11==0.14.0
httpcore==1.0.6
httpx==0.27.2
idna==3.10
imageio==2.36.0
itsdangerous==2.2.0
Jinja2==3.1.4
jiter==0.6.1
lazy_loader==0.4
MarkupSafe==3.0.2
marshmallow==3.23.0
mpmath==1.3.0
multidict==6.1.0
mypy-extensions==1.0.0
networkx==3.4.1
ninja==1.11.1.1
numpy==2.1.2
openai==1.52.0
opencv-python==4.10.0.84
opencv-python-headless==4.10.0.84
packaging==24.1
pillow==11.0.0
propcache==0.2.0
pyclipper==1.3.0.post6
pydantic==2.9.2
pydantic_core==2.23.4
python-bidi==0.6.3
python-dotenv==1.0.1
PyYAML==6.0.2
requests==2.32.3
scikit-image==0.24.0
scipy==1.14.1
setuptools==75.2.0
shapely==2.0.6
sniffio==1.3.1
sympy==1.13.1
tifffile==2024.9.20
torch==2.5.0
torchvision==0.20.0
tqdm==4.66.5
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.3
websocket==0.2.1
websockets==13.1
Werkzeug==3.0.4
yarl==1.15.5
zope.event==5.0
zope.interface==7.1.0
```
### app.py
```python
import os
from gtts import gTTS
import multiprocessing
import time
import platform
from websockets.sync.server import serve
from flask import Flask, send_from_directory
from flask_cors import CORS # Import CORS
import openai
from deepgram import DeepgramClient, SpeakWSOptions, SpeakWebSocketEvents
from dotenv import load_dotenv
load_dotenv()
model = "aura-asteria-en"
# Flask App
app = Flask(__name__, static_folder="./public", static_url_path="/public")
CORS(
app, resources={r"/*": {"origins": "http://localhost:3000"}}
) # Enable CORS for localhost:3000
def receive_websocket(websocket, queue):
"""
WebSocket server that receives data from clients.
"""
try:
while True:
message = websocket.recv() # Receive message from client
if message:
print(f"Received message: {message}")
# Store the message in the queue
queue.put(message)
except Exception as e:
print(f"Error in receiving WebSocket server: {e}")
def send_websocket(websocket, queue):
"""
WebSocket server that sends data to clients.
"""
global last_time
last_time = time.time() - 5
connected = False
deepgram = DeepgramClient()
dg_connection = deepgram.speak.websocket.v("1")
def on_open(self, open, **kwargs):
print(f"\n\n{open}\n\n")
def on_flush(self, flushed, **kwargs):
print(f"\n\n{flushed}\n\n")
flushed_str = str(flushed)
websocket.send(flushed_str)
def on_binary_data(self, data, **kwargs):
global last_time
print("Received binary data")
if time.time() - last_time > 3:
print("------------ [Binary Data] Attach header.\n")
# WAV audio container header to ensure the audio is playable
header = bytes(
[
0x52,
0x49,
0x46,
0x46, # "RIFF"
0x00,
0x00,
0x00,
0x00, # Placeholder for file size
0x57,
0x41,
0x56,
0x45, # "WAVE"
0x66,
0x6D,
0x74,
0x20, # "fmt "
0x10,
0x00,
0x00,
0x00, # Chunk size (16)
0x01,
0x00, # Audio format (1 for PCM)
0x01,
0x00, # Number of channels (1)
0x80,
0xBB,
0x00,
0x00, # Sample rate (48000)
0x00,
0xEE,
0x02,
0x00, # Byte rate (48000 * 2)
0x02,
0x00, # Block align (2)
0x10,
0x00, # Bits per sample (16)
0x64,
0x61,
0x74,
0x61, # "data"
0x00,
0x00,
0x00,
0x00, # Placeholder for data size
]
)
websocket.send(header)
last_time = time.time()
websocket.send(data)
def on_close(self, close, **kwargs):
print(f"\n\n{close}\n\n")
dg_connection.on(SpeakWebSocketEvents.Open, on_open)
dg_connection.on(SpeakWebSocketEvents.AudioData, on_binary_data)
dg_connection.on(SpeakWebSocketEvents.Flushed, on_flush)
dg_connection.on(SpeakWebSocketEvents.Close, on_close)
try:
# Are we connected to the Deepgram TTS WS?
if not connected:
print("Connecting to Deepgram TTS WebSocket")
options = SpeakWSOptions(
model=model,
encoding="linear16",
sample_rate=48000,
)
if not dg_connection.start(options):
print("Unable to start Deepgram TTS WebSocket connection")
connected = True
spoken_words = set()
while True:
if not queue.empty():
last_message = queue.get() # Get the last message from the queue
if last_message in spoken_words:
continue
spoken_words.add(last_message)
print(f"Sending message: {last_message}")
dg_connection.send_text(last_message)
dg_connection.flush() # Make sure the data is flushed
time.sleep(1) # Add a slight delay between messages
else:
time.sleep(1) # Wait for a message to be received
except Exception as e:
print(f"Error in sending WebSocket server: {e}")
dg_connection.finish()
def run_ws_receive(queue):
"""
Runs the WebSocket server that receives messages from clients on localhost:4000.
"""
with serve(lambda ws: receive_websocket(ws, queue), "localhost", 4000) as server:
server.serve_forever()
def run_ws_send(queue):
"""
Runs the WebSocket server that sends messages to clients on localhost:4100.
"""
with serve(lambda ws: send_websocket(ws, queue), "localhost", 4100) as server:
server.serve_forever()
@app.route("/")
def serve_others(filename):
return send_from_directory(app.static_folder, filename)
@app.route("/audio/connected")
def serve_connected_audio():
text = "Device successfully connected."
tts = gTTS(text=text, lang="en")
audio_file_path = "connected.mp3"
tts.save(audio_file_path)
return send_from_directory(os.getcwd(), audio_file_path)
@app.route("/assets/")
def serve_image(filename):
return send_from_directory(app.static_folder, "assets/" + filename)
@app.route("/", methods=["GET"])
def serve_index():
return app.send_
[truncated — 837 more characters]
```
### app/layout.tsx
```typescript
import type { Metadata } from "next";
import localFont from "next/font/local";
import "./globals.css";
const geistSans = localFont({
src: "./fonts/GeistVF.woff",
variable: "--font-geist-sans",
weight: "100 900",
});
const geistMono = localFont({
src: "./fonts/GeistMonoVF.woff",
variable: "--font-geist-mono",
weight: "100 900",
});
export const metadata: Metadata = {
title: "Create Next App",
description: "Generated by create next app",
};
export default function RootLayout({
children,
}: Readonly<{
children: React.ReactNode;
}>) {
return (
{children}
);
}
```
### app/page.tsx
```typescript
import React from "react";
import ConnectSocket from "@/components/ConnectSocket"; // Adjust the import path as necessary
import VapiUI from "@/components/VapiUI"; // Adjust the import path as necessary
const CombinedPage: React.FC = () => {
return (
Text To Dot
{/* h1 */}
{/* ConnectSocket and VapiUI components */}
);
};
export default CombinedPage;
```
### app/api/extract_image/route.ts
```typescript
import { NextRequest, NextResponse } from 'next/server';
const HYPERBOLIC_API_KEY = process.env.HYPERBOLIC_API_KEY!;
export async function POST(req: NextRequest) {
try {
const { content } = await req.json();
const url = 'https://api.hyperbolic.xyz/v1/chat/completions';
const response = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${HYPERBOLIC_API_KEY}`,
},
body: JSON.stringify({
model: 'meta-llama/Llama-3.2-90B-Vision-Instruct',
messages: [
{
role: 'user',
content: [
{ type: 'text', text: 'What is this image?' },
{
type: 'image_url',
image_url: {
url: 'https://i.natgeofe.com/n/4f5aaece-3300-41a4-b2a8-ed2708a0a27c/domestic-dog_thumb_square.jpg',
},
},
],
},
],
max_tokens: 2048,
temperature: 0.7,
top_p: 0.9,
stream: false,
}),
});
if (!response.ok) {
return NextResponse.json(
{ error: 'Failed to fetch response from Hyperbolic API' },
{ status: response.status }
);
}
const json = await response.json();
const output = json.choices[0].message.content;
return NextResponse.json({ output });
} catch (error) {
return NextResponse.json(
{ error: 'Something went wrong', details: error },
{ status: 500 }
);
}
}
```
### tailwind.config.ts
```typescript
import type { Config } from "tailwindcss";
const config: Config = {
darkMode: ["class"],
content: [
"./pages/**/*.{js,ts,jsx,tsx,mdx}",
"./components/**/*.{js,ts,jsx,tsx,mdx}",
"./app/**/*.{js,ts,jsx,tsx,mdx}",
],
theme: {
extend: {
colors: {
background: 'hsl(var(--background))',
foreground: 'hsl(var(--foreground))',
card: {
DEFAULT: 'hsl(var(--card))',
foreground: 'hsl(var(--card-foreground))'
},
popover: {
DEFAULT: 'hsl(var(--popover))',
foreground: 'hsl(var(--popover-foreground))'
},
primary: {
DEFAULT: 'hsl(var(--primary))',
foreground: 'hsl(var(--primary-foreground))'
},
secondary: {
DEFAULT: 'hsl(var(--secondary))',
foreground: 'hsl(var(--secondary-foreground))'
},
muted: {
DEFAULT: 'hsl(var(--muted))',
foreground: 'hsl(var(--muted-foreground))'
},
accent: {
DEFAULT: 'hsl(var(--accent))',
foreground: 'hsl(var(--accent-foreground))'
},
destructive: {
DEFAULT: 'hsl(var(--destructive))',
foreground: 'hsl(var(--destructive-foreground))'
},
border: 'hsl(var(--border))',
input: 'hsl(var(--input))',
ring: 'hsl(var(--ring))',
chart: {
'1': 'hsl(var(--chart-1))',
'2': 'hsl(var(--chart-2))',
'3': 'hsl(var(--chart-3))',
'4': 'hsl(var(--chart-4))',
'5': 'hsl(var(--chart-5))'
}
},
borderRadius: {
lg: 'var(--radius)',
md: 'calc(var(--radius) - 2px)',
sm: 'calc(var(--radius) - 4px)'
}
}
},
plugins: [require("tailwindcss-animate")],
};
export default config;
```
### lib/utils.ts
```typescript
import { clsx, type ClassValue } from "clsx"
import { twMerge } from "tailwind-merge"
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs))
}
```
### app/globals.css
```css
@tailwind base;
@tailwind components;
@tailwind utilities;
body {
font-family: Arial, Helvetica, sans-serif;
}
@layer utilities {
.text-balance {
text-wrap: balance;
}
}
@layer base {
:root {
--background: 0 0% 100%;
--foreground: 0 0% 3.9%;
--card: 0 0% 100%;
--card-foreground: 0 0% 3.9%;
--popover: 0 0% 100%;
--popover-foreground: 0 0% 3.9%;
--primary: 0 0% 9%;
--primary-foreground: 0 0% 98%;
--secondary: 0 0% 96.1%;
--secondary-foreground: 0 0% 9%;
--muted: 0 0% 96.1%;
--muted-foreground: 0 0% 45.1%;
--accent: 0 0% 96.1%;
--accent-foreground: 0 0% 9%;
--destructive: 0 84.2% 60.2%;
--destructive-foreground: 0 0% 98%;
--border: 0 0% 89.8%;
--input: 0 0% 89.8%;
--ring: 0 0% 3.9%;
--chart-1: 12 76% 61%;
--chart-2: 173 58% 39%;
--chart-3: 197 37% 24%;
--chart-4: 43 74% 66%;
--chart-5: 27 87% 67%;
--radius: 0.5rem;
}
.dark {
--background: 0 0% 3.9%;
--foreground: 0 0% 98%;
--card: 0 0% 3.9%;
--card-foreground: 0 0% 98%;
--popover: 0 0% 3.9%;
--popover-foreground: 0 0% 98%;
--primary: 0 0% 98%;
--primary-foreground: 0 0% 9%;
--secondary: 0 0% 14.9%;
--secondary-foreground: 0 0% 98%;
--muted: 0 0% 14.9%;
--muted-foreground: 0 0% 63.9%;
--accent: 0 0% 14.9%;
--accent-foreground: 0 0% 98%;
--destructive: 0 62.8% 30.6%;
--destructive-foreground: 0 0% 98%;
--border: 0 0% 14.9%;
--input: 0 0% 14.9%;
--ring: 0 0% 83.1%;
--chart-1: 220 70% 50%;
--chart-2: 160 60% 45%;
--chart-3: 30 80% 55%;
--chart-4: 280 65% 60%;
--chart-5: 340 75% 55%;
}
}
@layer base {
* {
@apply border-border;
}
body {
@apply bg-background text-foreground;
}
}
```
### Xiao_Sense_CameraWebServer_Audio/ESP32CAM_stream.html
```html
ESP32-CAM + Livestream Audio
ESP32-CAM + Livestream Audio
```
[12 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]