# Project export: Bull Rider
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: Cal Hacks 11.0
- Tagline: Seamless blockchain transactions made easy via AI guided tutorial
- Devpost: https://devpost.com/software/bull-rider
- GitHub: https://github.com/tusharg0101/bull_rider
- Team: 4 GitHub contributor(s) — tusharg0101 (23 commits), Ronak Agarwal (15 commits), iabhi4 (8 commits), krith-raju (4 commits)
## Devpost submission (written by the team)
### Inspiration
In the rapidly evolving world of blockchain and cryptocurrency, many users find themselves overwhelmed by the complexity of managing digital assets. We were inspired to create Bull Rider after observing the steep learning curve faced by newcomers to the Sui blockchain ecosystem. Our goal was to develop an intuitive, voice-controlled assistant that simplifies blockchain interactions and makes cryptocurrency management accessible to everyone. What It Does Bull Rider is an innovative voice-controlled assistant designed to simplify interactions with the Sui blockchain. Key features include: Voice-activated tutorials: Users can ask questions about Sui wallet operations, and Bull Rider provides step-by-step audio guidance, complemented by on-screen instructions. Voice-controlled transactions: Users can initiate cryptocurrency transfers using natural language commands, making sending tokens as easy as speaking to a friend. Context-aware assistance: Bull Rider uses RAG (Retrieval-Augmented Generation) to provide accurate, up-to-date information about Sui wallet operations. Dynamic tutorial generation: The assistant analyzes the user's screen and query to create personalized, context-specific tutorials. Seamless integration: Bull Rider operates as a menu bar application, always ready to assist without interrupting the user's workflow. How We Built It Bull Rider is built using a combination of cutting-edge technologies: Frontend: Python with rumps for the macOS menu bar interface. Backend: FastAPI for the REST API. Natural Language Processing: Groq API for parsing voice commands and generating responses. Speech-to-Text and Text-to-Speech: Deepgram API for accurate transcription and natural-sounding speech synthesis. Image Analysis: Hyperbolic API for screen capture analysis and tutorial generation. Database: SQLite for lightweight, serverless data storage. RAG System: Sentence Transformers and FAISS for efficient information retrieval. Blockchain Integration: Custom Sui blockchain client for executing transactions. Challenges We Ran Into Integrating multiple AI services (Groq, Deepgram, Hyperbolic) seamlessly. Implementing an efficient RAG system for context-aware responses. Ensuring accurate voice command parsing for blockchain transactions. Optimizing the tutorial generation process for real-time responsiveness. Balancing between providing detailed guidance and maintaining simplicity in user interactions. Accomplishments That We're Proud Of Creating a voice-controlled assistant that simplifies complex blockchain operations. Successfully implementing a RAG system for providing accurate, context-aware information. Developing a dynamic tutorial generation system that adapts to the user's screen and query. Integrating multiple AI services to create a seamless, intelligent user experience. Building a non-intrusive, always-available assistant as a menu bar application. What We Learned The importance of context in AI-generated responses for blockchain applications. Techniques for efficient information retrieval and embedding in RAG systems. Strategies for integrating multiple AI services into a cohesive application. The complexities of voice-controlled interfaces for financial transactions. The potential of AI to simplify complex technological interactions. What's Next for Bull Rider Expanding support for multiple blockchain ecosystems beyond Sui. Implementing more advanced voice authentication for enhanced security. Developing a mobile version of the assistant for on-the-go blockchain management. Integrating real-time market data and portfolio management features. Collaborating with blockchain projects to provide tailored assistance for specific dApps and services. Implementing a feedback loop to continuously improve the RAG system and tutorial generation. Bull Rider represents a significant step towards making blockchain technology accessible to everyone. By combining voice control, AI-driven assistance, and intuitive design, we're paving the way for wider adoption of cryptocurrency and decentralized technologies.
## README (from the GitHub repository)
# Bull Rider
## Inspiration
In the rapidly evolving world of blockchain and cryptocurrency, many users find themselves overwhelmed by the complexity of managing digital assets. We were inspired to create **Bull Rider** after observing the steep learning curve faced by newcomers to the Sui blockchain ecosystem. Our goal was to develop an intuitive, voice-controlled assistant that simplifies blockchain interactions and makes cryptocurrency management accessible to everyone.
## What It Does
**Bull Rider** is an innovative voice-controlled assistant designed to simplify interactions with the Sui blockchain. Key features include:
- **Voice-activated tutorials**: Users can ask questions about Sui wallet operations, and Bull Rider provides step-by-step audio guidance, complemented by on-screen instructions.
- **Voice-controlled transactions**: Users can initiate cryptocurrency transfers using natural language commands, making sending tokens as easy as speaking to a friend.
- **Context-aware assistance**: Bull Rider uses RAG (Retrieval-Augmented Generation) to provide accurate, up-to-date information about Sui wallet operations.
- **Dynamic tutorial generation**: The assistant analyzes the user's screen and query to create personalized, context-specific tutorials.
- **Seamless integration**: Bull Rider operates as a menu bar application, always ready to assist without interrupting the user's workflow.
## How We Built It
Bull Rider is built using a combination of cutting-edge technologies:
- **Frontend**: Python with rumps for the macOS menu bar interface.
- **Backend**: FastAPI for the REST API.
- **Natural Language Processing**: Groq API for parsing voice commands and generating responses.
- **Speech-to-Text and Text-to-Speech**: Deepgram API for accurate transcription and natural-sounding speech synthesis.
- **Image Analysis**: Hyperbolic API for screen capture analysis and tutorial generation.
- **Database**: SQLite for lightweight, serverless data storage.
- **RAG System**: Sentence Transformers and FAISS for efficient information retrieval.
- **Blockchain Integration**: Custom Sui blockchain client for executing transactions.
## Challenges We Ran Into
- Integrating multiple AI services (Groq, Deepgram, Hyperbolic) seamlessly.
- Implementing an efficient RAG system for context-aware responses.
- Ensuring accurate voice command parsing for blockchain transactions.
- Optimizing the tutorial generation process for real-time responsiveness.
- Balancing between providing detailed guidance and maintaining simplicity in user interactions.
## Accomplishments That We're Proud Of
- Creating a voice-controlled assistant that simplifies complex blockchain operations.
- Successfully implementing a RAG system for providing accurate, context-aware information.
- Developing a dynamic tutorial generation system that adapts to the user's screen and query.
- Integrating multiple AI services to create a seamless, intelligent user experience.
- Building a non-intrusive, always-available assistant as a menu bar application.
## What We Learned
- The importance of context in AI-generated responses for blockchain applications.
- Techniques for efficient information retrieval and embedding in RAG systems.
- Strategies for integrating multiple AI services into a cohesive application.
- The complexities of voice-controlled interfaces for financial transactions.
- The potential of AI to simplify complex technological interactions.
## What's Next for Bull Rider
1. Expanding support for multiple blockchain ecosystems beyond Sui.
2. Implementing more advanced voice authentication for enhanced security.
3. Developing a mobile version of the assistant for on-the-go blockchain management.
4. Integrating real-time market data and portfolio management features.
5. Collaborating with blockchain projects to provide tailored assistance for specific dApps and services.
6. Implementing a feedback loop to continuously improve the RAG system and tutorial generation.
**Bull Rider** represents a significant step towards making blockchain technology accessible to everyone. By combining voice control, AI-driven assistance, and intuitive design, we're paving the way for wider adoption of cryptocurrency and decentralized technologies.
## Detected evidence (automated analysis)
Indexed codebase: 41 recognized source files, 74 KB.
- CSS (language) — detected in the code
- FastAPI (technology) — detected in the code
- HTML (language) — detected in the code
- Hugging Face (technology) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- PyTorch (technology) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
## Codebase structure (from repository index)
### Files (51 of 51)
```
.gitignore
backend/__init__.py
backend/main.py
backend/Procfile
backend/requirements.txt
backend/routes.py
backend/services/__init__.py
backend/services/certs/singlestore_bundle.pem
backend/services/db.py
backend/services/deepgram.py
backend/services/feedback.py
backend/services/groq.py
backend/services/hyperbolic.py
backend/services/scrape_and_rag.py
backend/services/singlestore_project1.py
backend/services/sui.py
backend/services/test_db_connection.py
backend/services/voice_auth.py
backend/tutorial.db
frontend/br-nlp-web_app/.gitignore
frontend/br-nlp-web_app/package.json
frontend/br-nlp-web_app/public/index.html
frontend/br-nlp-web_app/public/manifest.json
frontend/br-nlp-web_app/public/robots.txt
frontend/br-nlp-web_app/README.md
frontend/br-nlp-web_app/src/App.css
frontend/br-nlp-web_app/src/App.js
frontend/br-nlp-web_app/src/App.test.js
frontend/br-nlp-web_app/src/components/Footer.js
frontend/br-nlp-web_app/src/components/Header.js
frontend/br-nlp-web_app/src/components/HeroSection.css
frontend/br-nlp-web_app/src/components/HeroSection.js
frontend/br-nlp-web_app/src/components/MainContent.js
frontend/br-nlp-web_app/src/index.css
frontend/br-nlp-web_app/src/index.js
frontend/br-nlp-web_app/src/pages/Contacts.css
frontend/br-nlp-web_app/src/pages/Contacts.js
frontend/br-nlp-web_app/src/pages/Home.js
frontend/br-nlp-web_app/src/pages/LandingPage.css
frontend/br-nlp-web_app/src/pages/LandingPage.js
frontend/br-nlp-web_app/src/pages/Profile.css
frontend/br-nlp-web_app/src/pages/Profile.js
frontend/br-nlp-web_app/src/pages/SignUp.js
frontend/br-nlp-web_app/src/reportWebVitals.js
frontend/br-nlp-web_app/src/setupTests.js
frontend/br-nlp-web_app/tailwind.config.js
frontend/menu_bar/menu_app.py
frontend/menu_bar/utils.py
main.py
README.md
requirements.txt
```
### Dependencies
- backend/requirements.txt: aenum@==3.1.15, aiofiles@==24.1.0, aiohappyeyeballs@==2.4.3, aiohttp@==3.10.10, aiomysql@==0.2.0, aiosignal@==1.3.1, aiosqlite@==0.20.0, annotated-types@==0.7.0, anyio@==4.6.2.post1, attrs@==24.2.0, certifi@==2024.8.30, charset-normalizer@==3.4.0, click@==8.1.7, dataclasses-json@==0.6.7, deepgram-sdk@==3.7.4, deprecation@==2.1.0, distro@==1.9.0, dnspython@==2.7.0, email_validator@==2.2.0, fastapi@==0.115.2, fastapi-cli@==0.0.5, frozenlist@==1.4.1, groq@==0.11.0, h11@==0.14.0, httpcore@==1.0.6, httptools@==0.6.4, httpx@==0.27.2, idna@==3.10, Jinja2@==3.1.4, jsonschema@==4.23.0, jsonschema-specifications@==2024.10.1, markdown-it-py@==3.0.0, MarkupSafe@==3.0.2, marshmallow@==3.23.0, mdurl@==0.1.2, MouseInfo@==0.1.3, mss@==9.0.2, multidict@==6.1.0, mypy-extensions@==1.0.0, packaging@==24.1, pillow@==11.0.0, propcache@==0.2.0, PyAudio@==0.2.14, pydantic@==2.9.2, pydantic_core@==2.23.4, pydub@==0.25.1, PyGetWindow@==0.0.9, Pygments@==2.18.0, PyMsgBox@==1.0.9, PyMySQL@==1.1.1, pyobjc-core@==10.3.1, pyobjc-framework-Cocoa@==10.3.1, pyobjc-framework-Quartz@==10.3.1, pyperclip@==1.9.0, PyRect@==0.2.0, PyScreeze@==1.0.1, python-dotenv@==1.0.1, python-multipart@==0.0.12, pytweening@==1.2.0, PyYAML@==6.0.2, referencing@==0.35.1, requests@==2.32.3, rich@==13.9.2, rpds-py@==0.20.0, rubicon-objc@==0.4.9, rumps@==0.4.0, shellingham@==1.5.4, sniffio@==1.3.1, starlette@==0.40.0, typer@==0.12.5, typing_extensions@==4.12.2, typing-inspect@==0.9.0, urllib3@==2.2.3, uvicorn@==0.32.0, uvloop@==0.21.0, watchfiles@==0.24.0, websockets@==13.1, yarl@==1.15.5
- frontend/br-nlp-web_app/package.json: @testing-library/jest-dom@^5.14.1, @testing-library/react@^13.0.0, @testing-library/user-event@^13.2.1, autoprefixer@^10.4.20, axios@^1.7.7, mic-recorder-to-mp3@^2.2.2, postcss@^8.4.47, react@^18.3.1, react-dom@^18.3.1, react-router-dom@^6.27.0, react-scripts@^5.0.1, reconnecting-websocket@^4.4.0, recordrtc@^5.6.2, tailwindcss@^3.4.14, web-vitals@^2.1.0
- requirements.txt: aenum@==3.1.15, aiofiles@==24.1.0, aiohappyeyeballs@==2.4.3, aiohttp@==3.10.10, aiomysql@==0.2.0, aiosignal@==1.3.1, aiosqlite@==0.20.0, annotated-types@==0.7.0, anyio@==4.6.2.post1, astroid@==3.3.5, attrs@==24.2.0, beautifulsoup4@==4.12.3, bs4@==0.0.2, certifi@==2024.8.30, charset-normalizer@==3.4.0, click@==8.1.7, dataclasses-json@==0.6.7, deepgram-sdk@==3.7.4, deprecation@==2.1.0, dill@==0.3.9, distro@==1.9.0, dnspython@==2.7.0, email_validator@==2.2.0, faiss-cpu@==1.9.0, fastapi@==0.115.2, fastapi-cli@==0.0.5, filelock@==3.16.1, frozenlist@==1.4.1, fsspec@==2024.9.0, groq@==0.11.0, h11@==0.14.0, httpcore@==1.0.6, httptools@==0.6.4, httpx@==0.27.2, huggingface-hub@==0.26.0, idna@==3.10, isort@==5.13.2, Jinja2@==3.1.4, joblib@==1.4.2, jsonschema@==4.23.0, jsonschema-specifications@==2024.10.1, markdown-it-py@==3.0.0, MarkupSafe@==3.0.2, marshmallow@==3.23.0, mccabe@==0.7.0, mdurl@==0.1.2, MouseInfo@==0.1.3, mpmath@==1.3.0, mss@==9.0.2, multidict@==6.1.0, mypy-extensions@==1.0.0, networkx@==3.4.1, numpy@==2.1.2, packaging@==24.1, pillow@==11.0.0, platformdirs@==4.3.6, propcache@==0.2.0, PyAudio@==0.2.14, pydantic@==2.9.2, pydantic_core@==2.23.4, pydub@==0.25.1, PyGetWindow@==0.0.9, Pygments@==2.18.0, pylint@==3.3.1, PyMsgBox@==1.0.9, PyMySQL@==1.1.1, pyobjc-core@==10.3.1, pyobjc-framework-Cocoa@==10.3.1, pyobjc-framework-Quartz@==10.3.1, pyperclip@==1.9.0, PyRect@==0.2.0, PyScreeze@==1.0.1, python-dotenv@==1.0.1, python-multipart@==0.0.12, pytweening@==1.2.0, PyYAML@==6.0.2, referencing@==0.35.1, regex@==2024.9.11, requests@==2.32.3, rich@==13.9.2, rpds-py@==0.20.0, rubicon-objc@==0.4.9, rumps@==0.4.0, safetensors@==0.4.5, scikit-learn@==1.5.2, scipy@==1.14.1, sentence-transformers@==3.2.0, setuptools@==75.2.0, shellingham@==1.5.4, sniffio@==1.3.1, soupsieve@==2.6, starlette@==0.40.0, sympy@==1.13.1, threadpoolctl@==3.5.0, tokenizers@==0.20.1, tomlkit@==0.13.2, torch@==2.5.0, tqdm@==4.66.5, transformers@==4.45.2, typer@==0.12.5, typing_extensions@==4.12.2, typing-inspect@==0.9.0, urllib3@==2.2.3, uvicorn@==0.32.0, uvloop@==0.21.0, watchfiles@==0.24.0, websockets@==13.1, yarl@==1.15.5
### Recent commits (newest first)
- Update README.md
- Merge branch 'main' of github.com:tusharg0101/bull_rider
- fixes
- Merge pull request #6 from tusharg0101/ronak
- Added files via upload
- Merge branch 'ronak' of https://github.com/tusharg0101/bull_rider into ronak
- Styling
- Landing Page:
- Merge branch 'ronak' of https://github.com/tusharg0101/bull_rider into ronak
- Sign up to DB working, login not working
- Update db.py
- Update sui.py
- Update groq.py
- Add files via upload
- Update routes.py
- Merge pull request #5 from tusharg0101/rag
- Resolved merge conflicts + frontend changes
- rag successful
- attempt at rag
- Merge pull request #3 from tusharg0101/change_model
## Key source files (fetched from GitHub, selected and truncated for size)
### requirements.txt
```
aenum==3.1.15
aiofiles==24.1.0
aiohappyeyeballs==2.4.3
aiohttp==3.10.10
aiomysql==0.2.0
aiosignal==1.3.1
aiosqlite==0.20.0
annotated-types==0.7.0
anyio==4.6.2.post1
astroid==3.3.5
attrs==24.2.0
beautifulsoup4==4.12.3
bs4==0.0.2
certifi==2024.8.30
charset-normalizer==3.4.0
click==8.1.7
dataclasses-json==0.6.7
deepgram-sdk==3.7.4
deprecation==2.1.0
dill==0.3.9
distro==1.9.0
dnspython==2.7.0
email_validator==2.2.0
faiss-cpu==1.9.0
fastapi==0.115.2
fastapi-cli==0.0.5
filelock==3.16.1
frozenlist==1.4.1
fsspec==2024.9.0
groq==0.11.0
h11==0.14.0
httpcore==1.0.6
httptools==0.6.4
httpx==0.27.2
huggingface-hub==0.26.0
idna==3.10
isort==5.13.2
Jinja2==3.1.4
joblib==1.4.2
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
markdown-it-py==3.0.0
MarkupSafe==3.0.2
marshmallow==3.23.0
mccabe==0.7.0
mdurl==0.1.2
MouseInfo==0.1.3
mpmath==1.3.0
mss==9.0.2
multidict==6.1.0
mypy-extensions==1.0.0
networkx==3.4.1
numpy==2.1.2
packaging==24.1
pillow==11.0.0
platformdirs==4.3.6
propcache==0.2.0
PyAudio==0.2.14
pydantic==2.9.2
pydantic_core==2.23.4
pydub==0.25.1
PyGetWindow==0.0.9
Pygments==2.18.0
pylint==3.3.1
PyMsgBox==1.0.9
PyMySQL==1.1.1
pyobjc-core==10.3.1
pyobjc-framework-Cocoa==10.3.1
pyobjc-framework-Quartz==10.3.1
pyperclip==1.9.0
PyRect==0.2.0
PyScreeze==1.0.1
python-dotenv==1.0.1
python-multipart==0.0.12
pytweening==1.2.0
PyYAML==6.0.2
referencing==0.35.1
regex==2024.9.11
requests==2.32.3
rich==13.9.2
rpds-py==0.20.0
rubicon-objc==0.4.9
rumps==0.4.0
safetensors==0.4.5
scikit-learn==1.5.2
scipy==1.14.1
sentence-transformers==3.2.0
setuptools==75.2.0
shellingham==1.5.4
sniffio==1.3.1
soupsieve==2.6
starlette==0.40.0
sympy==1.13.1
threadpoolctl==3.5.0
tokenizers==0.20.1
tomlkit==0.13.2
torch==2.5.0
tqdm==4.66.5
transformers==4.45.2
typer==0.12.5
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.3
uvicorn==0.32.0
uvloop==0.21.0
watchfiles==0.24.0
websockets==13.1
yarl==1.15.5
```
### backend/requirements.txt
```
aenum==3.1.15
aiofiles==24.1.0
aiohappyeyeballs==2.4.3
aiohttp==3.10.10
aiomysql==0.2.0
aiosignal==1.3.1
aiosqlite==0.20.0
annotated-types==0.7.0
anyio==4.6.2.post1
attrs==24.2.0
certifi==2024.8.30
charset-normalizer==3.4.0
click==8.1.7
dataclasses-json==0.6.7
deepgram-sdk==3.7.4
deprecation==2.1.0
distro==1.9.0
dnspython==2.7.0
email_validator==2.2.0
fastapi==0.115.2
fastapi-cli==0.0.5
frozenlist==1.4.1
groq==0.11.0
h11==0.14.0
httpcore==1.0.6
httptools==0.6.4
httpx==0.27.2
idna==3.10
Jinja2==3.1.4
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
markdown-it-py==3.0.0
MarkupSafe==3.0.2
marshmallow==3.23.0
mdurl==0.1.2
MouseInfo==0.1.3
mss==9.0.2
multidict==6.1.0
mypy-extensions==1.0.0
packaging==24.1
pillow==11.0.0
propcache==0.2.0
PyAudio==0.2.14
pydantic==2.9.2
pydantic_core==2.23.4
pydub==0.25.1
PyGetWindow==0.0.9
Pygments==2.18.0
PyMsgBox==1.0.9
PyMySQL==1.1.1
pyobjc-core==10.3.1
pyobjc-framework-Cocoa==10.3.1
pyobjc-framework-Quartz==10.3.1
pyperclip==1.9.0
PyRect==0.2.0
PyScreeze==1.0.1
python-dotenv==1.0.1
python-multipart==0.0.12
pytweening==1.2.0
PyYAML==6.0.2
referencing==0.35.1
requests==2.32.3
rich==13.9.2
rpds-py==0.20.0
rubicon-objc==0.4.9
rumps==0.4.0
shellingham==1.5.4
sniffio==1.3.1
starlette==0.40.0
typer==0.12.5
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.3
uvicorn==0.32.0
uvloop==0.21.0
watchfiles==0.24.0
websockets==13.1
yarl==1.15.5
```
### frontend/br-nlp-web_app/package.json
```
{
"name": "br-nlp-web_app",
"version": "0.1.0",
"private": true,
"dependencies": {
"@testing-library/jest-dom": "^5.14.1",
"@testing-library/react": "^13.0.0",
"@testing-library/user-event": "^13.2.1",
"axios": "^1.7.7",
"mic-recorder-to-mp3": "^2.2.2",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-router-dom": "^6.27.0",
"react-scripts": "^5.0.1",
"reconnecting-websocket": "^4.4.0",
"recordrtc": "^5.6.2",
"web-vitals": "^2.1.0"
},
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject"
},
"eslintConfig": {
"extends": [
"react-app",
"react-app/jest"
]
},
"browserslist": {
"production": [
">0.2%",
"not dead",
"not op_mini all"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
},
"devDependencies": {
"autoprefixer": "^10.4.20",
"postcss": "^8.4.47",
"tailwindcss": "^3.4.14"
}
}
```
### main.py
```python
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
async def read_root():
return {"message": "Hello, FastAPI World!"}
@app.get("/items/{item_id}")
async def read_item(item_id: int, q: str = None):
return {"item_id": item_id, "q": q}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="127.0.0.1", port=8000)
```
### backend/main.py
```python
import os
from dotenv import load_dotenv
# Load environment variables from the .env file
load_dotenv()
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from routes import router # Main routes
from services.voice_auth import router as voice_auth_router # Voice auth routes
import uvicorn
app = FastAPI()
# Add CORS middleware to allow localhost requests
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # In production, replace "*" with specific domains
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Include routers
app.include_router(router)
app.include_router(voice_auth_router)
@app.get("/")
def read_root():
return {"message": "Hello, FastAPI!"}
if __name__ == "__main__":
uvicorn.run("main:app", host="0.0.0.0", port=8084, reload=True)
```
### frontend/br-nlp-web_app/src/index.js
```javascript
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';
const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
);
// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();
```
### frontend/br-nlp-web_app/src/App.js
```javascript
// App.js
import React from "react";
import { BrowserRouter as Router, Route, Routes, useLocation } from "react-router-dom";
import "./App.css";
import Header from "./components/Header";
import HeroSection from "./components/HeroSection";
import Footer from "./components/Footer";
import SignUp from "./pages/SignUp";
import LandingPage from "./pages/LandingPage";
import Profile from "./pages/Profile";
import Contacts from "./pages/Contacts"; // Import the Contacts component
function AppWrapper() {
const location = useLocation();
// Hide header on specific pages
const hideHeaderPages = ["/profile", "/contacts", "/login"];
const isHeaderVisible = !hideHeaderPages.includes(location.pathname);
return (
{/* Render Header only if it's not a hidden page */}
{isHeaderVisible && }
} />
} />
} /> {/* Login page */}
} />
} />
);
}
function App() {
return (
);
}
export default App;
```
### backend/routes.py
```python
import os
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from services.deepgram import transcribe_audio, generate_speech
from services.groq import parse_transaction
from services.sui import get_sui_client, send_transaction
from services.db import store_audio, get_audio, get_total_steps, clear_audio, init_db, init_name_address, get_address_from_name, set_name_address
import logging
from dotenv import load_dotenv
import asyncio
import json
from services.scrape_and_rag import init_rag
from services.hyperbolic import generate_tutorial
# Set up logging
log_directory = "logs"
if not os.path.exists(log_directory):
os.makedirs(log_directory)
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler(f"{log_directory}/app.log"),
logging.StreamHandler()
]
)
logger = logging.getLogger(__name__)
router = APIRouter()
ROOT_DIR = os.path.abspath(os.curdir)
TEMP_AUDIO_DIR = os.path.join("/" + ROOT_DIR.strip('/backend'), "temp_audio")
# Define a pydantic model to validate incoming requests
class TutorialRequest(BaseModel):
audio_file_path: str
image_file_path: str
class TutorialStateRequest(BaseModel):
state: bool
class CurrentStepRequest(BaseModel):
step: int
@router.on_event("startup")
async def startup_event():
await init_db()
await init_rag() # This should now work correctly
await init_name_address()
# Global variables to store the tutorial state and current step
tutorial_active = False
current_step = 0
@router.get("/tutorial_state")
async def get_tutorial_state():
return {"tutorial_active": tutorial_active}
@router.post("/tutorial_state")
async def set_tutorial_state(request: TutorialStateRequest):
global tutorial_active
tutorial_active = request.state
return {"tutorial_active": tutorial_active}
@router.get("/current_step")
async def get_current_step():
return {"current_step": current_step}
@router.post("/current_step")
async def set_current_step(request: CurrentStepRequest):
global current_step
current_step = request.step
return {"current_step": current_step}
# Global variables to store the tutorial state and current step
tutorial_active = False
current_step = 0
@router.get("/tutorial_state")
async def get_tutorial_state():
return {"tutorial_active": tutorial_active}
@router.post("/tutorial_state")
async def set_tutorial_state(request: TutorialStateRequest):
global tutorial_active
tutorial_active = request.state
return {"tutorial_active": tutorial_active}
@router.get("/current_step")
async def get_current_step():
return {"current_step": current_step}
@router.post("/current_step")
async def set_current_step(request: CurrentStepRequest):
global current_step
current_step = request.step
return {"current_step": current_step}
@router.post("/tutorial")
async def tutorial(request: TutorialRequest):
global tutorial_active, current_step
audio_file_path = request.audio_file_path
image_file_path = request.image_file_path
logger.info(f"tutorial_active: {tutorial_active}, current_step: {current_step}")
logger.info(f"Received audio file path: {audio_file_path}, image file path: {image_file_path}")
try:
if tutorial_active == True:
# Check if files exist
if not os.path.exists(audio_file_path) or not os.path.exists(image_file_path):
raise HTTPException(status_code=400, detail="One or both of the specified files do not exist")
# Transcribe the audio
transcript = transcribe_audio(audio_file_path)
logger.info(f"Audio transcription successful. Transcript: {transcript[:100]}...")
# Generate the tutorial
steps = generate_tutorial(transcript, image_file_path)
logger.info(f"Tutorial generation successful. Number of steps: {len(steps)}")
# Generate and store audio for all steps asynchronously
async def store_all_audio():
for i, step in enumerate(steps[1:], start=1):
audio_file = generate_speech(step, i)
await store_audio(i, audio_file)
logger.info(f"Generated and stored audio for step {i}")
tutorial_active = False
current_step = 0
# Return the first step's audio
first_step_audio = generate_speech(steps[0], 0)
asyncio.create_task(store_all_audio())
return {
"step_number": 0,
"audio_file": first_step_audio,
"is_last_step": len(steps) == 1
}
else:
# This is a subsequent call, get the next step
next_step = current_step + 1
total_steps = await get_total_steps()
logger.info(f"Current step: {current_step}, Next step: {next_step}, Total steps: {total_steps}")
if next_step <= total_steps:
audio_file = await get_audio(next_step)
current_step = next_step
if next_step == total_steps:
tutorial_active = False
current_step = 0
await clear_audio()
return {
"step_number": next_step,
"audio_file": audio_file,
"is_last_step": next_step == total_steps
}
else:
tutorial_active = False
current_step = 0
await clear_audio()
except Exception as e:
logger.error(f"An error occurred: {str(e)}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e))
@router.post("/initiate-transaction")
async def initiate_transaction(audio_file_path: str):
logger.info("Received transaction request")
try:
# Step 1: Transcribe the audio
logger.debug(f"Transcribing audio from file: {audio_file_path}")
transcript = transcribe_audio(audio_file_path)
logger.info(f"Transcription successful: {transcript[:100]}...") # Log first 100 chars
# Step 2: Parse the transcript for task details (recipient, amount)
logger.debug("Parsing transcript for transaction details")
transaction_details = parse_transaction(transcript)
#transaction_details = json.loads(transaction_details)
logger.info(f"Transaction details parsed: {transaction_details}")
# Extract recipient and amount
#recipient_name
[truncated — 1112 more characters]
```
### frontend/br-nlp-web_app/tailwind.config.js
```javascript
module.exports = {
purge: ["./src/**/*.{js,jsx,ts,tsx}", "./public/index.html"],
darkMode: false, // or 'media' or 'class'
theme: {
extend: {
colors: {
'discord-gray': '#2c2f33', // Base color
'discord-purple': '#5865F2', // Highlight purple
'discord-blue': '#7289da', // Highlight blue
},
},
},
variants: {
extend: {},
},
plugins: [],
};
```
### backend/services/test_db_connection.py
```python
# test_db_connection.py
import pymysql
import os
from dotenv import load_dotenv
import traceback
# Load environment variables
load_dotenv()
host = os.getenv('SINGLESTORE_HOST')
user = os.getenv('SINGLESTORE_USER')
password = os.getenv('SINGLESTORE_PASSWORD')
database = os.getenv('SINGLESTORE_DB')
port = os.getenv('SINGLESTORE_PORT')
print("Testing database connection with the following parameters:")
print(f"Host: {host}")
print(f"Port: {port}")
print(f"User: {user}")
print(f"Database: {database}")
def get_ssl_cert_path():
project_root = os.path.dirname(os.path.abspath(__file__))
cert_path = os.path.join(project_root, 'certs', 'singlestore_bundle.pem')
print(f"SSL cert path: {cert_path}")
if not os.path.isfile(cert_path):
print("ERROR: SSL certificate file not found at the specified path.")
return cert_path
try:
connection = pymysql.connect(
host=host,
user=user,
password=password,
database=database,
port=int(port),
ssl={
'ca': get_ssl_cert_path()
}
)
print("Database connection successful!")
connection.close()
except Exception as e:
print(f"Database connection failed: {e}")
print("Exception type:", type(e))
print("Exception args:", e.args)
traceback.print_exc()
```
[28 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]