# Project export: Bloom AI
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: Cal Hacks 11.0
- Tagline: Bloom AI is a smart plant care system using IoT sensors and AI to monitor temperature, humidity, soil moisture, and light levels in real-time. With data visualization and AI-powered insights
- Devpost: https://devpost.com/software/bloom-ai
- GitHub: https://github.com/knhn1004/bloom-ai
- Demo: https://deanza.my.canva.site/bloom-ai
- Video: https://www.youtube.com/embed/fk4uUVfGRvg?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 3 GitHub contributor(s) — knhn1004 (21 commits), Enkhbold Ganbold (18 commits), joshuakojko (11 commits)
## Devpost submission (written by the team)
### Inspiration
I lived in a yurt in Mongolia, where planting indoors during -40°C winters was nearly impossible. With Bloom AI, we're solving this by using IoT and AI to monitor plant health, helping families like mine grow plants even in harsh conditions. Have you ever struggled to understand what your plants need to thrive? Whether you're a seasoned gardener or a newbie, ensuring your plants get optimal care can be challenging. We created Bloom AI to eliminate the guesswork in plant care by providing real-time insights using cutting-edge technology. What It Does Bloom AI is a smart plant health monitoring system that allows you to: Monitor plant health in real-time with IoT sensors tracking temperature, humidity, soil moisture, and light levels. Receive AI-powered advice on plant care, answering questions about watering schedules, light requirements, and more. Visualize plant health data in 3D, giving you an interactive understanding of your plant's status. Analyze plant images for health assessments, offering actionable insights on issues like under-watering or poor lighting. With Bloom AI, plant care becomes easier, more efficient, and data-driven. How We Built It Frontend: Developed with Next.js, React, and TypeScript for a fast, responsive user interface. Backend: Utilized Node.js and Flask. Flask handles AI integrations and analysis, while Node.js manages data exchange between sensors and the frontend. IoT Integration: Connected sensors to Arduino and ESP8266/ESP32 boards to collect real-time environmental data. Data Visualization: Employed Recharts to visualize trends in plant health data. 3D Rendering: Used Three.js to create interactive 3D models of plant health status. AI-Powered Insights: Integrated APIs like Groq for language processing and Hume for computer vision to provide personalized plant care advice. Challenges We Ran Into Hardware Integration: Ensuring accurate data collection required precise calibration and testing of IoT sensors. AI Image Analysis: Implementing reliable image recognition across different plant types was complex. 3D Visualization: Optimizing performance and usability for interactive 3D models posed technical challenges. Accomplishments That We're Proud Of Real-Time Monitoring: Successfully built a system that provides instant feedback on plant health. AI Assistance: Developed an AI assistant that makes plant care advice accessible to all users. Interactive Visualizations: Created 3D models that simplify complex data into an understandable format. Seamless Integration: Effectively combined multiple IoT sensors into a user-friendly dashboard. What We Learned IoT Development: Gained experience in setting up and integrating sensors with web applications. AI and Machine Learning: Explored advanced algorithms for image analysis and natural language processing. Full-Stack Development: Enhanced our skills in building applications from the ground up, managing both frontend and backend components. What's Next for Bloom AI Expanded Sensor Support: Adding sensors for air quality, pH levels, and more environmental factors. Community Platform: Creating features for users to share data and gardening tips. Mobile Application: Developing a mobile app for easier access and monitoring on the go. Enhanced AI Models: Improving accuracy and personalization of insights based on specific plant types and conditions.
## README (from the GitHub repository)
# Bloom AI: Smart Plant Monitoring System
Bloom AI is an innovative plant health monitoring system that combines IoT sensors, AI, and data visualization to help you take better care of your plants.
## Features
- **Real-time Plant Monitoring**: Track temperature, humidity, soil moisture, and light levels.
- **AI-powered Plant Assistant**: Get answers to your plant care questions.
- **3D Plant Visualization**: View your plant's health status in an interactive 3D model.
- **Data Analytics**: Visualize trends and patterns in your plant's health data.
- **Image Analysis**: Upload plant images for AI-powered health assessment.
## Technology Stack
- Frontend: Next.js, React, TypeScript
- Backend: Node.js, FastAPI, ThingSpeak
- IoT: Arduino, ESP8266/ESP32
- APIs: ThingSpeak, Groq, Deepgram
- Data Visualization: Recharts
- 3D Rendering: Three.js
## Getting Started
### Prerequisites
- Node.js (v14 or later)
- Arduino IDE
- ThingSpeak account
- Groq API key
- Hume API key
### Installation
1. Clone the repository:
```
git clone https://github.com/your-username/bloom-ai.git
```
2. Install dependencies:
```
cd bloom-ai
npm install
```
3. Set up environment variables:
Create a `.env` file in the root directory and add the following:
```
THINGSPEAK_API_KEY=your_thingspeak_api_key
GROQ_API_KEY=your_groq_api_key
HUME_API_KEY=your_hume_api_key
```
4. Run the development server:
```
npm run dev
```
5. Open [http://localhost:3000](http://localhost:3000) in your browser.
### Hardware Setup
1. Connect the sensors to your Arduino or ESP8266/ESP32 board according to the wiring diagram.
2. Upload the `main_code.ino` sketch to your board.
3. Configure your Wi-Fi credentials in the `secrets.h` file.
## Usage
1. Place the sensor near your plant.
2. Access the Bloom AI dashboard through your web browser.
3. Monitor your plant's health metrics in real-time.
4. Ask the AI assistant for plant care advice.
5. Upload images of your plant for detailed analysis.
## Contributing
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for more details.
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Acknowledgments
- ThingSpeak for IoT data platform
- Groq for AI language model and computer vision analysis
- Deepgram for Voice AI
## Detected evidence (automated analysis)
Indexed codebase: 95 recognized source files, 550 KB.
- C (language) — detected in the code
- C++ (language) — detected in the code
- CSS (language) — detected in the code
- FastAPI (technology) — detected in the code
- Firebase (technology) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- Flask (technology) — claimed on Devpost, not found in the code
- Node.js (technology) — claimed on Devpost, not found in the code
## Codebase structure (from repository index)
### Files (120 of 190)
```
.env.local.example
.eslintrc.json
.gitignore
app/actions/deepgram-chat.ts
app/actions/groq-chat.ts
app/actions/groq-image.ts
app/actions/hume.ts
app/firebase.config.ts
app/globals.css
app/layout.tsx
app/lib/data.ts
app/lib/interfaces.ts
app/page.tsx
components.json
components/ui/button.tsx
components/ui/card.tsx
components/ui/chart.tsx
components/ui/scroll-area.tsx
components/ui/tabs.tsx
cron/cron.py
hardware_src/DHT/DHT.ino
hardware_src/libraries/Adafruit_Unified_Sensor/.gitignore
hardware_src/libraries/Adafruit_Unified_Sensor/Adafruit_Sensor.cpp
hardware_src/libraries/Adafruit_Unified_Sensor/Adafruit_Sensor.h
hardware_src/libraries/Adafruit_Unified_Sensor/examples/sensortest/sensortest.ino
hardware_src/libraries/Adafruit_Unified_Sensor/library.properties
hardware_src/libraries/Adafruit_Unified_Sensor/LICENSE.txt
hardware_src/libraries/Adafruit_Unified_Sensor/README.md
hardware_src/libraries/DHT_sensor_library/code-of-conduct.md
hardware_src/libraries/DHT_sensor_library/CONTRIBUTING.md
hardware_src/libraries/DHT_sensor_library/DHT_U.cpp
hardware_src/libraries/DHT_sensor_library/DHT_U.h
hardware_src/libraries/DHT_sensor_library/DHT.cpp
hardware_src/libraries/DHT_sensor_library/DHT.h
hardware_src/libraries/DHT_sensor_library/examples/DHT_Unified_Sensor/DHT_Unified_Sensor.ino
hardware_src/libraries/DHT_sensor_library/examples/DHTtester/DHTtester.ino
hardware_src/libraries/DHT_sensor_library/keywords.txt
hardware_src/libraries/DHT_sensor_library/library.properties
hardware_src/libraries/DHT_sensor_library/license.txt
hardware_src/libraries/DHT_sensor_library/README.md
hardware_src/libraries/ThingSpeak/examples/ArduinoEthernet/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoEthernet/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoEthernet/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoEthernet/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoEthernet/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoEthernet/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/ReadFieldSecure/ReadFieldSecure.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/ReadFieldSecure/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/ReadMultipleFields/ReadMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/ReadMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/ReadMultipleFieldsSecure/ReadMultipleFieldsSecure.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/ReadMultipleFieldsSecure/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/WriteMultipleFieldsSecure/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/WriteMultipleFieldsSecure/WriteMultipleFieldsSecure.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/WriteSingleFieldSecure/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKR1000/WriteSingleFieldSecure/WriteSingleFieldSecure.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRETHShield/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRETHShield/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRETHShield/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRETHShield/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRETHShield/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRETHShield/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRGSM1400/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRGSM1400/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRGSM1400/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRGSM1400/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRGSM1400/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRGSM1400/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRVIDOR4000/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRVIDOR4000/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRVIDOR4000/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRVIDOR4000/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRVIDOR4000/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRVIDOR4000/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRWiFi1010/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRWiFi1010/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRWiFi1010/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRWiFi1010/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRWiFi1010/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoMKRWiFi1010/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoUnoWiFi Rev2/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoUnoWiFi Rev2/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoUnoWiFi Rev2/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoUnoWiFi Rev2/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoUnoWiFi Rev2/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoUnoWiFi Rev2/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield101/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield101/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield101/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield101/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield101/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoWiFiShield101/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoYun/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoYun/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoYun/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoYun/WriteMultipleFields/WriteMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ArduinoYun/WriteSingleField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ArduinoYun/WriteSingleField/WriteSingleField.ino
hardware_src/libraries/ThingSpeak/examples/ESP32/ReadField/ReadField.ino
hardware_src/libraries/ThingSpeak/examples/ESP32/ReadField/secrets.h
hardware_src/libraries/ThingSpeak/examples/ESP32/ReadFieldSecure/ReadFieldSecure.ino
hardware_src/libraries/ThingSpeak/examples/ESP32/ReadFieldSecure/secrets.h
hardware_src/libraries/ThingSpeak/examples/ESP32/ReadMultipleFields/ReadMultipleFields.ino
hardware_src/libraries/ThingSpeak/examples/ESP32/ReadMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ESP32/ReadMultipleFieldsSecure/ReadMultipleFieldsSecure.ino
hardware_src/libraries/ThingSpeak/examples/ESP32/ReadMultipleFieldsSecure/secrets.h
hardware_src/libraries/ThingSpeak/examples/ESP32/WriteMultipleFields/secrets.h
hardware_src/libraries/ThingSpeak/examples/ESP32/WriteMultipleFields/WriteMultipleFields.ino
[70 more files omitted for size]
```
### Dependencies
- hardwareBackend/requirements.txt: pyserial@==3.5, python-dotenv@==1.0.1, requests@==2.32.3
- package.json: @deepgram/sdk@^3.8.1, @radix-ui/react-icons@^1.3.0, @radix-ui/react-scroll-area@^1.2.0, @radix-ui/react-slot@^1.1.0, @radix-ui/react-tabs@^1.1.1, @react-three/drei@^9.114.6, @react-three/fiber@^8.17.10, @types/node@^20, @types/react@^18, @types/react-dom@^18, class-variance-authority@^0.7.0, cloudinary@^2.5.1, clsx@^2.1.1, eslint@^8, eslint-config-next@14.2.15, firebase@^10.14.1, groq-sdk@^0.7.0, hume@^0.9.1, lucide-react@^0.453.0, next@14.2.15, next-pwa@^5.6.0, postcss@^8, react@^18, react-dom@^18, react-markdown@^9.0.1, recharts@^2.13.0, tailwind-merge@^2.5.4, tailwindcss@^3.4.1, tailwindcss-animate@^1.0.7, typescript@^5
- voice-agent/requirements.txt: fastapi, firebase_admin, janus@==1.0.0, PyAudio@==0.2.14, pyngrok, python-dotenv, uvicorn, websockets@==12.0
### Recent commits (newest first)
- Update README.md
- Update tool metric
- fix: fixed bug in plant image analysis process
- Update README.md
- Update env.local.example
- Plant image analysis
- Update README.md
- Update README.md
- Update project README and add Bloom AI logo and features
- Update theme color in layout.tsx
- Add next-pwa package to package.json
- Mobile view update
- Update favicon and font in layout.tsx
- Update metrics
- update prompt
- voice agent prompt
- plant image analysis with groq
- 3d model
- cron, glb
- functions get current iot data
## Key source files (fetched from GitHub, selected and truncated for size)
### hardware_src/libraries/DHT_sensor_library/CONTRIBUTING.md
```markdown
# Contribution Guidlines
This library is the culmination of the expertise of many members of the open source community who have dedicated their time and hard work. The best way to ask for help or propose a new idea is to [create a new issue](https://github.com/adafruit/DHT-sensor-library/issues/new) while creating a Pull Request with your code changes allows you to share your own innovations with the rest of the community.
The following are some guidelines to observe when creating issues or PRs:
- Be friendly; it is important that we can all enjoy a safe space as we are all working on the same project and it is okay for people to have different ideas
- [Use code blocks](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet#code); it helps us help you when we can read your code! On that note also refrain from pasting more than 30 lines of code in a post, instead [create a gist](https://gist.github.com/) if you need to share large snippets
- Use reasonable titles; refrain from using overly long or capitalized titles as they are usually annoying and do little to encourage others to help :smile:
- Be detailed; refrain from mentioning code problems without sharing your source code and always give information regarding your board and version of the library
```
### hardware_src/libraries/ThingSpeak/license.md
```markdown
Copyright :copyright: 2017, The MathWorks, Inc.
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
4. In all cases, the software is, and all modifications and derivatives of the software shall be, licensed to you solely for use in conjunction with MathWorks products and service offerings.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
```
### package.json
```
{
"name": "bloom-ai",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"dependencies": {
"@deepgram/sdk": "^3.8.1",
"@radix-ui/react-icons": "^1.3.0",
"@radix-ui/react-scroll-area": "^1.2.0",
"@radix-ui/react-slot": "^1.1.0",
"@radix-ui/react-tabs": "^1.1.1",
"@react-three/drei": "^9.114.6",
"@react-three/fiber": "^8.17.10",
"class-variance-authority": "^0.7.0",
"cloudinary": "^2.5.1",
"clsx": "^2.1.1",
"firebase": "^10.14.1",
"groq-sdk": "^0.7.0",
"hume": "^0.9.1",
"lucide-react": "^0.453.0",
"next": "14.2.15",
"next-pwa": "^5.6.0",
"react": "^18",
"react-dom": "^18",
"react-markdown": "^9.0.1",
"recharts": "^2.13.0",
"tailwind-merge": "^2.5.4",
"tailwindcss-animate": "^1.0.7"
},
"devDependencies": {
"@types/node": "^20",
"@types/react": "^18",
"@types/react-dom": "^18",
"eslint": "^8",
"eslint-config-next": "14.2.15",
"postcss": "^8",
"tailwindcss": "^3.4.1",
"typescript": "^5"
}
}
```
### hardwareBackend/requirements.txt
```
pyserial==3.5
requests==2.32.3
python-dotenv==1.0.1
```
### voice-agent/requirements.txt
```
PyAudio==0.2.14
websockets==12.0
janus==1.0.0
python-dotenv
fastapi
uvicorn
firebase_admin
pyngrok
```
### app/layout.tsx
```typescript
import type { Metadata, Viewport } from 'next';
// import localFont from "next/font/local";
import './globals.css';
import { Roboto } from 'next/font/google';
export const viewport: Viewport = {
themeColor: '#FFFFFF',
};
// const geistSans = localFont({
// src: "./fonts/GeistVF.woff",
// variable: "--font-geist-sans",
// weight: "100 900",
// });
// const geistMono = localFont({
// src: "./fonts/GeistMonoVF.woff",
// variable: "--font-geist-mono",
// weight: "100 900",
// });
const roboto = Roboto({
subsets: ['latin'],
weight: ['300', '900'],
});
export const metadata: Metadata = {
icons: {
icon: '🪴',
apple: '🪴',
},
title: 'Bloom.ai',
description: 'Bloom AI is a platform for plant health monitoring.',
};
export default function RootLayout({
children,
}: {
children: React.ReactNode;
}) {
return (
{children}
);
}
```
### hardwareBackend/main.py
```python
import serial
import time
import requests
import os
from dotenv import load_dotenv
# Load environment variables from .env file
load_dotenv()
# Replace with your ThingSpeak API key from environment variable
THINGSPEAK_API_KEY = os.getenv("THINGSPEAK_API_KEY")
rateLimit = 30
# Configure serial port (update port if necessary)
serial_port = "/dev/ttyACM0" # Linux
# serial_port = "COM10" # Windows
baud_rate = 9600
# Open the serial connection
ser = serial.Serial(serial_port, baud_rate, timeout=1)
time.sleep(2) # Give some time for the connection to establish
def parse_sensor_data(data):
"""
Parses the serial data in the format:
Humidity: 45.00%, Temp: 24.00°C / 75.20°F, Heat index: 25.00°C / 77.00°F, Light: 345, Soil Moisture: 689
"""
try:
data = data.strip()
# Extract values from the string
humidity = float(data.split("Humidity: ")[1].split("%")[0])
temp_c = float(data.split("Temp: ")[1].split("°C")[0])
temp_f = float(data.split("/ ")[1].split("°F")[0])
light = int(data.split("Light: ")[1].split(",")[0])
soil_moisture = int(data.split("Soil Moisture: ")[1])
return humidity, temp_c, temp_f, light, soil_moisture
except Exception as e:
print(f"Error parsing sensor data: {e}")
return None, None, None, None, None
def update_thingspeak(humidity, temp_c, temp_f, light, soil_moisture):
"""
Send data to ThingSpeak via HTTP GET request.
"""
try:
# Construct the ThingSpeak API URL
url = f"https://api.thingspeak.com/update?api_key={THINGSPEAK_API_KEY}&field1={humidity}&field2={temp_c}&field3={temp_f}&field4={light}&field5={soil_moisture}"
response = requests.get(url)
if response.status_code == 200:
print(f"Data successfully sent to ThingSpeak: {response.text}")
else:
print(f"Failed to send data to ThingSpeak: {response.status_code}")
except Exception as e:
print(f"Error sending data to ThingSpeak: {e}")
# Continuously read from serial and send data to ThingSpeak
while True:
if ser.in_waiting > 0:
# Read the serial data
serial_data = ser.readline().decode("utf-8")
print(f"Received: {serial_data}")
# Parse the sensor data
humidity, temp_c, temp_f, light, soil_moisture = parse_sensor_data(serial_data)
# If data was parsed successfully, send it to ThingSpeak
if humidity is not None:
print(f"Parsed Data: Humidity={humidity}, TempC={temp_c}, TempF={temp_f}, Light={light}, SoilMoisture={soil_moisture}")
update_thingspeak(humidity, temp_c, temp_f, light, soil_moisture)
# Add a small delay before the next reading
time.sleep(rateLimit) # Update every 15 seconds as per ThingSpeak's rate limit
```
### app/page.tsx
```typescript
'use client';
import React, { useState, useEffect, useRef } from 'react';
import { Button } from '@/components/ui/button';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { ScrollArea } from '@/components/ui/scroll-area';
import { LineChart, Line, XAxis, YAxis, CartesianGrid, Legend } from 'recharts';
import { Canvas } from '@react-three/fiber';
import { OrbitControls, useGLTF, Environment } from '@react-three/drei';
import Markdown from 'react-markdown';
import {
ChartContainer,
ChartTooltip,
ChartTooltipContent,
} from '@/components/ui/chart';
import {
Thermometer,
Droplet,
Sun,
Sprout,
PhoneCall,
PhoneOff,
} from 'lucide-react';
import { getSentimentInfo, setupPlantMetricsListener } from './lib/data';
import type { Message, PlantMetrics } from './lib/interfaces';
import { toggleDeepgramConnection } from './actions/deepgram-chat';
import { db } from './firebase.config';
import { onSnapshot, collection, orderBy, query } from 'firebase/firestore';
import { getImageDescription } from './actions/groq-image';
import Image from 'next/image';
export default function Dashboard() {
const [messages, setMessages] = useState([]);
const [isListening, setIsListening] = useState(false);
const [chatId, setChatId] = useState('');
const [plantMetrics, setPlantMetrics] = useState([]);
const scrollAreaRef = useRef(null);
const messagesEndRef = useRef(null);
const [imageDescription, setImageDescription] = useState<{
imageUrl: string;
description: string;
} | null>(null);
const [isAnalyzing, setIsAnalyzing] = useState(false);
useEffect(() => {
if (chatId) {
const messagesRef = collection(db, 'chats', chatId, 'messages');
const q = query(messagesRef, orderBy('timestamp', 'asc'));
const unsubscribe = onSnapshot(q, snapshot => {
const newMessages = snapshot.docs.map(doc => ({
id: doc.id,
...(doc.data() as Omit),
})) as Message[];
setMessages(newMessages);
});
return () => unsubscribe();
}
}, [chatId]);
useEffect(() => {
// Scroll to the bottom of the chat area when messages change
if (scrollAreaRef.current) {
const scrollContainer = scrollAreaRef.current.querySelector(
'[data-radix-scroll-area-viewport]'
);
if (scrollContainer) {
scrollContainer.scrollTop = scrollContainer.scrollHeight;
}
}
}, [messages]);
useEffect(() => {
const unsubscribe = setupPlantMetricsListener(data => {
setPlantMetrics(data);
});
return () => {
if (unsubscribe) {
unsubscribe();
}
};
}, []);
const toggleConnection = async () => {
if (isListening) {
await toggleDeepgramConnection('stop', chatId);
} else {
const chatId = Math.random().toString(36).substring(2, 15);
setChatId(chatId);
await toggleDeepgramConnection('start', chatId);
}
setIsListening(prev => !prev);
};
//const handleUserInput = async (input: string) => {
// setMessages(prev => [...prev, { role: 'user', content: input }]);
// const aiResponse = await processUserInput(input);
// setMessages(prev => [...prev, { role: 'ai', ...aiResponse } as Message]);
//};
const handleGetImageDescription = async () => {
setIsAnalyzing(true);
const result = await getImageDescription();
setImageDescription(result);
setIsAnalyzing(false);
};
function PlantModel() {
const { scene } = useGLTF('/models/plant.glb');
return (
<>
);
}
return (