# Project export: NoWaste.ai

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2024
- Tagline: Providing a practical and scalable solution to every-day food waste.
- Devpost: https://devpost.com/software/nowaste-ai
- GitHub: https://github.com/RussellBustamante/NoWasteAI
- Demo: https://www.canva.com/design/DAF9JEwxwt4/CpUcKgNiGDjDeKiWDRBMqg/view
- Video: https://www.youtube.com/embed/b5w4uaHembk?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (Postman: Best Use of an API with Postman (4x Flipper0 + Postman Merch))
- Team: 0 GitHub contributor(s) — 

## Devpost submission (written by the team)

### Overview

Our

### Inspiration

Have you ever looked inside the fridge, saw a particular piece of food, and wondered if it was still edible? Maybe you felt it, or looked it up online, or even gave it the old sniff test! Yet, at the end of the day, you still threw it away. Did you know that US citizens waste an average of 200lbs of food per year, per person? Much of that waste can be traced back to food casually tossed out of the fridge without a second thought. Additionally, 80% of Americans rely on the printed ‘best by’ dates to determine if their food is still fresh, when in reality it is almost never a true indicator of expiration. We aim to address these problems and more. What do we do? NoWaste.ai is a two-pronged approach to the same issue, allowing us to extend our support to users all over the world. At its core, NoWaste allows users to snap photos of an edible product, describe its context, and harness the power of generative AI to receive a gauge on the quality of their food. An example could be the following: "Pizza left on a warm counter for 6 hours, then refrigerated for 8" + [Picture of the pizza], which would return the AI's score of the food's safety. In developed countries, everyone has a smartphone, which is why we provide a mobile application to make this process as convenient as possible. In order to create a product that users will be more likely to consistently use, the app has been designed to be extremely streamlined, rapidly responsive, and very lightweight. However, in other parts of the world, this luxury is not so common. Hence, we have researched extensively to design and provide a cheap and scalable solution for communities around the world via Raspberry Pi Cameras. At just $100 per assembly and $200 per deployment, these individual servers have potential to be dispersed all across the world, saving thousands of lives, millions of pounds of waste, and billions of dollars.

### How we built it

Our hardware stack is hosted completely on the Raspberry Pi in a React/Flask environment. We utilize cloud AI models, such as LLAMA 70B on Together.ai and GPT4 Vision, to offload computation and make our solution as cheap and scalable as possible. Our software stack was built using Swift and communicates with similar APIs. We began by brainstorming the potential services and technologies we could use to create lightweight applications as quickly as possible. Once we settled on a general idea, we split off and began implementing our own responsibilities: while some of us prototyped the frontend, others were experimenting with AI models or doing market research. This delegation of responsibility allowed us to work in parallel and design a comprehensive solution for a problem as large (yet seemingly so clear) as this. Of course, our initial ideas were far from what we eventually settled on.

### Challenges we ran into

Finalizing and completing our stack was one of our greatest challenges. Our frontend technologies changed the most over the course of the weekend as we experimented with Django, Reflex, React, and Flask, not to mention the different APIs and LLM hosts that we researched. Additionally, we were ambitious in wanting to only use open-source solutions to further drive home the idea of world-wide collaboration for sustainability and the greater good, but we failed to identify a solution for our Vision model. We attempted to train LLAVA using Intel cloud machines, but our lack of time made it difficult as beginners. Our team also faced hardware issues, from broken cameras to faulty particle sensors. However, we were successful in remedying each of these issues in their own unique ways, and we are happy to present a product within the timeframe we had.

### Accomplishments we're proud of

We are incredibly proud of what we were able to accomplish in such a short amount of time. We were passionate about both the underlying practicalities of our application as well as the core implementation. We created not only a webapp hosted on a Raspberry Pi, equipped with odor sensors and a camera, but a mobile app prototype and a mini business plan as well. We were able to target multiple audiences and clear areas where this issue prevails, and we have proposed solutions that suit all of them.

### What's next

The technology that we propose is infinitely extensible. Beyond this weekend at TreeHacks, there is room for fine-tuning or training more models to produce more accurate results on rotting and spoiled food. Dedicated chips and board designs can bring the cost of production and extension down, making it even easier to provide solutions across the world. Augmented Reality is becoming more prevalent every day, and there is a clear spot for NoWaste to streamline our technology to work seamlessly with humans. The possibilities are endless, and we hope you can join and support us on our journey!

## README (from the GitHub repository)

# NoWaste.ai - Treehacks 2024
Sponsor Award Winner for Best Use of an API with Postman


# Hardware:
Raspberry Pi
Gas sensor (MQ-X)
8-megapixel Camera Module 2

# Software & Technologies:
React
Flask
Swift
PostMan
Tailwind
OpenAI API (GPT 4 Vision)
Together.Ai (LLaMA70B)

# Slide Deck
https://www.canva.com/design/DAF9JEwxwt4/CpUcKgNiGDjDeKiWDRBMqg/view


## Detected evidence (automated analysis)

Indexed codebase: 14 recognized source files, 15 KB.
- CSS (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Swift (language) — detected in the code
- Tailwind CSS (technology) — detected in the code
- Flask (technology) — claimed on Devpost, not found in the code
- OpenAI (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (18 of 18)

```
Contents.json
fitcurve.m
javascript.js
package.json
README
src/App.css
src/App.js
src/app.py
src/App.test.js
src/cameraScript.py
src/gasScript.py
src/index.css
src/index.js
src/python.py
src/python2.py
src/reportWebVitals.js
src/setupTests.js
WasteLessApp.swift
```

### Dependencies

- package.json: @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, autoprefixer@^10.4.17, child_process@^1.0.2, postcss@^8.4.35, react@^18.2.0, react-dom@^18.2.0, react-scripts@5.0.1, tailwindcss@^3.4.1, web-vitals@^2.1.4

### Recent commits (newest first)

- Update README
- Cleanup Commit

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "th-project",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@testing-library/jest-dom": "^5.17.0",
    "@testing-library/react": "^13.4.0",
    "@testing-library/user-event": "^13.5.0",
    "child_process": "^1.0.2",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-scripts": "5.0.1",
    "web-vitals": "^2.1.4"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  },
  "devDependencies": {
    "autoprefixer": "^10.4.17",
    "postcss": "^8.4.35",
    "tailwindcss": "^3.4.1"
  }
}

```

### src/index.js

```javascript
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

```

### src/app.py

```python
from flask import Flask, jsonify, request
from flask_cors import CORS, cross_origin
from cameraScript import camera
import subprocess


app = Flask(__name__)
CORS(app)


@app.route('/run-script', methods=['POST'])
@cross_origin()
def run_script():
    data = request.json
    days = data['days']
    ppm = data['ppm']
    prompt1 = data['prompt1']
    print(days, ppm)
    result = subprocess.run(['python', 'python2.py', str(days), str(ppm), str(prompt1)], capture_output=True, text=True)
    output = result.stdout
    return jsonify({"output": output})

# image
@app.route('/run-script1', methods=['POST'])
@cross_origin()
def run_script1():
    result = subprocess.run(['python', 'python.py'], capture_output=True, text=True)
    output = result.stdout
    return jsonify({"output": output})

@app.route('/run-script2', methods=['POST'])
@cross_origin()
def run_script2():
    result = subprocess.run(['python', 'gasScript.py'], capture_output=True, text=True)
    output = result.stdout
    camera()
    return jsonify({"output": output})

if __name__ == "__main__":
    app.run(debug=True)

```

### src/App.js

```javascript
import React, { useState } from 'react';
import placeholder from './images/placeholder.png';
import foodimg from './food.png';


// Define the API key
const OPENAI_API_KEY = "API_KEY";

function App() {
  const [inputText, setInputText] = useState('');
  const [days, setDays] = useState(''); // New state for days input
  const [ppm, setPpm] = useState(''); // New state for ppm input
  const [image, setImage] = useState(placeholder); // Use a local placeholder image
  const [showAnalyzeButton, setShowAnalyzeButton] = useState(false);
  const [showFinalAnalysisButton, setShowFinalAnalysisButton] = useState(false);
  const [visionText, setVisionText] = useState(''); // Store vision API response for later use
  const [gptResponse, setGptResponse] = useState('');
  const [prompt1, setPrompt1] = useState('');
  const [scriptOutput, setScriptOutput] = useState('');

  const convertImageToBase64 = (imageUrl) => {
    return new Promise((resolve, reject) => {
      const img = new Image();
      img.crossOrigin = 'Anonymous';
      img.onload = () => {
        let canvas = document.createElement("canvas");
        canvas.width = img.width;
        canvas.height = img.height;
        let ctx = canvas.getContext("2d");
        ctx.drawImage(img, 0, 0);
        const dataURL = canvas.toDataURL("image/png");
        resolve(dataURL);
      };
      img.onerror = error => reject(error);
      img.src = imageUrl;
      if (img.complete || img.complete === undefined) {
        img.src = "data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///ywAAAAAAQABAAACAUwAOw==";
        img.src = imageUrl;
      }
    });
  };

  const handleCaptureImage = async () => {
    try {
        const response = await fetch('http://localhost:5000/run-script1', {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
            },
            body: JSON.stringify({
                days: days,
                ppm: ppm,
            })
        });

        if (!response.ok) {
            throw new Error('Network response was not ok');
        }

        const data = await response.json();
        console.log("Received data from Python script:", data);
        
        setPrompt1(data.output);
        setShowAnalyzeButton(true);
        setPpm(data.ppm);
    } catch (error) {
        console.error('Error:', error);
    }
};

const handleSendData = async () => {
  const requestData = {
    days: days,
    ppm: ppm,
    prompt1: prompt1
  }

  try {
    const response = await fetch('http://localhost:5000/run-script1', {
      method: 'POST',
      headers: {
          'Content-Type': 'application/json',
      },
      body: JSON.stringify(requestData),
    });

    if (!response.ok) {
        throw new Error('Network response was not ok');
    }

    const data = await response.json();
    console.log("Received data from Python script:", data);
    setScriptOutput(data.output); // Update state with the script output
  } catch (error) {
    console.error('Error:', error);
    setScriptOutput("Error occurred while fetching data."); // Handle error by updating state
  }
};

  const handleFinalAnalysis = async () => {
    try {
      const gptResponse = await fetch('https://api.openai.com/v1/completions', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
          'Authorization': `Bearer ${OPENAI_API_KEY}`,
        },
        body: JSON.stringify({
          model: "text-davinci-003",
          prompt: visionText,
          max_tokens: 100,
        }),
      });

      if (!gptResponse.ok) {
        throw new Error('Network response was not ok from GPT API');
      }

      const gptData = await gptResponse.json();
      setGptResponse(gptData.choices[0].text);
      

    } catch (error) {
      console.error('Error:', error);
      setGptResponse("Failed to process the image or get a response.");
    }
  };


  return (
    <div className="min-h-screen bg-gray-100 flex flex-col items-center justify-center p-4">
      <div className="bg-white shadow-xl rounded-lg p-6 w-full max-w-4xl">
        <div className="grid grid-cols-1 md:grid-cols-2 gap-6">
          <div className="flex flex-col space-y-4">
            <textarea
              className="textarea textarea-bordered h-24"
              placeholder="Enter text here..."
              value={inputText}
              onChange={(e) => setInputText(e.target.value)}
            ></textarea>
            <input
              type="text"
              className="input input-bordered"
              placeholder="Enter days..."
              value={days}
              onChange={(e) => setDays(e.target.value)}
            />
            <button
              className="btn btn-primary"
              onClick={handleCaptureImage}
            >
              Capture Image
            </button>
            {showAnalyzeButton && (
              <button
                className="btn btn-secondary mt-2"
                onClick={handleSendData}
              >
                Analyze Image
              </button>
            )}
            {showFinalAnalysisButton && (
              <button
                className="btn btn-success mt-2"
                onClick={handleFinalAnalysis}
              >
                Finalize Analysis
              </button>
            )}
          </div>
          <span id='here'>

          </span>
          <div className="flex flex-col items-center space-y-4">
          
            <textarea
              className="textarea textarea-bordered h-24 w-full"
              placeholder="API Response..."
              value={gptResponse}
              readOnly
            ></textarea>
            <img src={foodimg} alt="Dynamic" />
          </div>
          <div className="text-sm mt-4">{scriptOutput}</div>
        </div>
      </div>
    </div>
  );
}

export default App;

```

### WasteLessApp.swift

```swift
//
//  WasteLessApp.swift
//  WasteLess
//
//  Created by Joyce Zhou on 2/18/24.
//

import SwiftUI

@main
struct WasteLessApp: App {
    var body: some Scene {
        WindowGroup {
            ContentView()
        }
    }
}

```

### javascript.js

```javascript
// NOTE
// response_2 = the response of the user input
const { spawn } = require('child_process');

const days = 100;
const ppm = 10000;
var response_2 = '';
const data_to_pass_in_2 = JSON.stringify({ days, ppm });

console.log('Data sent to python script:', data_to_pass_in_2);
const python_process_2 = spawn('python', ['./python2.py', data_to_pass_in_2]);

python_process_2.stdout.on('data', (data) => {
    response_2 += data.toString();
});

python_process_2.on('close', (code) => {
    console.log('Python script finished with code:', code);
    console.log('Response from Python script:', response_2);
});




```

### src/setupTests.js

```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

```

### src/App.test.js

```javascript
import { render, screen } from '@testing-library/react';
import App from './App';

test('renders learn react link', () => {
  render(<App />);
  const linkElement = screen.getByText(/learn react/i);
  expect(linkElement).toBeInTheDocument();
});

```

### src/reportWebVitals.js

```javascript
const reportWebVitals = onPerfEntry => {
  if (onPerfEntry && onPerfEntry instanceof Function) {
    import('web-vitals').then(({ getCLS, getFID, getFCP, getLCP, getTTFB }) => {
      getCLS(onPerfEntry);
      getFID(onPerfEntry);
      getFCP(onPerfEntry);
      getLCP(onPerfEntry);
      getTTFB(onPerfEntry);
    });
  }
};

export default reportWebVitals;

```

### src/index.css

```css
@tailwind base;
@tailwind components;
@tailwind utilities;

body {
  margin: 0;
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
    'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
    sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

code {
  font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',
    monospace;
}

```

[5 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]