# Project export: Ohm-Snap

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: CruzHacks 2024
- Tagline: A website that analyzes images of resistors and returns their resistance in Ohms.
- Devpost: https://devpost.com/software/ohm-snap
- GitHub: https://github.com/bearcbass/Ohm_Snap
- Team: 2 GitHub contributor(s) — Tyler Morton (18 commits), Sebastian Osorio (9 commits)

## Devpost submission (written by the team)

### Inspiration

Resistors very small and difficult to read the color bands to determine their resistance. It is especially difficult with those who have difficulty seeing as well as those who are color blind.

### What it does

Upload a picture of a resistor, select the resistor, and get the resistance value.

### How we built it

We built the application with Next.JS as our frontend framework, Auth0 for authentication, Flask for our backend API, Segment Anything by Meta for object detection and masking, and OpenCV for image manipulation.

### Challenges we ran into

Color classification is a difficult problem, and one that might not have a straight-forward implementation to solve. The project was ambitious for a weekend, especially due to our limited knowledge of computer vision.

### Accomplishments we're proud of

We are proud to have a website that can mask an object a user selects.

### What's next

We hope to have a basic color classifier to eventually begin predictions for resistance values.

## README (from the GitHub repository)

# Ohm Snap
An application for determining resistance values of resistors.
Using an Meta's SAM model to crop out resistor from an image. Then running the images into another model to return resistor values. 


## Detected evidence (automated analysis)

Indexed codebase: 34 recognized source files, 55 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- Python (language) — detected in the code
- PyTorch (technology) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code

## Codebase structure (from repository index)

### Files (45 of 45)

```
.DS_Store
backend/.gitignore
backend/asdf.png.jpeg.npy
backend/client_request.py
backend/curl.sh
backend/model_install.sh
backend/mongo.py
backend/rand.npy
backend/README.md
backend/requirements.txt
backend/sam/create_model.py
backend/sam/onnx_model.py
backend/sam/util.py
backend/server.py
backend/test.txt
backend/vision/detect.py
backend/vision/edge_detection.py
backend/vision/mask_to_colors.py
backend/vision/mask_to_original.py
backend/vision/ohm_calculation.py
backend/vision/result_to_colors.py
backend/vision/test.py
frontend/.eslintrc.json
frontend/.gitignore
frontend/next.config.mjs
frontend/package.json
frontend/postcss.config.js
frontend/README.md
frontend/src/app/api/auth/[auth0]/route.js
frontend/src/app/components/helpers/scaleHelper.tsx
frontend/src/app/components/hooks/context.tsx
frontend/src/app/components/hooks/createContext.tsx
frontend/src/app/components/navbar.tsx
frontend/src/app/components/particles.tsx
frontend/src/app/components/stage.tsx
frontend/src/app/dashboard/dashboard.tsx
frontend/src/app/dashboard/mousepos.tsx
frontend/src/app/dashboard/page.tsx
frontend/src/app/globals.css
frontend/src/app/layout.tsx
frontend/src/app/login/page.tsx
frontend/src/app/page.tsx
frontend/tailwind.config.ts
frontend/tsconfig.json
README.md
```

### Dependencies

- backend/requirements.txt: asgiref@==3.7.2, blinker@==1.7.0, certifi@==2023.11.17, charset-normalizer@==3.3.2, click@==8.1.7, coloredlogs@==15.0.1, contourpy@==1.2.0, cycler@==0.12.1, dnspython@==2.4.2, ecdsa@==0.18.0, filelock@==3.13.1, Flask@==2.3.3, Flask-Cors@==4.0.0, Flask-PyMongo@==2.3.0, flatbuffers@==23.5.26, fonttools@==4.47.2, fsspec@==2023.12.2, humanfriendly@==10.0, idna@==3.6, itsdangerous@==2.1.2, Jinja2@==3.1.3, kiwisolver@==1.4.5, MarkupSafe@==2.1.4, matplotlib@==3.8.2, mpmath@==1.3.0, networkx@==3.2.1, numpy@==1.26.3, nvidia-cublas-cu12@==12.1.3.1, nvidia-cuda-cupti-cu12@==12.1.105, nvidia-cuda-nvrtc-cu12@==12.1.105, nvidia-cuda-runtime-cu12@==12.1.105, nvidia-cudnn-cu12@==8.9.2.26, nvidia-cufft-cu12@==11.0.2.54, nvidia-curand-cu12@==10.3.2.106, nvidia-cusolver-cu12@==11.4.5.107, nvidia-cusparse-cu12@==12.1.0.106, nvidia-nccl-cu12@==2.18.1, nvidia-nvjitlink-cu12@==12.3.101, nvidia-nvtx-cu12@==12.1.105, onnx@==1.15.0, onnxruntime@==1.16.3, opencv-python@==4.9.0.80, packaging@==23.2, pillow@==10.2.0, protobuf@==4.25.2, pyasn1@==0.5.1, pycocotools@==2.0.7, pymongo@==4.6.1, pyparsing@==3.1.1, python-dateutil@==2.8.2, python-dotenv@==1.0.0, python-jose@==3.3.0, requests@==2.31.0, rsa@==4.9, segment-anything@==1.0, six@==1.16.0, sqlparse@==0.4.4, sympy@==1.12, torch@==2.1.2, torch-vision@==0.1.6.dev0, torchvision@==0.16.2, triton@==2.1.0, typing_extensions@==4.9.0, urllib3@==2.1.0, Werkzeug@==3.0.1
- frontend/package.json: @auth0/nextjs-auth0@^3.5.0, @tailwindcss/nesting@^0.0.0-insiders.565cd3e, @tailwindcss/typography@^0.5.10, @types/node@^20, @types/react@^18, @types/react-dom@^18, autoprefixer@^10.0.1, eslint@^8, eslint-config-next@14.1.0, next@14.1.0, onnxruntime-web@^1.16.3, postcss@^8, react@^18.2.0, react-dom@^18, react-refresh@^0.14.0, tailwindcss@^3.3.0, tailwindcss-debug-screens@^2.2.1, typescript@^5, underscore@^1.13.6

### Recent commits (newest first)

- last minute
- frontend touches
- Major cleanup, restructure, and features
- Merge pull request #5 from bearcbass/computer-vision
- Merge pull request #4 from bearcbass/frontend
- working on full product
- Updated the file to add more values
- Got rid of other code thats redundant
- Added images and python files that give us calculations and values
- Merge pull request #3 from bearcbass/backend
- made somer server optimizations and changed endpoint around
- Got the mask to work
- Adding images and python files to help with edge detection, overlaying images, etc.
- sam module is taking shape
- backend everything
- Added cv2 image cropping and rotation
- Merge pull request #2 from bearcbass/frontend
- Merge pull request #1 from bearcbass/ai
- start of image upload api endpoint
- refactor of create model

## Key source files (fetched from GitHub, selected and truncated for size)

### frontend/package.json

```
{
  "name": "ohm-snap-frontend",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "@auth0/nextjs-auth0": "^3.5.0",
    "@tailwindcss/nesting": "^0.0.0-insiders.565cd3e",
    "@tailwindcss/typography": "^0.5.10",
    "next": "14.1.0",
    "onnxruntime-web": "^1.16.3",
    "react": "^18.2.0",
    "react-dom": "^18",
    "react-refresh": "^0.14.0",
    "tailwindcss-debug-screens": "^2.2.1",
    "underscore": "^1.13.6"
  },
  "devDependencies": {
    "@types/node": "^20",
    "@types/react": "^18",
    "@types/react-dom": "^18",
    "autoprefixer": "^10.0.1",
    "eslint": "^8",
    "eslint-config-next": "14.1.0",
    "postcss": "^8",
    "tailwindcss": "^3.3.0",
    "typescript": "^5"
  }
}

```

### backend/requirements.txt

```
asgiref==3.7.2
blinker==1.7.0
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
coloredlogs==15.0.1
contourpy==1.2.0
cycler==0.12.1
dnspython==2.4.2
ecdsa==0.18.0
filelock==3.13.1
Flask==2.3.3
Flask-Cors==4.0.0
Flask-PyMongo==2.3.0
flatbuffers==23.5.26
fonttools==4.47.2
fsspec==2023.12.2
humanfriendly==10.0
idna==3.6
itsdangerous==2.1.2
Jinja2==3.1.3
kiwisolver==1.4.5
MarkupSafe==2.1.4
matplotlib==3.8.2
mpmath==1.3.0
networkx==3.2.1
numpy==1.26.3
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.18.1
nvidia-nvjitlink-cu12==12.3.101
nvidia-nvtx-cu12==12.1.105
onnx==1.15.0
onnxruntime==1.16.3
opencv-python==4.9.0.80
packaging==23.2
pillow==10.2.0
protobuf==4.25.2
pyasn1==0.5.1
pycocotools==2.0.7
pymongo==4.6.1
pyparsing==3.1.1
python-dateutil==2.8.2
python-dotenv==1.0.0
python-jose==3.3.0
requests==2.31.0
rsa==4.9
segment-anything==1.0
six==1.16.0
sqlparse==0.4.4
sympy==1.12
torch==2.1.2
torch-vision==0.1.6.dev0
torchvision==0.16.2
triton==2.1.0
typing_extensions==4.9.0
urllib3==2.1.0
Werkzeug==3.0.1

```

### backend/server.py

```python
import json
from six.moves.urllib.request import urlopen
from functools import wraps
from PIL import Image

from flask import (
    Flask,
    redirect,
    request,
    jsonify,
    _request_ctx_stack,
    flash,
    send_file,
)
from flask_cors import cross_origin, CORS

from werkzeug.utils import secure_filename
from jose import jwt

import os
from io import BytesIO
from sam.create_model import embed_image, load_model
from sam.create_model import mask_query

# SAM

AUTH0_DOMAIN = "ttps://ohm-snap-auth0.com"
API_AUDIENCE = "https://ohm-snap-auth0.com"
ALGORITHMS = ["RS256"]

APP = Flask(__name__)
CORS(APP)
model = load_model()


# Error handler
class AuthError(Exception):
    def __init__(self, error, status_code):
        self.error = error
        self.status_code = status_code


@APP.errorhandler(AuthError)
def handle_auth_error(ex):
    response = jsonify(ex.error)
    response.status_code = ex.status_code
    return response


# /server.py


# Format error response and append status code
def get_token_auth_header():
    """Obtains the Access Token from the Authorization Header"""
    auth = request.headers.get("Authorization", None)
    if not auth:
        raise AuthError(
            {
                "code": "authorization_header_missing",
                "description": "Authorization header is expected",
            },
            401,
        )

    parts = auth.split()

    if parts[0].lower() != "bearer":
        raise AuthError(
            {
                "code": "invalid_header",
                "description": "Authorization header must start with" " Bearer",
            },
            401,
        )
    elif len(parts) == 1:
        raise AuthError(
            {"code": "invalid_header", "description": "Token not found"}, 401
        )
    elif len(parts) > 2:
        raise AuthError(
            {
                "code": "invalid_header",
                "description": "Authorization header must be" " Bearer token",
            },
            401,
        )

    token = parts[1]
    return token


def requires_auth(f):
    """Determines if the Access Token is valid"""

    @wraps(f)
    def decorated(*args, **kwargs):
        token = get_token_auth_header()
        jsonurl = urlopen("https://" + AUTH0_DOMAIN + "/.well-known/jwks.json")
        jwks = json.loads(jsonurl.read())
        unverified_header = jwt.get_unverified_header(token)
        rsa_key = {}
        for key in jwks["keys"]:
            if key["kid"] == unverified_header["kid"]:
                rsa_key = {
                    "kty": key["kty"],
                    "kid": key["kid"],
                    "use": key["use"],
                    "n": key["n"],
                    "e": key["e"],
                }
        if rsa_key:
            try:
                payload = jwt.decode(
                    token,
                    rsa_key,
                    algorithms=ALGORITHMS,
                    audience=API_AUDIENCE,
                    issuer="https://" + AUTH0_DOMAIN + "/",
                )
            except jwt.ExpiredSignatureError:
                raise AuthError(
                    {"code": "token_expired", "description": "token is expired"}, 401
                )
            except jwt.JWTClaimsError:
                raise AuthError(
                    {
                        "code": "invalid_claims",
                        "description": "incorrect claims,"
                        "please check the audience and issuer",
                    },
                    401,
                )
            except Exception:
                raise AuthError(
                    {
                        "code": "invalid_header",
                        "description": "Unable to parse authentication" " token.",
                    },
                    401,
                )

            _request_ctx_stack.top.current_user = payload
            return f(*args, **kwargs)
        raise AuthError(
            {"code": "invalid_header", "description": "Unable to find appropriate key"},
            401,
        )

    return decorated


def requires_scope(required_scope):
    """Determines if the required scope is present in the Access Token
    Args:
        required_scope (str): The scope required to access the resource
    """
    token = get_token_auth_header()
    unverified_claims = jwt.get_unverified_claims(token)
    if unverified_claims.get("scope"):
        token_scopes = unverified_claims["scope"].split()
        for token_scope in token_scopes:
            if token_scope == required_scope:
                return True
    return False


# Controllers API


# This doesn't need authentication
@APP.route("/api/public")
@cross_origin(headers=["Content-Type", "Authorization"])
def public():
    response = (
        "Hello from a public endpoint! You don't need to be authenticated to see this."
    )
    return jsonify(message=response)


# This needs authentication
@APP.route("/api/private")
@cross_origin(headers=["Content-Type", "Authorization"])
@requires_auth
def private():
    response = (
        "Hello from a private endpoint! You need to be authenticated to see this."
    )
    return jsonify(message=response)


# This needs authorization
@APP.route("/api/private-scoped")
@cross_origin(headers=["Content-Type", "Authorization"])
@requires_auth
def private_scoped():
    if requires_scope("read:messages"):
        response = "Hello from a private endpoint! You need to be authenticated and have a scope of read:messages to see this."
        return jsonify(message=response)
    raise AuthError(
        {
            "code": "Unauthorized",
            "description": "You don't have access to this resource",
        },
        403,
    )


ALLOWED_EXTENSIONS = {"jpg", "jpeg", "png"}


def allowed_file(filename):
    return "." in filename and filename.rsplit(".", 1)[1].lower() in ALLOWED_EXTENSIONS


@APP.route("/mask", methods=["POST", "GET"])
def mask():
    if request.method == "POST":
     
[truncated — 3933 more characters]
```

### frontend/src/app/layout.tsx

```typescript
import type { Metadata } from "next";
import { Inter } from "next/font/google";
import "./globals.css";
import { UserProvider } from "@auth0/nextjs-auth0/client";
import LocalFont from 'next/font/local'

export const metadata: Metadata = {
  title: "Create Next App",
  description: "Generated by create next app",
};

const inter = Inter({
  subsets: ["latin"],
  variable: "--font-inter",
});

const calSans = LocalFont({
  src: "../../public/fonts/CalSans-SemiBold.ttf",
  variable: "--font-calsans",
});

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  return (
    <html lang="en" className={[inter.variable, calSans.variable].join(" ")}>
      <UserProvider>
        <body className={inter.className}>{children}</body>
      </UserProvider>
    </html>
  );
}

```

### frontend/src/app/page.tsx

```typescript
import Image from "next/image";
import Particles from "./components/particles"
import Navbar from "./components/navbar";
import Link from "next/link";

export default function Home() {
  return (
    <div className="flex flex-row items-center justify-center w-screen h-screen overflow-hidden bg-gradient-to-tl from-black via-zinc-600/20 to-black">
      <Particles
        className="absolute inset-0 -z-10 animate-fade-in-instant"
        quantity={500}
        staticity={100}
      />
      <Navbar header="">
        <Link href='/api/auth/login?returnTo=/dashboard'>Login</Link>
      </Navbar>
      <div>
      </div>
      <h1 className="border-4 bg-white absolute top-1/2 left-1/2 z-10 p-2 bg-clip-text text-transparent white duration-100 cursor-default font-display text-3xl md:text-5xl xl:text-7xl animate-fade-in whitespace-nowrap transform -translate-x-1/2 -translate-y-1/2">
        Ohm Snap!
      </h1>
      </div>
  );
}

```

### frontend/src/app/dashboard/page.tsx

```typescript
'use client'
import AppContextProvider from "../components/hooks/context";
import Dashboard from "./dashboard";
export default function DashboardPage() {
    return(
        <AppContextProvider>
            <Dashboard />
        </AppContextProvider>
    )
};


```

### frontend/src/app/login/page.tsx

```typescript

import Particles from "../components/particles"
import Navbar from "../components/navbar"

export default function Login() {
    return(
    <div className="flex flex-row items-center justify-center w-screen h-screen overflow-hidden bg-gradient-to-tl from-black via-zinc-600/20 to-black">
      <Particles
        className="absolute inset-0 -z-10 animate-fade-in-instant"
        quantity={500}
        staticity={100}
      />
      <a className="bg-white absolute top-1/2 left-1/2 z-10 p-1 bg-clip-text text-transparent white duration-100 cursor-default font-display text-3xl md:text-5xl xl:text-7xl whitespace-nowrap transform -translate-x-1/2 -translate-y-1/2" href="/api/auth/login">Login</a>
      <h1 >
        Sign In
      </h1>
      </div>
    );
}
```

### frontend/src/app/api/auth/[auth0]/route.js

```javascript
// app/api/auth/[auth0]/route.js
import { handleAuth } from '@auth0/nextjs-auth0';

export const GET = handleAuth();
```

### frontend/postcss.config.js

```javascript
module.exports = {
    plugins: {
      tailwindcss: {
      },
      autoprefixer: {},
    },
  };
  
```

### backend/model_install.sh

```shell
#!/usr/bin/bash

wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth -O ./sam/sam_vit_h_4b8939.pth
```

[23 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]