# Project export: Identity AI

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Real Identity, Verified Instantly: Advanced facial matching and online profile tracking to stop fraud in its tracks.
- Devpost: https://devpost.com/software/identity-ai
- GitHub: https://github.com/mschubs/IdentityAI
- Video: https://www.youtube.com/embed/5B6Yhyr4uA0?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 4 GitHub contributor(s) — Derek (42 commits), Michael Guo (9 commits), nandansrikrishna (6 commits), mschubs (5 commits)

## Devpost submission (written by the team)

### Inspiration

Every year, 5 million human trafficking victims are transported using counterfeited or fake passports and ID cards. These fake identity documents not only fuel human trafficking but also facilitate identity theft, allowing criminals to assume false identities, evade law enforcement, and expand their illegal operations. The problem is only getting worse—according to The New York Times, a new generation of 'unbeatable' fake IDs is making it increasingly difficult to distinguish real documents from fraudulent ones. To combat this growing crisis, we developed IdentityAI.

### What it does

Our project can be used to detect even the most perfectly crafted false identity card. We know that at least one of the fields on this ID is falsified, whether it be the photo, name, birthday, address, or all of the above. And, if the fields are all valid, then we can check the similarity of the ID photo to the face of its holder. So, we leverage web scraping, machine learning, and AI agents to find these discrepancies. We take pictures of a person and their ID using the Meta Ray Bans glasses, process this information, and utilize agents to crawl the internet and verify the validity of the ID. We have a web UI where a user can view the results of our search, which include a face-matching confidence score, address, age, and other identifying information. Our project uses a multitude of strategies to achieve this goal. We built a web scraper to perform reverse image searching. We leverage Perplexity’s Sonar and other online sources to validate the person’s name, age, and address. At the end of this workflow, Identity AI presents results to the user, who will then know the validity of an ID regardless of its quality.

### How we built it

We built the backend using python and FastAPI. These include endpoints to OCR the ID, to gather additional information off internet searches, as well as interact with large language models with tools and agents. We leverage a Yolo model to detect the largest face in an image. This is then sent to our reverse image search tool to find related articles, pictures, and websites that could have correlating images. After receiving this corpus of information, we run the links acquired into crawler instances to retrieve the data from the links. This data is then fed into an Anthropic LLM to congregate the data into meaningful leads. Meta Ray Bans glasses do not have an API for us to use, so we got creative in order to take advantage of this cool technology. We created a facebook account to send images to from the glasses and had a chrome extension monitor the FaceBook Messenger DOM in order to then send these images to our server. Additionally, we leveraged OCR to compare the ID owner’s face and the photo on the ID, and we determine this similarity by taking an L2 norm of the image encodings. We then utilized a formula to convert this norm into a percentage based confidence score. We also queried Perplexity’s Sonar API to gather more leads on whether the information on the ID card is falsified. Furthermore, we experimented with an agentic workflow to leverage all these tools we created and make qualitative decisions about what information is still necessary in order to come to a conclusion about the realness of an ID. We built an orchestrator agent who interacts with OSINT agents and a decision agent. We provided these agents with function calls to various tools we built.

### Challenges we ran into

Creating agents that run together and deliberate with our dedicated tools to find the next best move. Whether that is to gather more information with our tools or if the information is already good enough, determine the final output. Our agents needed to decide whether to gather more information or finalize a decision based on existing data. Striking the right balance between exploration and resolution was tricky because without clear stopping conditions we risked infinite loops or premature decisions with low confidence. To tackle this, we implemented a structured decision loop that iterates through agent deliberation while enforcing constraints on information retrieval. Each iteration, the DecisionAgent assesses available data, determining whether to request additional OSINT queries or finalize its output. Other challenges that we ran into were the quality of the meta glasses. In testing, phone cameras capture images in high resolution allowing us to manually enlarge and crop the image. We also encountered difficulties in web scraping. Websites we interacted with had bot detection such as reCAPTCHA, and we got creative combining available tools to bypass these measures (e.g. mocking a real user’s chrome profile, randomizing delays when interacting with the DOM). All to allow us to continue our altruistic crawl of the internet!

### Accomplishments we're proud of

Based on just an individual's face, we can build a comprehensive profile leveraging smaller pieces of evidence from various sources on the web. We can then cross reference this information with any ID card to validate identification with a high degree of accuracy. We truly embraced the hacker mentality, interacting with technologies and tools in ways that are not their original intended purpose, and thus we had to think outside the box to still be able to achieve our goals. Overall, we are proud of the extent of the functionality we were able to build for this project.

### What we learned

Building complex agentic systems is challenging. Unlike standard software development, where logic is explicit, designing AI agents often feels like recursive programming, where you must construct the right scaffolding and then take a leap of faith, trusting the system to generalize and adapt dynamically. One of the biggest lessons we learned is that orchestrating multiple AI agents requires a different mindset. Instead of writing step-by-step instructions, we had to think about how agents interact, how they handle uncertainty, and how to design feedback loops to improve performance over time and rebound from errors. We also learned a lot about prompt engineering and system design in shaping agent behavior. Small changes in instructions, memory design, or action constraints could lead to vastly different outputs. For example, by adding a simple memory list of previous queries for one of our agents it effectively could learn from its past queries and make better requests as a result.

### What's next

A potential extension of our project would be to actively match faces against the FBI’s national kidnapping database. By integrating real-time facial recognition, we could help identify trafficking victims who need to be rescued and alert authorities in critical situations.

## README (from the GitHub repository)

For server:

start via python server.py

For extension:

upload to chrome via chrome://extensions/ (developer mode, then load unpacked and load the fb-extension folder)

activate the extension and in the popup, then it will start uploading new images to the server that are sent to that messenger chat.

activate the extension in a www.messenger.com chat

## Detected evidence (automated analysis)

Indexed codebase: 38 recognized source files, 146 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- Anthropic (technology) — claimed on Devpost, not found in the code
- FastAPI (technology) — claimed on Devpost, not found in the code
- OpenAI (technology) — claimed on Devpost, not found in the code
- TypeScript (language) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (55 of 55)

```
.DS_Store
.gitignore
.vscode/launch.json
backend/__init__.py
backend/agents/__init__.py
backend/agents/cookies.json
backend/agents/decision_agent.py
backend/agents/document_agent_helpers/face_detection.py
backend/agents/document_agent.py
backend/agents/face_verification_agent.py
backend/agents/orchestrator.py
backend/agents/osint_agent.py
backend/agents/reverse_image_agent_helpers/cookies.json
backend/agents/reverse_image_agent.py
backend/main.py
backend/server.py
backend/similarity.py
dashboard/.gitignore
dashboard/components.json
dashboard/eslint.config.js
dashboard/index.html
dashboard/jsconfig.json
dashboard/package.json
dashboard/README.md
dashboard/src/App.jsx
dashboard/src/components/ui/avatar.jsx
dashboard/src/components/ui/badge.jsx
dashboard/src/components/ui/button.jsx
dashboard/src/components/ui/card.jsx
dashboard/src/components/ui/dialog.jsx
dashboard/src/components/ui/label.jsx
dashboard/src/components/ui/scroll-area.jsx
dashboard/src/components/ui/separator.jsx
dashboard/src/data.json
dashboard/src/index.css
dashboard/src/lib/utils.js
dashboard/src/main.jsx
dashboard/tsconfig.app.json
dashboard/tsconfig.json
dashboard/vite.config.js
fastpeople.py
fb-extension/content.js
fb-extension/manifest.json
fb-extension/popup.html
fb-extension/popup.js
groq_test.py
id2text.py
output.json
README.md
results.json
reverse_image_agent_output.json
test_stuff/groq_test.py
trash/id2dataWVerify.py
trash/similarity.py
urls.json
```

### Dependencies

- dashboard/package.json: @eslint/js@^9.19.0, @radix-ui/react-avatar@^1.1.3, @radix-ui/react-dialog@^1.1.6, @radix-ui/react-label@^2.1.2, @radix-ui/react-scroll-area@^1.2.3, @radix-ui/react-separator@^1.1.2, @radix-ui/react-slot@^1.1.2, @shadcn/ui@^0.0.4, @tailwindcss/vite@^4.0.6, @types/node@^22.13.4, @types/react@^19.0.8, @types/react-dom@^19.0.3, @vitejs/plugin-react@^4.3.4, class-variance-authority@^0.7.1, clsx@^2.1.1, eslint@^9.19.0, eslint-plugin-react@^7.37.4, eslint-plugin-react-hooks@^5.0.0, eslint-plugin-react-refresh@^0.4.18, globals@^15.14.0, lucide-react@^0.475.0, react@^19.0.0, react-dom@^19.0.0, recharts@^2.15.1, tailwind-merge@^3.0.1, tailwindcss@^4.0.6, tailwindcss-animate@^1.0.7, vite@^6.1.0

### Recent commits (newest first)

- Merge pull request #9 from mschubs/agent-loop
- final push
- final commit
- fuck
- marcus ria
- idk if this will work but yea
- ok it kinda works
- add marcus agent thign
- jfc
- bout to make this better
- Updated oscint prompt
- perplexity route fucked up, but can run all iterations
- Added address split
- Modified oscint prompt and fastperson output dict
- initial loop setup
- Merge pull request #7 from mschubs/marcus/decision-agent
- Merge pull request #6 from mschubs/nandans/osint
- Merge branch 'main' into nandans/osint
- Merge pull request #5 from mschubs/derekmil/orch
- Merge branch 'main' into derekmil/orch

## Key source files (fetched from GitHub, selected and truncated for size)

### dashboard/package.json

```
{
  "name": "dashboard",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "lint": "eslint .",
    "preview": "vite preview"
  },
  "dependencies": {
    "@radix-ui/react-avatar": "^1.1.3",
    "@radix-ui/react-dialog": "^1.1.6",
    "@radix-ui/react-label": "^2.1.2",
    "@radix-ui/react-scroll-area": "^1.2.3",
    "@radix-ui/react-separator": "^1.1.2",
    "@radix-ui/react-slot": "^1.1.2",
    "@shadcn/ui": "^0.0.4",
    "@tailwindcss/vite": "^4.0.6",
    "class-variance-authority": "^0.7.1",
    "clsx": "^2.1.1",
    "lucide-react": "^0.475.0",
    "react": "^19.0.0",
    "react-dom": "^19.0.0",
    "recharts": "^2.15.1",
    "tailwind-merge": "^3.0.1",
    "tailwindcss": "^4.0.6",
    "tailwindcss-animate": "^1.0.7"
  },
  "devDependencies": {
    "@eslint/js": "^9.19.0",
    "@types/node": "^22.13.4",
    "@types/react": "^19.0.8",
    "@types/react-dom": "^19.0.3",
    "@vitejs/plugin-react": "^4.3.4",
    "eslint": "^9.19.0",
    "eslint-plugin-react": "^7.37.4",
    "eslint-plugin-react-hooks": "^5.0.0",
    "eslint-plugin-react-refresh": "^0.4.18",
    "globals": "^15.14.0",
    "vite": "^6.1.0"
  }
}

```

### backend/main.py

```python
from dotenv import load_dotenv
import os
from groq import Groq

load_dotenv()  # This loads the variables from .env

def run_groq_agent():
    try:
        # Ensure your API key is available
        api_key = os.environ.get("GROQ_API_KEY")
        if not api_key:
            raise ValueError("Please set the GROQ_API_KEY environment variable.")

        # Create a Groq client
        client = Groq(api_key=api_key)

        # Specify your desired model
        model_name = "llama-3.3-70b-versatile"

        # Define a conversation
        messages = [
            {"role": "system", "content": "You are a helpful AI assistant."},
            {"role": "user", "content": "How do I start using Groq API to build an agent?"}
        ]

        # # Request a completion from the Groq API
        # chat_completion = client.chat.completions.create(
        #     messages=messages,
        #     model=model_name,
        #     temperature=0.7,
        #     max_tokens=1000,
        # )

        # # Print the generated response
        # print(chat_completion.choices[0].message.content)
        
    except ValueError as ve:
        print(f"Configuration error: {ve}")
    except Exception as e:
        print(f"An error occurred: {e}")

if __name__ == "__main__":
    run_groq_chat()

```

### backend/server.py

```python
from fastapi import FastAPI, UploadFile, File, HTTPException
from fastapi.responses import FileResponse
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from similarity import compare_faces
import shutil
import os
from datetime import datetime

from agents.orchestrator import OrchestratorAgent
from agents.document_agent import DocumentParsingAgent
from agents.face_verification_agent import FaceVerificationAgent
from agents.osint_agent import OSINTAgent
from agents.decision_agent import DecisionAgent
from agents.orchestrator import OrchestratorStatus
from agents.reverse_image_agent import ReverseImageAgent
app = FastAPI()

# Add CORS middleware - allow all origins for now (not recommended for production)
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # Or specify the origins you want to allow
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Create uploads directory if it doesn't exist
UPLOAD_DIR = "uploads"
os.makedirs(UPLOAD_DIR, exist_ok=True)

class FaceCompareRequest(BaseModel):
    known_image: str
    unknown_image: str

doc_parser = DocumentParsingAgent()
face_verifier = FaceVerificationAgent()
osint_agent = OSINTAgent()
decision_agent = DecisionAgent()
reverse_image_agent = ReverseImageAgent()
# Create the orchestrator
orchestrator = OrchestratorAgent(
    document_parser=doc_parser,
    face_verifier=face_verifier,
    osint_agent=osint_agent,
    decision_agent=decision_agent,
    reverse_image_agent=reverse_image_agent
)

@app.post("/compare-faces")
async def compare_face_images(request: FaceCompareRequest):
    try:
        result = compare_faces(request.known_image, request.unknown_image)
        return {"success": True, "result": result}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e)) 

@app.post("/upload-image/")
async def upload_image(file: UploadFile = File(...)):
    # Validate that the uploaded file is an image
    if not file.content_type.startswith('image/'):
        return {"error": "File must be an image"}
    
    # Create unique filename using timestamp
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    file_extension = os.path.splitext(file.filename)[1]
    new_filename = f"{timestamp}{file_extension}"
    file_path = os.path.join(UPLOAD_DIR, new_filename)
    
    # Save the uploaded file
    with open(file_path, "wb") as buffer:
        shutil.copyfileobj(file.file, buffer)
        
    print("awaiting orchestrator")
    # Pass image to orchestrator (async call)
    await orchestrator.accept_image(file_path)
    print("orchestrator done")
    # Optionally, if the orchestrator just finished RUN_VERIFICATION,
    # you might want to return something about the results. For example:
    if orchestrator.status == OrchestratorStatus.PROCESSING:
        # We may have a "final_result" if run_verification is done.
        # But if you want to respond right away (non-blocking), you could do so.
        print(f"Orchestrator state: {orchestrator.status.value}")
        return {"status": "Verification started, results will come later"}
    else:
        print(f"Orchestrator state: {orchestrator.status.value}")
        return {"status": f"Orchestrator state: {orchestrator.status.value}"}
    
    # Pass image to id to text, and gets this out
    # "observed": {
    #     "profileImage": str,
    #     "name": str,
    #     "address": str,
    #     "dateOfBirth": str,
    #     "expiryDate": str,
    #     "nationality": str,
    #     "gender": str
    # }
    #
    # Also gets a picture of the face and id_face out
    #
    # take face pic and id_face pic and pass to similarty
    #
    # take the face and pass to the pimeyes/firecrawl pipeline
    # get result that includes the markdown of all the articles/webpages where a picture of their face is found
    # 
    
    # agent time!

@app.post("/reset")
async def reset():
    orchestrator.reset()
    return {"status": "Orchestrator reset"}

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)
```

### dashboard/src/main.jsx

```javascript
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import './index.css'
import App from './App.jsx'

createRoot(document.getElementById('root')).render(
  <StrictMode>
    <App />
  </StrictMode>,
)

```

### dashboard/src/App.jsx

```javascript
import {
  Card,
  CardContent,
  CardHeader,
  CardFooter,
} from "@/components/ui/card";
import { Avatar, AvatarFallback, AvatarImage } from "@/components/ui/avatar";
import { ArrowRight, Check, X } from "lucide-react";
import { useState, useEffect, useRef } from "react";
import { ScrollArea, ScrollBar } from "@/components/ui/scroll-area";
import { Separator } from "@/components/ui/separator";
import { Label } from "@/components/ui/label";
import { Button } from "@/components/ui/button";
import {
  Dialog,
  DialogContent,
  DialogHeader,
  DialogTitle,
  DialogTrigger,
} from "@/components/ui/dialog";
import { PieChart, Pie, Cell } from "recharts";
import data from "./data.json";

function App() {
  const [faceSimilarityResult, setFaceSimilarityResult] = useState(null);
  const [isLoading, setIsLoading] = useState(true);
  const [error, setError] = useState(null);
  const [currentIDIndex, setCurrentIDIndex] = useState(0);
  const [currentIndex, setCurrentIndex] = useState(0);
  const mountedRef = useRef(false);
  const [dialogOpen, setDialogOpen] = useState(false);
  const [allData, setAllData] = useState(data.allData);
  const [previousIDs, setPreviousIDs] = useState([]);

  useEffect(() => {
    setPreviousIDs(
      data.allData.map((data, index) => ({
        index,
        profileImage: data.observed.profileImage,
        name: data.observed.name,
        date_created: "2024-01-01",
      }))
    );
  }, [data.allData]);

  useEffect(() => {
    const loadData = async () => {
      try {
        setAllData(data.allData);
      } catch (error) {
        console.error("Error loading data:", error);
        setError("Error loading data");
      }
    };

    loadData();
  }, []);

  const calculateAge = (dateOfBirth) => {
    const [month, day, year] = dateOfBirth.split("/");
    const birthDate = new Date(year, month - 1, day);
    const today = new Date();
    let age = today.getFullYear() - birthDate.getFullYear();
    const monthDiff = today.getMonth() - birthDate.getMonth();

    if (
      monthDiff < 0 ||
      (monthDiff === 0 && today.getDate() < birthDate.getDate())
    ) {
      age--;
    }

    return age;
  };

  const getMatchScore = (data, observedData) => {
    let score = 0;
    // Name match
    if (data.name.toLowerCase() === observedData.name.toLowerCase()) {
      score += 1;
    }
    // Address match (both lines)
    if (
      data["address-line-1"].toLowerCase() ===
        observedData["address-line-1"].toLowerCase() &&
      data["address-line-2"].toLowerCase() ===
        observedData["address-line-2"].toLowerCase()
    ) {
      score += 1;
    }
    // Age match
    if (
      parseInt(data.age) === parseInt(calculateAge(observedData.dateOfBirth))
    ) {
      score += 1;
    }
    return score;
  };

  const identityData = allData.length > 0 && allData[currentIDIndex] 
    ? {
        ...allData[currentIDIndex],
        online: [...(allData[currentIDIndex].online || [])].sort((a, b) => {
          return (
            getMatchScore(b, allData[currentIDIndex].observed) -
            getMatchScore(a, allData[currentIDIndex].observed)
          );
        }),
      }
    : null;

  useEffect(() => {
    if (allData.length > 0) {
      setCurrentIndex(allData[currentIDIndex].online.length - 1);
    }
  }, [allData, currentIDIndex]);

  useEffect(() => {
    console.log("Effect running with currentIDIndex:", currentIDIndex);
    console.log("mountedRef.current:", mountedRef.current);

    if (!mountedRef.current) {
      mountedRef.current = true;
      return;
    }

    if (!identityData?.observed?.profileImage || !identityData?.IRL_image) {
      setError("Missing image data");
      setIsLoading(false);
      return;
    }

    const checkFaceSimilarity = async () => {
      try {
        const response = await fetch("http://localhost:8000/compare-faces", {
          method: "POST",
          headers: {
            "Content-Type": "application/json",
          },
          body: JSON.stringify({
            known_image: identityData.observed.profileImage,
            unknown_image: identityData.observed.faceImage,
          }),
        });

        if (!response.ok) {
          throw new Error(`HTTP error! status: ${response.status}`);
        }

        const data = await response.json();

        if (!data || !data.result) {
          throw new Error("Invalid response data");
        }

        console.log("Face comparison result:", data.result);
        setError(null);
        setFaceSimilarityResult([
          parseInt(data.result) > 50 ? true : false,
          parseFloat(data.result),
        ]);
      } catch (error) {
        console.error("Error comparing faces:", error);
        setError("Server not responding");
        setFaceSimilarityResult(null);
      } finally {
        setIsLoading(false);
      }
    };

    checkFaceSimilarity();
  }, [currentIDIndex]);

  const handleIdChange = (index) => {
    if (index !== currentIDIndex && allData[index]) {
      setCurrentIDIndex(index);
      setCurrentIndex(allData[index]?.online?.length - 1 || 0);
      setIsLoading(true);
      setFaceSimilarityResult(null);
      setDialogOpen(false);
    } else {
      setDialogOpen(false);
    }
  };

  return (
    <div className="h-screen w-full bg-gray-100 dark:bg-gray-900 overflow-hidden relative">
      <div className="container mx-auto p-8 h-full overflow-y-auto overflow-x-hidden">
        <div className="flex flex-col md:flex-row gap-8">
          <div className="flex-shrink-0">
            <div className="flex flex-col">
              <Label className="text-lg font-semibold mb-4 bg-gray-100 dark:bg-gray-900 py-2 z-10">
                Observed ID Data
              </Label>
              <div className="flex items-center">
                <Card className="w-[340px] border-2 hover:border-primary/50 transition-colors">
                  <CardHeader className="flex flex-row items-center gap-4">
                    <div>
                      <h2 c
[truncated — 16519 more characters]
```

### fastpeople.py

```python
import requests
import os
from dotenv import load_dotenv

# Load environment variables
load_dotenv('secret.env')

# Get credentials from environment variables
key_name = os.getenv('ENDATO_KEY_NAME')
key_pass = os.getenv('ENDATO_KEY_PASS')

def search_person(payload):
    payload['FilterOptions'] = [
            "IncludeEmptyFirstNameResults",
            "IncludeSevenDigitPhoneNumbers",
            "IncludeLowQualityAddresses"
        ]

    url = "https://devapi.endato.com/PersonSearch"

    headers = {
        "accept": "application/json",
        "galaxy-ap-name": key_name,
        "galaxy-ap-password": key_pass,
        "galaxy-search-type": "Person",
        "content-type": "application/json"
    }
    
    response = requests.post(url, headers=headers, json=payload)
    return response.text

# Example usage
if __name__ == "__main__":
    sample_payload = {
        "FirstName": "Nandan",
        "MiddleName": "M",
        "LastName": "Srikrishna",
    }

    result = search_person(sample_payload)
    print(result)
```

### groq_test.py

```python
import os
from dotenv import load_dotenv
import json
from groq import Groq

load_dotenv('secret.env')  # Load variables from .env

client = Groq(
    api_key=os.environ.get("GROQ_API_KEY"),
)

def create_chat_completion(content, model):  # New function to create chat completion
    return client.chat.completions.create(
        messages=[
            {
                "role": "user",
                "content": content,
            }
        ],
        model=model,
    )

def extract_json(text):  # Find content between json and markers
    start_marker = "json"
    end_marker = "```"
    
    try:
        # Find the start of JSON content
        start_index = text.find(start_marker) + len(start_marker)
        
        # Find the end of JSON content
        end_index = text.find(end_marker, start_index)
        
        if start_index == -1 or end_index == -1:
            raise ValueError("JSON markers not found in text")
            
        # Extract the JSON string
        json_str = text[start_index:end_index].strip()
        
        # Parse the JSON string
        return json.loads(json_str)
        
    except Exception as e:
        print(f"Error extracting JSON: {e}")
        return None

def run_groq(content, model):
    chat_completion = create_chat_completion(content, model)
    response_content = chat_completion.choices[0].message.content
    print(response_content)
    json_data = extract_json(response_content)
    print(json.dumps(json_data, indent=2))

# model = "llama-3.1-8b-instant"
if __name__ == "__main__":
    try:
        content = "Using JSON format, Please return the names of people mentioned in the following text and personal information about them: \n\n [{'markdown': '[Skip to content](https://www.regis.org/article?id=11851#content)\n\n## Regis Senior\'s Research Supports Solar System Formation Theory\n\n[![](https://wpnews.regis.org/wp-content/uploads/2021/05/Schubert.jpg)](https://wpnews.regis.org/wp-content/uploads/2021/05/Schubert.jpg)_Marcus Schubert \'21 poses with his research simulation in the Regis Quad._\n\nIn a remarkable achievement for Regis’ Science Research Program (SRP), senior Marcus Schubert ’21 has successfully simulated the collapse of an interstellar dust cloud due to gravity. His research not only corroborates running theories on the formation of our solar system, but supports the hypothesis that such a gravitational collapse can generate other planetary systems.\n\nAccording to the Solar Nebula Theory, our solar system was formed approximately 4.5 billion years ago from a disk of spinning stardust and gas. As gravity pressed on this collection of interstellar material, the build up of pressure caused an expulsion of energy that formed our Sun, planets, and smaller celestial bodies like asteroids, moons, and comets. While a widely accepted theory, any system of more than two bodies cannot be solved mathematically and must be modeled through simulations to prove, which prompted Schubert to pursue this research question throughout his senior year.\n\n“I have experience coding in Python from programming video games and building machine learning models in the past, so I thought it would be a good idea to try out a simulation using gravity,” said Schubert, who was mentored by Regis Physics teacher Dr. Luca Matone throughout his research project. “I find physics interesting since its various laws can be used to explain almost everything we see in the world. It\'s also exciting to know how much we still don\'t know in the field.”\n\nSchubert’s research consisted of developing an algorithm that computed the gravitational forces acting on each interstellar mass due to all the others at a particular moment in time. As this was completed, a code would then calculate the positions each of these masses would take as a result of the forces acting on them. As time was artificially advanced, the forces and positions of the celestial bodies were constantly recomputed, and after several methodological hurdles and some creative thinking, a simulation that proved the Solar Nebula Theory emerged. Schubert’s numerical calculations, organized visually in the below display, not only show that a sudden and sizable collapse in gravity would force space masses to coalesce into a larger body (our Sun), but also that such a collapse would force some bodies (our planets) to fall into this central mass’ gravitational pull.\n\n“Thanks to Marcus’ work, we can see how computer science plays a critical role in the sciences,” Dr. Matone said. “Scientists can test their ideas or hypotheses just to see if they are plausible before embarking on an expensive experiment or costly telescope observation. Well done, Marcus!”\n\nSchubert will be studying Engineering at the University of Michigan next fall, where he hopes to continue exploring the passions for science and research he has developed throughout his time at Regis.\n\n“SRP has been an awesome opportunity for me to explore my research interests,” said Schubert. “I appreciate how I can brainstorm ideas with my mentor and learn loads of invaluable advice. I really learned how to get creative with research. If I have a hypothesis and can think of a way to test it, that\'s research!”\n\nMarcus Schubert \'21 SRP Project: Gravitational Collapse Simulation - YouTube\n\nRegis High School\n\n760 subscribers\n\n[Marcus Schubert \'21 SRP Project: Gravitational Collapse Simulation](https://www.youtube.com/watch?v=u2sAC1-Ihso)\n\nRegis High School\n\nSearch\n\nWatch later\n\nShare\n\nCopy link\n\nInfo\n\nShopping\n\nTap to unmute\n\nIf playback doesn\'t begin shortly, try restarting your device.\n\nMore videos\n\n## More videos\n\nYou\'re signed out\n\nVideos you watch may be added to the TV\'s watch history and influence TV recommendations. To avoid this, cancel and sign in to YouTube on your computer.\n\nCancelConfirm\n\nShare\n\nInclude playlist\n\nAn error occurred while retrieving sharing information. Please try again later.\n\n[Watc
[truncated — 294 more characters]
```

### id2text.py

```python
from anthropic import Anthropic
import base64
import os
from dotenv import load_dotenv
import cv2
import json
from backend.agents.document_agent_helpers.face_detection import detect_primary_faces_yolo

# Load environment variables
load_dotenv('secret.env')

# Function to encode the image
def encode_image(image_path):
  with open(image_path, "rb") as image_file:
    return base64.b64encode(image_file.read()).decode('utf-8')

schema = {
    "observed": {
        "profileImage": str,
        "name": str,
        "address-line-1": str,
        "address-line-2": str,
        "dateOfBirth": str,
        "expiryDate": str,
        "nationality": str,
        "gender": str
    }
}

stateIDFormats = {
  "Alabama":      ["first+middle", "last"],  # Verified via sample images & user reports
  "Alaska":       ["last", "first+middle"],
  "Arizona":      ["last", "first+middle"],
  "Arkansas":     ["last", "first+middle"],
  "California":   ["first+middle", "last"],  # California DMV: FN = first+middle, LN = last
  "Colorado":     ["last", "first+middle"],
  "Connecticut":  ["last", "first+middle"],
  "Delaware":     ["last", "first+middle"],
  "District of Columbia": ["last", "first+middle"],
  "Florida":      ["last", "first+middle"],
  "Georgia":      ["last", "first+middle"],
  "Hawaii":       ["last", "first+middle"],
  "Idaho":        ["last", "first+middle"],
  "Illinois":     ["last", "first+middle"],
  "Indiana":      ["last", "first+middle"],
  "Iowa":         ["last", "first+middle"],
  "Kansas":       ["last", "first+middle"],
  "Kentucky":     ["last", "first+middle"],
  "Louisiana":    ["last", "first+middle"],
  "Maine":        ["last", "first+middle"],
  "Maryland":     ["last", "first+middle"],
  "Massachusetts":["last", "first+middle"],
  "Michigan":     ["first+middle", "last"],  # Michigan's new design confirmed on Michigan.gov
  "Minnesota":    ["last", "first+middle"],
  "Mississippi":  ["last", "first+middle"],
  "Missouri":     ["last", "first+middle"],
  "Montana":      ["last", "first+middle"],
  "Nebraska":     ["last", "first+middle"],
  "Nevada":       ["last", "first+middle"],
  "New Hampshire":["last", "first+middle"],
  "New Jersey":   ["last", "first+middle"],
  "New Mexico":   ["last", "first+middle"],
  "New York":     ["last", "first+middle"],
  "North Carolina":["last", "first+middle"],
  "North Dakota": ["last", "first+middle"],
  "Ohio":         ["last", "first+middle"],
  "Oklahoma":     ["last", "first+middle"],
  "Oregon":       ["last", "first+middle"],
  "Pennsylvania": ["last", "first+middle"],
  "Rhode Island": ["last", "first+middle"],
  "South Carolina":["last", "first+middle"],
  "South Dakota": ["last", "first+middle"],
  "Tennessee":    ["last", "first+middle"],
  "Texas":        ["last", "first+middle"],
  "Utah":         ["last", "first+middle"],
  "Vermont":      ["last", "first+middle"],
  "Virginia":     ["last", "first+middle"],
  "Washington":   ["last", "first+middle"],
  "West Virginia":["last", "first+middle"],
  "Wisconsin":    ["last", "first+middle"],
  "Wyoming":      ["last", "first+middle"]
};

def process_id_image(image_path):
    # Process face detection first
    _, _, cropped_faces, id_card_image = detect_primary_faces_yolo(image_path)
    
    # Save cropped face if detected
    cropped_IRL_image_path = None
    if cropped_faces is not None:
        base_name = image_path.split('/')[-1]
        cropped_IRL_image_path = "cropped_" + base_name
        cv2.imwrite(cropped_IRL_image_path, cv2.cvtColor(cropped_faces[0], cv2.COLOR_RGB2BGR))

    # Use id_card_image if available, otherwise use original image
    image_to_encode = image_path
    if id_card_image is not None:
        # Save the ID card image temporarily
        base_name = image_path.split('/')[-1]
        temp_id_path = "temp_id_" + base_name
        cv2.imwrite(temp_id_path, cv2.cvtColor(id_card_image, cv2.COLOR_RGB2BGR))
        image_to_encode = temp_id_path

    # Getting the base64 string
    base64_image = encode_image(image_to_encode)

    # # Clean up temporary file if it was created
    # if id_card_image is not None:
    #     os.remove(temp_id_path)

    # Initialize client with API key from environment
    client = Anthropic(api_key=os.getenv('ANTHROPIC_API_KEY'))

    chat_completion = client.messages.create(
        temperature=0,
        system="""
        You are a specialized OCR system for ID verification. Focus on:
        1. Exact text extraction without inference
        We are working to distinguish between fake and real IDs. 
        Do not create any NEW INFORMATION that is not on the ID itself. 
        Do not add respond with any additional information, only respond with the JSON object.""",
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": f"""
                        Please extract the information from this ID into this scheme {schema}. 
                        The ID has some combination of first name, last name and middle name. 
                        Use the state to determine the order of the names in the ID. 
                        The middle name is always on the same line as the first name, 
                        so if you can see two names on one line that must be first name followed by middle name.
                        As a backup, use this dictionary of state to format: {stateIDFormats}
                        Name should be in the format of FirstName MiddleName LastName.
                        """
                    },
                    {
                        "type": "image",
                        "source": {
                            "type": "base64",
                            "media_type": "image/jpeg",
                            "data": base64_image
                        }
                    }
                ]
            }
        ],
        model="claude-3-5-sonnet-202
[truncated — 515 more characters]
```

### dashboard/vite.config.js

```javascript
import path from "path"
import tailwindcss from "@tailwindcss/vite"
import react from "@vitejs/plugin-react"
import { defineConfig } from "vite"

export default defineConfig({
  plugins: [react(), tailwindcss()],
  resolve: {
    alias: {
      "@": path.resolve(__dirname, "./src"),
    },
  },
})

```

### dashboard/index.html

```html
<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/svg+xml" href="/vite.svg" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Vite + React</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/src/main.jsx"></script>
  </body>
</html>

```

[27 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]