# Project export: Ground Truth

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: Remove the need to manually update your docs. When you make a PR, Doc's are auto updated based on the changes you committed.
- Devpost: https://devpost.com/software/ground-truth-97nqws
- GitHub: https://github.com/Hassanmushtaq524/groundtruth
- Result: winner (Chroma: Build your project with Chroma in an AI application)
- Team: 3 GitHub contributor(s) — Hassan Mushtaq (18 commits), Joe Malatesta (16 commits), Yashwanth Alluri (5 commits)

## Devpost submission (written by the team)

### Inspiration

When I was interviewing at a ~20 person startup and reading through their introduction doc, I read that the co-founders would often spend lots of nights just catching up on docs from the prior week. To me, the last thing any employee, much less a founder, should be spending their time on is writing docs. Ground Truth lets you simply review and accept changes to your docs based on what you have been coding.

### What it does

Each time you make a commit to a specific repo, a new entry is made

### How we built it

Chroma Docs can be pretty large. Far too large to pass multiple pages into a context window. To account for this, we embedded each page of the docs into a chromaDB using their built in functions, and we query the Vector DB we created to find the most similar doc so we can pass that as context. Groq We used Groq for basically all of our small context text completion. This was especially helpful when passing a code diff into the llama model and getting a description of what was being updated. This allowed us to consistently retrieve the right documentation that we would then update Reflex.dev Since most of our initialization and ChromaDB stuff was already being done in Python, it made sense to continue using it and go ahead with Reflex for the full frontend and backend.

### Challenges we ran into

The biggest issue for us was finding good contenders for projects. Firstly, they had to be open source, non negotiable for us to test with. Secondly, they had to have a developer program or some need for docs that we could reasonably update.

### Accomplishments we're proud of

This was all of our first times working with any of these technologies (RAG anything, really) and we're proud to have put it all together in a somewhat attractive way over the 36 hours. The approach we took probably was a good 10 hours of ideation so it's nice to have it built and working

### What we learned

Pretty much everything about RAG and Reflex. We all knew very little about the domain coming into this.

### What's next

This project has real potential as a company but is an incredibly hard engineering problem. If we stay excited about building it, we could make it widespread and very modular.

## README (from the GitHub repository)

# GROUNDTRUTH

### Description
GROUNDTRUTH is a DevTool that suggests automatic documentation changes as you develop your code. Using webhooks, GROUNDTRUTH can receive your code changes and suggest real-time documentation changes. 


### Table of Contents
1. [Installation](#installation)
2. [Dependencies](#dependencies)
3. [Usage](#usage)
4. [Contributing](#contributing)

### Installation
Step-by-step guide to install and run the project.

1. Clone the repository:
   ```bash
   git clone https://github.com/Hassanmushtaq524/groundtruth.git

2. Navigate to your project
   ```bash
   cd groundtruth
   
3. Navigate to your project
   ```bash
   cd groundtruth
   
4. Install required dependencies
   ```bash
   pip install -r requirements.txt

5. Setup backend/.env 
   ```plaintext
   # Database configuration
   GROQ_API_KEY=<your_groq_key>
   OPENAI_API_KEY=<your_openai_key>
   
6. Setup ngrok:
   Login to ngrok and follow instructions
   ```bash
   ngrok HTTP <BACKEND_PORT>

### Dependencies
List of dependencies:

1. reflex
2. chromadb
3. openai
4. groq
5. reflex>=0.6.0a
6. reflex-chakra
7. requests
8. json
9. fastapi
10. os
11. re
12. base64
13. dotenv
14. typing
15. httpx
   
### Usage
Step-by-step guide to use and run the project.

1. Run reflex
   ```bash
   reflex run

Make sure reflex is set up to run. This will launch both your backend and your frontend.

### Contributers
Hassan Mushtaq: [Hassanmushtaq524](https://github.com/hassanmushtaq524)
Joe Malatesta: [Joe Malatesta](https://github.com/joemmalatesta)
Yashwanth Alluri: [Yashwanth Alluri](https://github.com/yashalluri)




## Detected evidence (automated analysis)

Indexed codebase: 22 recognized source files, 34 KB.
- JavaScript (language) — detected in the code
- Python (language) — detected in the code

## Codebase structure (from repository index)

### Files (27 of 27)

```
.gitignore
assets/chakra_color_mode_provider.js
backend/__init__.py
backend/api.py
backend/test.py
backend/utils/__init__.py
backend/utils/utils.py
chroma_db/.DS_Store
chroma_db/chroma.sqlite3
possibleUpdates.md
project/__init__.py
project/components/__init__.py
project/components/navbar.py
project/pages/__init__.py
project/pages/changes.py
project/pages/home.py
project/project.py
project/state.py
project/states/__init__.py
project/states/base.py
project/states/queries.py
project/styles/__init__.py
project/styles/styles.py
README.md
requirements.txt
rxconfig.py
updates.json
```

### Dependencies

- requirements.txt: reflex@>=0.6.0a, reflex-chakra

### Recent commits (newest first)

- push updates
- Merge branch 'new-brnach'
- storing for later
- Added link for documentation
- Update README.md
- Update README.md
- Added real-time updating
- Working version
- Merge branch 'frontend_dev'
- fixed uncommenting by accident
- Connected the frontend with backend
- added home page and fixed some styling
- renamed files
- minor changes
- fe looks aight. just need to add groundtruth somewhere
- kind of what I was thinking
- use markdown for doc updates
- decent looking shit for fe
- getting updates on the frontend works.
- Merge branch 'update-docs'

## Key source files (fetched from GitHub, selected and truncated for size)

### possibleUpdates.md

```markdown
## Crons
https://github.com/Hassanmushtaq524/sentry-javascript/blob/develop/packages/node/src/cron/cron.ts
BEFORE
```js
async function monitoredTick(context: unknown, onComplete?: unknown): Promise<void> {
      const checkInId = Sentry.captureCheckIn({
  monitorSlug: monitorSlug,
  status: 'in_progress',
});
```
AFTER
```js
async function monitoredTick(context: unknown, onComplete?: unknown): Promise<void> {
  return withMonitor(
    monitorSlug,
    async () => {
      try {
        await onTick(context, onComplete);
        Sentry.captureCheckIn({
          monitorSlug,
          status: 'ok',
        });
      } catch (e) {
        captureException(e);
        Sentry.captureCheckIn({
          monitorSlug,
          status: 'error',
        });
        throw e;
      }
    },
    {
      schedule: { type: 'crontab', value: cronString },
      timezone: timeZone || undefined,
    }
  );
}```




## Feedback Widget
https://github.com/Hassanmushtaq524/sentry-javascript/blob/ea9200c078a539d29078a67cc33d39a8274db01b/packages/feedback/src/core/createMainStyles.ts
```js
const GREEN = 'rgba(34, 139, 34, 1)';

const DEFAULT_CUSTOM: InternalTheme = {
  foreground: '#1f2937', // dark navy
  background: '#f9fafb', // off-white
  accentForeground: '#ffffff', // white
  accentBackground: GREEN, // custom green color
  successColor: '#10b981', // light green
  errorColor: '#ef4444', // light red
  border: '2px solid rgba(31, 41, 55, 0.2)', // darker border
  boxShadow: '0px 6px 30px rgba(0, 0, 0, 0.15)', // stronger shadow
  outline: '2px auto var(--accent-background)',
  interactiveFilter: 'brightness(98%)',
};
```



## Session Replay (Add Framerate)
https://github.com/Hassanmushtaq524/sentry-javascript/blob/develop/packages/replay-canvas/src/canvas.ts
```js
interface ReplayCanvasOptions {
  enableManualSnapshot?: boolean;
  maxCanvasSize?: [width: number, height: number];
  quality: 'low' | 'medium' | 'high';
  frameRate?: number;  // Add frame rate control
}```
```js
const manager = new CanvasManager({
  ...getCanvasManagerOptions,
  enableManualSnapshot,
  maxCanvasSize,
  frameRate: options.frameRate || 2,  // Default frame rate is 2fps
});
```
```

### requirements.txt

```
reflex>=0.6.0a
reflex-chakra

```

### rxconfig.py

```python
import reflex as rx

config = rx.Config(
    app_name="project",
)
```

### project/__init__.py

```python
"""Base template for Reflex."""

```

### project/state.py

```python
import reflex as rx

class State(rx.State):
    pass


```

### project/project.py

```python
import reflex as rx
from project.pages import home, changes
from project.styles.styles import custom_theme
from backend.api import app as fastapi_app 
from dotenv import load_dotenv

style = {
    "background-color": "white"
}

app = rx.App(style=style, theme=custom_theme)

app.api = fastapi_app

if __name__ == "__main__":
    app.run()
```

### assets/chakra_color_mode_provider.js

```javascript
import { useColorMode as chakraUseColorMode } from "@chakra-ui/react";
import { useTheme } from "next-themes";
import { useEffect, useState } from "react";
import { ColorModeContext, defaultColorMode } from "/utils/context.js";

export default function ChakraColorModeProvider({ children }) {
  const { theme, resolvedTheme, setTheme } = useTheme();
  const { colorMode, toggleColorMode } = chakraUseColorMode();
  const [resolvedColorMode, setResolvedColorMode] = useState(colorMode);

  useEffect(() => {
    if (colorMode != resolvedTheme) {
      toggleColorMode();
    }
    setResolvedColorMode(resolvedTheme);
  }, [theme, resolvedTheme]);

  const rawColorMode = colorMode;
  const setColorMode = (mode) => {
    const allowedModes = ["light", "light", "light"];
    if (!allowedModes.includes(mode)) {
      console.error(
        `Invalid color mode "${mode}". Defaulting to "${defaultColorMode}".`
      );
      mode = defaultColorMode;
    }
    setTheme(mode);
  };
  return (
    <ColorModeContext.Provider
      value={{ rawColorMode, resolvedColorMode, toggleColorMode, setColorMode }}
    >
      {children}
    </ColorModeContext.Provider>
  );
}

```

### backend/api.py

```python
# api.py
from fastapi import FastAPI, Request
from backend.utils.utils import generate_code_description, update_docs, find_most_similar_doc
import requests
import json

# Specify the path to the .env file
app = FastAPI()
UPDATES_FILE = "updates.json"


"""
Endpoints
---------
    GET /api/: testing
    POST /api/webhook/: webhook listener (processes input and gets description of code,
    which is then queried using chroma, which will return relevant documentation,
    we can store the relevant documentation using JSON token, and use that on frontend to retrieve
    the suggested changes
    GET /api/changes/: returns suggested changes

"""


# Webhook endpoint to listen for POST requests
@app.post("/api/webhook")
async def handle_webhook(request: Request):
    payload = await request.json()
    if payload['ref'] in ["refs/heads/main", "refs/heads/master", "refs/heads/develop"]:
        owner = payload['repository']['owner']['name']
        name = payload['repository']['name']
        # Get diffs and filenames to pass to Groq
        code_diffs =  get_commit_details(owner, name, payload['after'])
        # Make a code description with Groq
        code_desc = generate_code_description(code_diffs)
        # Query Chroma for most similar doc
        closest_match = find_most_similar_doc(code_desc)
        # Now update docs with Groq.
        updated_docs = update_docs(closest_match, code_diffs)
        
        summary = {
            "commit_id": payload['after'],
            "commit_message": payload['head_commit']['message'],
            "relevant_doc": closest_match['metadata']['title'],
            "doc_url": closest_match['metadata']['url'],
            "code_summary": code_desc,
            "doc_updates": updated_docs
        }
        # Read existing updates
        try:
            with open(UPDATES_FILE, 'r') as f:
                updates = json.load(f)
                if not isinstance(updates, list):
                    updates = [updates] if updates else []
        except (FileNotFoundError, json.JSONDecodeError):
            updates = []
        
        # Append new update
        updates.append(summary)
        
        # Keep only the last 10 updates (or adjust as needed)
        updates = updates[-10:]
        
        # Write updated list back to file
        with open(UPDATES_FILE, 'w') as f:
            json.dump(updates, f)
        
        return {"status": "success", "message": "Updates processed and stored"}
    
@app.get("/api/recent-updates")
async def get_recent_updates():
    try:
        with open(UPDATES_FILE, 'r') as f:
            updates = json.load(f)
        return updates
    except FileNotFoundError:
        return {"status": "no updates", "message": "No recent updates found"}


def get_commit_details(owner: str, repo: str, commit_sha: str):
    """
    Returns the details for files changed in a specific commit
    """
    # Construct the Commit API URL
    url = f"https://api.github.com/repos/{owner}/{repo}/commits/{commit_sha}"
    
    # Make the GET request
    response = requests.get(url)
    if response.status_code == 200:
        commit_data = response.json()
        files_data = commit_data['files']
        
        file_changes = []
        for file in files_data:
            file_changes.append({
                'filename': file['filename'],
                'patch': file.get('patch', '')
            })
        
        return file_changes
    else:
        print(f"Error fetching commit details: {response.status_code} {response.text}")
        return None





```

### backend/test.py

```python
from transformers import LEDTokenizer, LEDForConditionalGeneration
from sentence_transformers import SentenceTransformer
import chromadb

# Load the LED model and tokenizer from Hugging Face
tokenizer = LEDTokenizer.from_pretrained("allenai/led-base-16384")
model = LEDForConditionalGeneration.from_pretrained("allenai/led-base-16384")

# Load the E5-Small embedding model from Hugging Face
embedding_model = SentenceTransformer('intfloat/e5-small')

# Initialize Chroma client and create a collection
client = chromadb.Client()
collection = client.create_collection(name="doc_summaries")

# Function to generate embeddings using E5-Small model
def generate_embedding(text):
    return embedding_model.encode(text, convert_to_tensor=True)

# List of file paths
docs = ["file_path_1", "file_path_2", "file_path_3"]  # Replace with actual file paths

for doc_path in docs:
    # Read the document
    with open(doc_path, "r") as f:
        document_text = f.read()

    # Step 1: Summarize the document using LED
    inputs = tokenizer(document_text, return_tensors="pt", max_length=16384, truncation=True)
    summary_ids = model.generate(inputs["input_ids"], max_length=512, num_beams=4, early_stopping=True)
    summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)

    # Step 2: Generate embedding for the summary using E5-Small
    embedding = generate_embedding(summary)

    # Step 3: Add the summary, embedding, and doc ID to ChromaDB
    collection.add(
        embeddings=[embedding],
        metadatas=[{"doc_id": doc_path}],
        documents=[summary]
    )

    print(f"Document {doc_path} summarized and added to ChromaDB.")

print("All documents summarized and added to ChromaDB.")

code_description = """
This function handles user authentication by verifying the provided credentials 
against the database and generating an authentication token if the credentials are valid.
"""

# Step 3: Create an embedding for the code description using the same embedding model
code_embedding = generate_embedding(code_description)


# Step 4: Locate the doc index with the highest cosine similarity to the code
def find_most_similar_doc(embedding, collection):
    # Perform a cosine similarity search in ChromaDB for the most relevant document
    results = collection.query(
        query_embeddings=[embedding],
        n_results=1  # Get the top result
    )

    # Retrieve the most similar document and its metadata (e.g., doc_id)
    most_similar_doc = results["documents"][0]
    most_similar_doc_id = results["metadatas"][0]["doc_id"]

    return most_similar_doc, most_similar_doc_id


most_similar_doc, most_similar_doc_id = find_most_similar_doc(code_embedding, collection)


# Step 5: Pass the most similar doc and the code description into GPT
def generate_new_doc_with_gpt(most_similar_doc, code_description):
    prompt = f"""
    You are given a document and a code description.

    The document is:
    {most_similar_doc}

    The code description is:
    {code_description}

    Based on the provided document and code description, generate a new document that incorporates both the information from the document and details from the code description.
    """

    # Whatever chat model
    response = openai.Completion.create(
        model="gpt-4",  # or "gpt-3.5-turbo" depending on your subscription
        prompt=prompt,
        max_tokens=1000  # Adjust based on document size
    )

    new_document = response.choices[0].text.strip()
    return new_document


# Generate the new document using GPT
new_doc = generate_new_doc_with_gpt(most_similar_doc, code_description)

# Output the result
print(f"Generated new document based on doc {most_similar_doc_id}:")
print(new_doc)
```

### project/pages/__init__.py

```python
from .changes import changes
from .home import home
```

[7 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]