# Project export: LegalLingua

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 10.0
- Tagline: It is already hard enough navigating the real world, legal documents make it even harder. We aim to ease the lives of people who are not skilled at English comprehension by leveraging LLMs.
- Devpost: https://devpost.com/software/legallingua
- GitHub: https://github.com/Alex-RV/LegalLingua
- Demo: https://legal-lingua.vercel.app/
- Team: 3 GitHub contributor(s) — Alex-RV (42 commits), jjestrada3 (21 commits), sushiphie (13 commits)

## Devpost submission (written by the team)

### Inspiration

We all came together to solve a problem that people struggle with on a regular basis. We are we were immigrants who hail from different places in the world where English is not a prevalent language! We thought to ourselves, what is someone that is difficult and is often. detriment to various communities? We discovered that we could leverage our knowledge to help communities that either are drawn into confusion or are taken advantage of through legal documentation that is often difficult to understand, even for those who are fluent in English.

### What it does

We leveraged large language models, ORC, and base 64 to scan documents, summarize them, and translate them to a person's language. We do this summarization and translation by implementing Together.ai's api and fine tuning models using our datasets.

### How we built it

We built it by splitting the team into various roles - Juan: Front-end, Aleks: Front-end / Back-end, Randolf: APIs, ML/AI, and Xiang (UI/UX / Front-end). Our technology/ Design stack: Tailwind.css, Python, Javascript, Together.ai API, Google Vision API, Typescript, Javascript, Next.js, Libraries for transforming PDFs.

### Challenges we ran into

We all ran into tough challenges as we struggled to connect components together which include: - Working with Pdf documents: searching for the best libraries to help convert documents to images - Connecting to google vision api after obtaining said images - Working with timeouts with displaying words using visual effects in the front-end - Loading effects, creating try-catch statements using logical processing - Designing the website using sigma to have a simple but elegant UI/UX - Congregating data to use in together.ai's LLMs (LLaMA - for summarization, llama 2 - for key points / insights) using python scripting - Working with Together.ai's prompting to ensure the best answer quality for the queries needed to operate our platforms

## README (from the GitHub repository)

This is a [Next.js](https://nextjs.org/) project bootstrapped with [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app).

## Getting Started

First, run the development server:

```bash
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
```

Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.

You can start editing the page by modifying `pages/index.tsx`. The page auto-updates as you edit the file.

[API routes](https://nextjs.org/docs/api-routes/introduction) can be accessed on [http://localhost:3000/api/hello](http://localhost:3000/api/hello). This endpoint can be edited in `pages/api/hello.ts`.

The `pages/api` directory is mapped to `/api/*`. Files in this directory are treated as [API routes](https://nextjs.org/docs/api-routes/introduction) instead of React pages.

This project uses [`next/font`](https://nextjs.org/docs/basic-features/font-optimization) to automatically optimize and load Inter, a custom Google Font.

## Learn More

To learn more about Next.js, take a look at the following resources:

- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.

You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js/) - your feedback and contributions are welcome!

## Deploy on Vercel

The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.

Check out our [Next.js deployment documentation](https://nextjs.org/docs/deployment) for more details.


## Detected evidence (automated analysis)

Indexed codebase: 39 recognized source files, 41 KB.
- CSS (language) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code

## Codebase structure (from repository index)

### Files (44 of 44)

```
.env
.eslintrc.json
.gitignore
lib/fetches.ts
lib/interfaces.ts
lib/utils.ts
next.config.js
package.json
postcss.config.js
README.md
src/components/ChatDisplay.jsx
src/components/Hero.jsx
src/components/LanguageSelector.jsx
src/components/Loading.jsx
src/components/Navbar.jsx
src/NexUIComponents/Acordion.jsx
src/NexUIComponents/Button.jsx
src/NexUIComponents/Footer.jsx
src/NexUIComponents/Image.jsx
src/NexUIComponents/input.jsx
src/NexUIComponents/InputFIle.jsx
src/NexUIComponents/Loading.jsx
src/NexUIComponents/Navbar.jsx
src/NexUIComponents/ProgressBar.jsx
src/NexUIComponents/Select.jsx
src/NexUIComponents/Table.jsx
src/NexUIComponents/TextArea.jsx
src/NexUIComponents/TextAreaDinamic.jsx
src/NexUIComponents/UserProfile.jsx
src/pages/_app.tsx
src/pages/_document.tsx
src/pages/api/pdf2text.ts
src/pages/api/upload-pdf.ts
src/pages/index.tsx
src/pages/translate.tsx
src/styles/globals.css
tailwind.config.ts
together_ai_training/api_call.py
together_ai_training/example.py
together_ai_training/PdfToJsonl.py
together_ai_training/SampleLeaseData/retrain_model.py
together_ai_training/TextToJsonl.py
tsconfig.json
types.d.ts
```

### Dependencies

- package.json: @google-cloud/vision@^4.0.2, @nextui-org/react@^2.1.13, @types/node@^20, @types/pdfjs-dist@^2.10.378, @types/react@^18, @types/react-dom@^18, @vercel/analytics@^1.1.1, autoprefixer@^10.4.16, cn@^0.1.1, eslint@^8, eslint-config-next@14.0.0, framer-motion@^10.16.4, micro@^10.0.1, next@14.0.0, pdf-lib@^1.17.1, pdf-parse@^1.1.1, pdfjs-dist@^3.11.174, postcss@^8.4.31, react@^18, react-dom@^18, react-icons@^4.11.0, tailwindcss@^3.3.5, tesseract.js@^5.0.2, typescript@^5

### Recent commits (newest first)

- Python scripting for LLMs
- [add]:queries
- Fixed build bugs
- More UI edits
- Translate page frontend edits
- UI changes to Home and About
- Merge branch 'main' of https://github.com/Alex-RV/LegalLingua
- UI changes
- [add]: text
- [add]: mobile friendly
- [add]:select drop down mobile friendly
- [fix]: model error
- [add]: stop words
- [add]:new lang
- Table edit
- Merge Xiang Juan Index
- [add]: changed bg color
- PersonalExp Table
- [add]:model choosing&minor link fix
- Minor UI changes

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "legallingua",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "@google-cloud/vision": "^4.0.2",
    "@nextui-org/react": "^2.1.13",
    "@vercel/analytics": "^1.1.1",
    "cn": "^0.1.1",
    "framer-motion": "^10.16.4",
    "micro": "^10.0.1",
    "next": "14.0.0",
    "pdf-lib": "^1.17.1",
    "pdf-parse": "^1.1.1",
    "pdfjs-dist": "^3.11.174",
    "react": "^18",
    "react-dom": "^18",
    "react-icons": "^4.11.0",
    "tesseract.js": "^5.0.2"
  },
  "devDependencies": {
    "@types/node": "^20",
    "@types/pdfjs-dist": "^2.10.378",
    "@types/react": "^18",
    "@types/react-dom": "^18",
    "autoprefixer": "^10.4.16",
    "eslint": "^8",
    "eslint-config-next": "14.0.0",
    "postcss": "^8.4.31",
    "tailwindcss": "^3.3.5",
    "typescript": "^5"
  }
}

```

### src/pages/index.tsx

```typescript
import React, { useEffect, useState } from "react";
import Head from "next/head";
import Hero from "../components/Hero";
import AccordionComponent from "../NexUIComponents/Acordion";
import TextArea from "../NexUIComponents/TextArea";
import { Link } from "@nextui-org/react";
import Table from "../NexUIComponents/Table";

export default function Home() {
  const [summary, setSummary] = useState("");
  const sentence =
    "Legalingua isn't just a project; it's a heartfelt response to the real struggles faced by immigrants.";

  useEffect(() => {
    for (let i = 0; i < sentence.length; i++) {
      setTimeout(() => {
        setSummary((prevText) => prevText + sentence[i]);
      }, i * 300);
    }
  }, []);

  return (
    <div>
      <Head>
        <title>Legalingua</title>
        <meta name="description" content="Generated Legalingua" />
      </Head>

      <Hero
        heading="Skip the jargon"
        message={
          <>
            AI tool that instantly explains your legal document to you, in your
            language of choice.
            <br />
            No more second-guessing.
          </>
        }
        redirect={
          <Link className="text-white" href="/translate#translate">
            Try LegalLingua
          </Link>
        }
      />
      <div className="grid grid-cols-1 md:grid-cols-2 gap-4 mt-8 w-full">
        <AccordionComponent />
        <div className="pr-3">
          <div className="blue-text font-bold text-center py-2 text-lg">
            LegalLingua in Action
          </div>
          <TextArea title="Summary" chatText={summary} />

          <div className="py-2">
            <div className="blue-text font-bold text-center py-2 text-lg">
              LegalLingua Stories
            </div>
            <Table />
          </div>
        </div>
      </div>
    </div>
  );
}

```

### types.d.ts

```typescript
declare module 'pdfjs-dist/build/pdf.worker.min.js';

```

### postcss.config.js

```javascript
module.exports = {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
  },
}

```

### next.config.js

```javascript
/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
}

module.exports = {
  ...nextConfig,
  images: {
    domains: [
      'images.unsplash.com',
    ]
  }
}

```

### tailwind.config.ts

```typescript
import type { Config } from 'tailwindcss'
const {nextui} = require("@nextui-org/react");
const config: Config = {
  content: [
    './src/pages/**/*.{js,ts,jsx,tsx,mdx}',
    './src/components/**/*.{js,ts,jsx,tsx,mdx}',
    './src/app/**/*.{js,ts,jsx,tsx,mdx}',
    "./node_modules/@nextui-org/theme/dist/**/*.{js,ts,jsx,tsx}",
  ],
  theme: {
    extend: {
      backgroundImage: {
        'gradient-radial': 'radial-gradient(var(--tw-gradient-stops))',
        'gradient-conic':
          'conic-gradient(from 180deg at 50% 50%, var(--tw-gradient-stops))',
      },
      colors: {
        'blue': '#015483',
        'light-blue': '#539CC2'
      },
    },
  },
  darkMode: "class",
  plugins: [nextui()],
}
export default config

```

### together_ai_training/api_call.py

```python
import requests
url = "https://legal-lingua.vercel.app/api/pdf2text"
headers = {
    "Content-Type": "application/pdf",
}

with open("Test.pdf", "rb") as f:
    response = requests.post(url, headers=headers, data=f)

print(response.text)

```

### lib/interfaces.ts

```typescript
export interface InferenceResponse {
    status: string;
    prompt: string[];
    model: string;
    model_owner: string;
    tags: Record<string, any>;
    num_returns: number;
    args: {
      model: string;
      prompt: string;
      temperature: number;
      top_p: number;
      top_k: number;
      max_tokens: number;
      repetition_penalty: number;
    };
    subjobs: any[]; 
    output: {
      choices: Array<{ text: string }>; 
      request_id: string;
    };
  }
  
```

### together_ai_training/example.py

```python
import together
from decouple import config

API_USER = config('USER')
API_KEY = config('TOGETHER_API_KEY')

print(API_USER)
print(API_KEY)

together.api_key = API_KEY
# see available models
model_list = together.Models.list()

print(f"{len(model_list)} models available")

# print the first 10 models on the menu
model_names = [model_dict['name'] for model_dict in model_list]
model_names[:10]

output = together.Complete.create(
  prompt = "<human>: What are Isaac Newton's 3 laws of physics? ", 
  model = "togethercomputer/llama-2-7b", 
  max_tokens = 256,
  temperature = 0.8,
  top_k = 60,
  top_p = 0.6,
  repetition_penalty = 1.1,
  stop = ['<human>', '\n\n']
)

# print generated text
print(output['prompt'][0]+output['output']['choices'][0]['text'])

```

### together_ai_training/TextToJsonl.py

```python
import pandas as pd
import json

# Python script that reads file input from txt into jsonl
dataframe = pd.read_json(path_or_buf='/Users/randolf/Desktop/Tutorials/togetherai/train.txt', lines=True)
dataframe = dataframe.iloc[:2000]
dataframe = dataframe.rename(columns={"report": "long", "summary": "short"})
dataframe = dataframe.drop('document_type', axis=1)
lists = dataframe.values.tolist()

json_list = []
for list in lists:
    training_prompt = f"summarize this text and give your answer between <summary></summary> tags. text: {str(list[0])} <summary> {str(list[1])} </summary>"
    json_list.append({"text": str(training_prompt)})

with open('training_set.jsonl', 'w') as outfile:
    for entry in json_list:
        json.dump(entry, outfile)
        outfile.write('\n')

```

[27 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]