LiteRAG

A small Next.js app and Python backend that build tiny LiteRAG and Chroma indexes, show on-disk size, and let you query both. LiteRAG is the focus; Chroma is included solely as a baseline for comparison. Optionally, responses are summarized by a Groq LLM. A separate Flask app demonstrates a local Ollama streaming chat UI.

What’s in here

• Next.js 14 frontend (app/) with Tailwind UI
• FastAPI backend (backend.py) exposing:
  • GET /setup — builds the LiteRAG index and a persistent Chroma collection (baseline), returns storage stats
  • GET /ask?q=... — searches both and returns results plus optional Groq LLM response
• Optional Flask SSE server (chat.py) that streams responses from a local Ollama model

Note: Chroma is only used as a familiar baseline to highlight LiteRAG’s advantages (smaller storage footprint with comparable results on this demo dataset).

Prerequisites

• Node.js 18+ and npm
• Python 3.10+
• Optional for Groq summaries: a Groq API key
• Optional for local chat: [Ollama] installed with a local model pulled (e.g. qwen7b)

Quick start

1. Install frontend deps

npm install

2. Install Python deps (add missing runtime libs used by the code)

python -m venv .venv && source .venv/bin/activate  # optional but recommended
pip install -r requirements.txt
pip install fastapi uvicorn chromadb leann groq flask

3. Start the backend (FastAPI)

uvicorn backend:app --reload --port 8000

4. Start the frontend (Next.js)

npm run dev

Open http://localhost:3000

5. In the UI

• Click “Download & Build Demo” to run GET /setup and build both indexes
• Ask a question; the app calls GET /ask?q=... and shows results for LiteRAG (primary) and Chroma (baseline)

API reference (FastAPI)

GET /setup

Builds a tiny LiteRAG index and a persistent Chroma collection (baseline) under demo_data/, then returns storage stats.

Response shape:

{
  "leann_size_kb": 1.23,
  "chroma_size_kb": 8.45,
  "storage_reduction_percent": 85.4
}

Notes

• LiteRAG index path: demo_data/leann_index.index (field name leann_* is legacy in code and refers to LiteRAG)
• Chroma data dir: demo_data/chroma_db/

GET /ask?q=...

Searches both backends and includes an optional Groq-generated answer per context.

Response shape (truncated):

{
  "query": "your question",
  "leann": {
    "results": [
      { "text": "...", "score": 0.1234, "metadata": {} }
    ],
    "groq_response": "...",
    "context": "..."
  },
  "chroma": {
    "results": {
      "documents": [["...", "..."]],
      "distances": [[0.12, 0.34]]
    },
    "groq_response": "...",
    "context": "..."
  }
}

Optional: Local Ollama streaming chat (separate app)

chat.py runs a minimal Flask server that streams tokens from a local Ollama model via SSE.

Run it (cannot share port 8000 with the FastAPI backend at the same time):

python chat.py

Endpoints

• / — simple in-browser chat UI
• /ping_ollama — health check (non-streaming)
• /stream_test — SSE smoke test
• /chat_sse?session_id=...&text=... — main streaming endpoint

Environment

• OLLAMA_HOST (default http://127.0.0.1:11434)
• MODEL_NAME (default qwen7b) — must exist in ollama list

Port note: chat.py is hardcoded to 127.0.0.1:8000. If the FastAPI backend is running, stop it first or change the port in chat.py.

Environment and configuration

Groq (optional summaries in backend.py)

• The function get_groq_response() currently contains a hardcoded API key. Replace it with an env read (e.g., os.environ.get("GROQ_API_KEY")) before committing or sharing the code.
• If Groq is not installed or the key is missing/invalid, the backend returns an error string in groq_response, and the demo still functions.

Frontend backend URL

• The UI calls http://localhost:8000. If you change the backend host/port, update the URLs in app/LiteRAGDemo.tsx (/setup, /ask).

Data locations

• LiteRAG index (legacy name in code): demo_data/leann_index.index
• Chroma DB: demo_data/chroma_db/
• Delete the demo_data/ folder to reset.

Troubleshooting

• ModuleNotFoundError for fastapi, uvicorn, chromadb, leann, groq, or flask → install the packages shown above.
• Port already in use (8000) → stop the other server or change the port (and adjust the frontend URLs if needed).
• CORS errors → backend.py already allows all origins; ensure you are calling the correct host/port.
• Groq errors in groq_response → install groq and set a valid API key; or ignore if summaries are not needed.
• LiteRAG import error → ensure the leann Python package installs successfully for your platform.

License

No license specified in this repository.

---

Made for a simple, reproducible comparison that showcases LiteRAG, with Chroma included only as a baseline.