# Project export: Sourcerer This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: An AI tutor that finds its own sources, debates itself, and makes the debate visible. - Devpost: https://devpost.com/software/sourcerer-2x9trg - GitHub: https://github.com/binoygeorge97/edu_blog - Demo: https://edubot-swart.vercel.app/ - Video: https://www.youtube.com/embed/YloaFLpoUEE?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — David Abell (20 commits), Claude Sonnet 4.6 (12 commits), phd-studies (3 commits), JakobBuehler (1 commits) ## Devpost submission (written by the team) ### Inspiration AI can already teach almost anything, but it can also hallucinate—and beginners often don’t know when an answer is wrong. We noticed that people trust learning on platforms like YouTube, blogs, or forums not because they are perfect, but because knowledge there is challenged, corrected, and discussed. AI chats today are isolated. There’s no visible debate, no peer review, and no sourcing. We built Sourcerer to add that missing trust layer to AI learning—combining critique, sourcing, and transparency into one system. ### What it does Sourcerer is an AI learning platform that transforms tutoring conversations into sourced, reviewable study posts. Users learn through an AI tutor chat. The conversation is turned into a structured study post. AI reviewer agents critique the content: Skeptic AI challenges weak reasoning Fact-Checker AI flags hallucinations Beginner AI asks clarifying questions Explainer AI improves clarity Consensus AI summarizes trust and uncertainty Skeptic AI challenges weak reasoning Fact-Checker AI flags hallucinations Beginner AI asks clarifying questions Explainer AI improves clarity Consensus AI summarizes trust and uncertainty A browser-grounded verifier checks important claims and attaches sources. A trust score shows which parts are reliable or uncertain. Users can view everything in: Thread View (Reddit-style comments) Visual Review View (AI reviewers attached to exact paragraphs) Thread View (Reddit-style comments) Visual Review View (AI reviewers attached to exact paragraphs) Instead of trusting one answer, learners can see knowledge being challenged, sourced, and improved in real time. ### How we built it We built Sourcerer as a multi-agent AI system with browser-grounded verification and an interactive frontend. Backend (Python + FastAPI) FastAPI + Uvicorn for a clean REST API (/ask, /chat, /convert, /reply) A central run_pipeline() function orchestrates all agents: Generator (initial answer) Critic agents (multi-perspective review using Claude Haiku) Verifier agent (browser grounding with Stagehand + Browserbase) Teacher (final synthesis using Claude Sonnet) Consensus (trust summary and scoring) A central run_pipeline() function orchestrates all agents: Generator (initial answer) Critic agents (multi-perspective review using Claude Haiku) Verifier agent (browser grounding with Stagehand + Browserbase) Teacher (final synthesis using Claude Sonnet) Consensus (trust summary and scoring) Anthropic SDK (Claude) powers all reasoning: Haiku for lightweight critique and evaluation Sonnet for generation, verification reasoning, and final teaching output Anthropic SDK (Claude) powers all reasoning: Haiku for lightweight critique and evaluation Sonnet for generation, verification reasoning, and final teaching output Stagehand + Browserbase enable live web browsing so the verifier can fetch and attach real evidence to claims. Stagehand + Browserbase enable live web browsing so the verifier can fetch and attach real evidence to claims. Arize Phoenix + OpenTelemetry provide traceability across the pipeline, letting us see where claims were introduced, challenged, or corrected. Arize Phoenix + OpenTelemetry provide traceability across the pipeline, letting us see where claims were introduced, challenged, or corrected. Pydantic + dotenv manage data models and environment configuration. Pydantic + dotenv manage data models and environment configuration. Frontend (React + TypeScript) React + Vite + TypeScript for fast, modular UI development Tailwind CSS for clean, responsive styling Lucide React for icons Built two key interfaces: Thread View (Reddit-style discussion) Visual Review View (AI reviewers attached to paragraphs) Thread View (Reddit-style discussion) Visual Review View (AI reviewers attached to paragraphs) Integrations Anthropic (Claude) for all core intelligence Browserbase + Stagehand for browser-grounded verification Arize Phoenix for observability and evaluation Fetch.ai (uAgents / ASI:One) to optionally expose Sourcerer as a discoverable AI tutor agent We focused on a polished vertical slice that clearly demonstrates the product experience rather than building a full production system. Deployment Frontend: Deployed on Vercel for fast, globally distributed hosting of our React application. Backend: Deployed on Render, running our FastAPI-based multi-agent pipeline. This combination allowed us to move quickly during the hackathon while maintaining a responsive UI and a reliable backend service for AI orchestration. ### Challenges we ran into Designing useful AI critique: Multiple agents can easily overwhelm users. We had to carefully define roles so each comment added distinct value. Linking comments to exact content: Attaching feedback to specific paragraphs required structuring outputs beyond typical LLM responses. Balancing grounding vs speed: Browser-based verification is powerful but slow, so we limited it to key claims for the demo. Keeping the demo polished: We had to balance backend complexity with a UI that judges could immediately understand. Time constraints: Building a multi-agent system, frontend experience, and integrations in 24 hours required aggressive prioritization and fallback strategies. ### Accomplishments we're proud of Built a system where AI answers are challenged, sourced, and defended, not just generated Created a visual review experience where AI agents interact with exact parts of content Integrated browser-grounded verification into the learning flow Designed a flexible architecture that supports multiple AI providers Delivered a compelling demo showing how a hallucination or overstatement is flagged and improved Made AI learning feel interactive, transparent, and collaborative ### What we learned AI is incredibly helpful for learning, but trust and transparency are just as important as accuracy Multi-agent systems are only effective when outputs are structured and interpretable Grounding answers with external sources significantly increases user confidence UI/UX matters deeply—how you show uncertainty can be as important as detecting it The best demos are not just technically complex—they are intuitive and tell a clear story ### What's next We see Sourcerer evolving into a trust layer for AI-powered education. Next steps include: Persistent study posts and a public knowledge-sharing system Real human community comments and moderation More robust and scalable browser-grounded verification Improved confidence scoring and claim-level validation Full support for multiple AI providers and user-defined reviewers Classroom and collaborative learning features A growing library of sourced, reviewed study content Long-term, we want Sourcerer to help anyone—from beginners to advanced learners—learn with AI without trusting it blindly. ## README (from the GitHub repository) # Sourcerer > An AI tutor that finds its own sources, debates itself, and makes the debate visible. ## The problem People use AI to learn, but beginners can't tell when AI is wrong. Unlike YouTube comments, Reddit threads, or classrooms, AI answers have no correction layer — no debate, no peer review, no trust signals. A single model call is confidently wrong in ways that compound as the learner builds on bad foundations. ## Our solution Turn the AI answer into a **structured blog post with agent comments**. The learner asks a question. Behind the scenes, a multi-agent pipeline drafts an answer, red-teams it with differentiated critic roles, and fetches live web evidence to verify each claim. Then — instead of hiding all of that deliberation — we surface it: - The **Teacher's final answer** is the blog post - Each **agent contribution** (Generator draft, Critic flags, Verifier citations) appears as a comment card with a role badge - The learner can **reply to any agent comment** to ask a follow-up question — that reply re-enters the pipeline with the commenting agent's reasoning as context This turns the multi-agent debate into the product. Learners see which claims survived scrutiny, which didn't, and why — and they can interrogate any step of the reasoning directly. ## Architecture ``` Question → Generator drafts a first answer (Sonnet) → Critics decompose into atomic claims, red-team in parallel (Haiku × N) → Verifier fetches web evidence per flagged claim via Browserbase + Stagehand (Sonnet) → Confidence multi-samples contested claims; semantic disagreement → low confidence → Teacher synthesizes, drops/hedges unsupported claims, adapts to learning mode (Sonnet) → Deliver PipelineResult: answer + confidence + agent comment thread ``` ### The data model ```python @dataclass class AgentComment: agent: Literal["generator", "critic", "verifier"] role: str # "Skeptical Fact-Checker", "Domain Expert", "Verifier", … content: str claim: str | None # the specific claim this comment addresses verdict: Literal["supports", "refutes", "unclear"] | None url: str | None # verifier citation @dataclass class PipelineResult: answer: str # the "post" comments: list[AgentComment] # the "comments" confidence: float confidence_level: Literal["high", "medium", "low"] ``` A second entry point `reply_to_comment(comment, followup)` re-enters the pipeline with the original question plus the commenting agent's context injected, so follow-up answers are grounded in that specific agent's perspective. ### Trust signals shown to the learner - Verified claims (green) — Verifier found supporting evidence - Disputed claims (amber) — critics flagged, verifier returned "unclear" - Refuted claims (red) — verifier evidence contradicts the draft - Confidence badge (high / medium / low) on the overall answer - Citations on each Verifier comment ## The accuracy proof We ran a 30-question factual eval against topics where LLMs commonly hallucinate. Answers were scored by a Haiku judge on correctness against known answers. | Pipeline | Factuality score | |---|---| | Single Sonnet call (baseline) | — | | Full pipeline (critics + verifier + confidence) | — | *(Numbers populated at eval milestone — see the Phoenix experiment linked below.)* ## Prize integrations **Anthropic** — built entirely with Claude Code and Claude models. Haiku for the high-volume critic swarm and eval judges; Sonnet for generation, verification reasoning, and synthesis. Prompt caching on all shared system prompts. Batch API for eval generation runs. **Browserbase + Stagehand** — the Verifier agent uses Stagehand on Browserbase cloud browsers to fetch live evidence per flagged claim, extracted via a narrow Pydantic schema so only relevant content enters the model context. **Arize Phoenix** — every pipeline run is a single span tree (Generator → Critics → Verifier → Teacher as child spans). Eval answers generated once, stored as a Phoenix dataset, judges re-run over stored answers as needed. A Phoenix trace surfaced the verifier grabbing off-topic pages; tightening the extraction schema moved the factuality score — before/after captured. **Fetch.ai** — `run_pipeline` wrapped as a Chat Protocol uAgent (Mailbox agent, full deps). Registered on Agentverse, discoverable on ASI:One. Any agent or user on the network can ask Sourcerer a question and get a fact-checked, cited answer. Claude stays the brain; ASI:One is the caller. ## Running locally ```bash cp .env.example .env # add ANTHROPIC_API_KEY (+ BROWSERBASE keys for Phase 2+) pip install -r requirements.txt phoenix serve # terminal 1 — observability at http://localhost:6006 streamlit run ui/streamlit_app.py # terminal 2 — UI # or: uvicorn app.api:app --reload --port 8000 ``` ## Repo layout ``` app/ agents/ generator, critics, verifier, teacher models.py AgentComment + PipelineResult dataclasses pipeline.py run_pipeline() + reply_to_comment() confidence.py multi-sample scoring grounding/ Browserbase + Stagehand client telemetry.py Phoenix auto-instrumentation api.py FastAPI (POST /ask, POST /reply) agent.py Fetch.ai uAgent wrapper (Phase 6) eval/ datasets/ qa_30.jsonl + qa_smoke.jsonl (5 questions) generate.py batch eval generation experiment.py baseline vs pipeline comparison ui/ streamlit_app.py blog-post + comment thread UI with reply boxes ``` ## Detected evidence (automated analysis) Indexed codebase: 34 recognized source files, 150 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Streamlit (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers ## Codebase structure (from repository index) ### Files (43 of 43) ``` .env.example .gitignore AGENT_README.md app/__init__.py app/agent.py app/agents/__init__.py app/agents/blogger.py app/agents/critics.py app/agents/generator.py app/agents/teacher.py app/agents/tutor.py app/agents/verifier.py app/api.py app/confidence.py app/grounding/__init__.py app/grounding/browser.py app/models.py app/pipeline.py app/telemetry.py CLAUDE.md eval/datasets/qa_30.jsonl eval/datasets/qa_smoke.jsonl eval/experiment.py eval/generate.py JUDGING.md README.md render.yaml requirements-api.txt requirements.txt scripts/smoke_flow.py SETUP.md ui/streamlit_app.py web/.gitignore web/index.html web/package.json web/README.md web/src/api.ts web/src/App.tsx web/src/CursorEffects.tsx web/src/main.tsx web/src/styles.css web/tsconfig.json web/vite.config.ts ``` ### Dependencies - requirements.txt: anthropic@>=0.49.0, arize-phoenix[otel]@>=4.0.0, browserbase@>=1.0.0, fastapi@>=0.111.0, opentelemetry-exporter-otlp@>=1.24.0, opentelemetry-sdk@>=1.24.0, playwright@>=1.40.0, python-dotenv@>=1.0.0, streamlit@>=1.35.0, uagents@>=0.25.0, uvicorn[standard]@>=0.29.0 - web/package.json: @tailwindcss/vite@^4.0.0, @types/react@^18.3.12, @types/react-dom@^18.3.1, @vitejs/plugin-react@^4.3.4, lucide-react@^0.468.0, react@^18.3.1, react-dom@^18.3.1, react-markdown@^10.1.0, remark-gfm@^4.0.1, tailwindcss@^4.0.0, typescript@^5.6.3, vite@^6.0.7 ### Recent commits (newest first) - Update live demo URL to edubot-swart.vercel.app - Remove ARIZE_PITCH.md from repo - Use requirements-api.txt on Render — drops uagents and streamlit to fix OOM - Add ARIZE_PITCH.md — Arize-specific integration pitch - Always verify at least one claim so Browserbase runs on every pipeline call - Add computer+magnifying glass avatar for verifier agent - Fix critic JSON extraction when model adds trailing text after the JSON object - Log eval results to Phoenix as spans (eval.experiment + eval.question per question) - Add 20s timeout to Browserbase fetches; revert warmup (not needed on paid tier) - Add remark-gfm for table rendering in markdown - Add JUDGING.md — project overview for hackathon judges - Render markdown in chat and agent cards - Add browserbase and playwright to requirements (dropped when stagehand-py was removed) - Fix CORS to allow Vercel production domains - Add Render deployment config and fix production API URL - Add PNG avatars and UI polish from edu_blog/binoy - Update SETUP.md: React frontend commands and PHOENIX_API_KEY env var - Merge Hogwarts-themed UI improvements from edu_blog/binoy - added better ui (hogwartz themed) - Merge React web frontend and UI improvements ## Key source files (fetched from GitHub, selected and truncated for size) ### AGENT_README.md ```markdown # Sourcerer — Verified AI Tutor An AI tutor that verifies its own answers using multi-agent debate and web evidence grounding. ## What it does Send any factual question and Sourcerer will: 1. Draft an answer using Claude (Sonnet) 2. Decompose it into atomic claims and red-team each with specialist critics 3. Fetch web evidence for disputed claims via Browserbase 4. Synthesize a final answer that drops or hedges unsupported claims 5. Score confidence based on multi-sample semantic disagreement ## How to use Send a plain-text question as a ChatMessage. You'll receive an acknowledgement immediately, then a verified answer within 15–60 seconds depending on question complexity. ## Powered by Claude (Anthropic) for reasoning, Stagehand/Browserbase for web grounding, Arize Phoenix for observability. ``` ### JUDGING.md ```markdown # Sourcerer — Judging Overview > AI tutoring that proves its own answers. Chat with a tutor, then convert the conversation into a fact-checked, source-backed study post reviewed by multiple AI agents. **Live demo:** [edubot-swart.vercel.app](https://edubot-swart.vercel.app) (frontend) · backend on Render **Repo:** github.com/davidmaking/education-blog-chatbot-unfinished --- ## The problem AI can teach almost anything. It can also hallucinate confidently. Learners — especially beginners — have no way to know when to trust the answer. Every AI chat today is a closed loop with no peer review, no citations, no visible disagreement. We don't just warn users that AI can be wrong. We show them the receipts. --- ## What it does **Phase 1 — Study.** The learner chats with an AI tutor. Fast, conversational, no interruptions. Nothing is fact-checked yet. **Phase 2 — Convert.** The learner clicks "Convert to Verifiable Blog Post." The system: 1. Synthesises the conversation into a structured post (Blogger agent) 2. Decomposes every paragraph into atomic claims (Critics — 3 differentiated roles, parallel) 3. Fetches web evidence for disputed claims (Verifier — Browserbase) 4. Scores confidence from semantic disagreement across multi-sample critic runs 5. Colour-codes each paragraph: **mint = verified · amber = disputed · rose = hallucination** 6. Surfaces every agent's reasoning as a comment card the learner can reply to The learner sees not just the answer but the argument behind it — and can challenge any agent directly. --- ## Architecture ``` Learner question → Tutor agent fast multi-turn chat (Sonnet) — no checking → [Convert clicked] → Blogger agent conversation → structured paragraphs (Sonnet) → Critic swarm 3 roles in parallel: Skeptical Fact-Checker, Domain Expert, Devil's Advocate — each decomposes + red-teams claims (Haiku) → Verifier agent Browserbase → DuckDuckGo → Claude extracts verdict + quote + URL (Haiku) → Confidence pass multi-sample semantic disagreement scoring → Deliver BlogPostResult: paragraphs with status + anchored agent comments ``` The same accuracy engine (`_review()`) also powers the `/ask` endpoint used by the eval harness and the Fetch.ai agent — one pipeline, three entry points. --- ## Main tools ### Claude — Anthropic API Every agent runs on Claude. Model routing follows cost discipline: | Role | Model | |---|---| | Tutor, Blogger, Generator, Teacher, Verifier reasoning | `claude-sonnet-4-6` | | Critic swarm, eval judges, evidence extraction | `claude-haiku-4-5-20251001` | Prompt caching is on for all shared system prompts. The eval generation uses the Batch API (50% off). Opus is never in any automated path. **Built entirely with Claude Code** — every file, every commit in this repo. ### Browserbase The Verifier opens a Browserbase remote browser session, navigates to DuckDuckGo, extracts page text, and passes it t [truncated — 4664 more characters] ``` ### requirements.txt ``` anthropic>=0.49.0 fastapi>=0.111.0 uvicorn[standard]>=0.29.0 streamlit>=1.35.0 arize-phoenix[otel]>=4.0.0 opentelemetry-sdk>=1.24.0 opentelemetry-exporter-otlp>=1.24.0 python-dotenv>=1.0.0 browserbase>=1.0.0 playwright>=1.40.0 uagents>=0.25.0 ``` ### web/package.json ``` { "name": "sourcerer-web", "private": true, "version": "0.1.0", "type": "module", "scripts": { "dev": "vite", "build": "vite build", "preview": "vite preview", "typecheck": "tsc --noEmit" }, "dependencies": { "lucide-react": "^0.468.0", "react": "^18.3.1", "react-dom": "^18.3.1", "react-markdown": "^10.1.0", "remark-gfm": "^4.0.1" }, "devDependencies": { "@tailwindcss/vite": "^4.0.0", "@types/react": "^18.3.12", "@types/react-dom": "^18.3.1", "@vitejs/plugin-react": "^4.3.4", "tailwindcss": "^4.0.0", "typescript": "^5.6.3", "vite": "^6.0.7" }, "allowScripts": { "esbuild@0.25.12": true, "fsevents@2.3.3": true } } ``` ### web/src/main.tsx ```typescript import React from "react"; import ReactDOM from "react-dom/client"; import App from "./App"; import "./styles.css"; ReactDOM.createRoot(document.getElementById("root")!).render( ); ``` ### web/src/App.tsx ```typescript import { useEffect, useRef, useState } from "react"; import ReactMarkdown from "react-markdown"; import remarkGfm from "remark-gfm"; import { Sparkles, ShieldCheck, Search, GraduationCap, ExternalLink, ArrowUp, Send, Plus, Loader2, } from "lucide-react"; import { chat, convert, replyToComment, type AgentComment, type BlogPostResult, type ChatTurn, type ConfidenceLevel, type ParagraphStatus, type Paragraph as ParagraphT, } from "./api"; import CursorEffects from "./CursorEffects"; function Prose({ children, className = "" }: { children: string; className?: string }) { return (

{children}

, strong: ({ children }) => {children}, em: ({ children }) => {children}, h1: ({ children }) =>

{children}

, h2: ({ children }) =>

{children}

, h3: ({ children }) =>

{children}

, ul: ({ children }) => , ol: ({ children }) =>
    {children}
, li: ({ children }) =>
  • {children}
    • , code: ({ children }) => {children}, hr: () =>
    , blockquote: ({ children }) =>
    {children}
    , table: ({ children }) =>
    {children}
    , thead: ({ children }) => {children}, th: ({ children }) => {children}, td: ({ children }) => {children}, }} > {children}     ); } type AppState = "home" | "chat" | "loading" | "workspace"; const SUGGESTIONS = [ "Explain how SSMs compare to Transformers", "Why is the sky blue?", "How do vaccines train the immune system?", "What caused the 2008 financial crisis?", ]; const LOADING_STEPS = [ "Drafting the study post…", "Domain Expert reviewing…", "Skeptical Critic checking claims…", "Verifier fetching web sources…", "Synthesizing trust signals…", ]; // ── small UI helpers ───────────────────────────────────────────────────────────── const STATUS_PARA: Record = { verified: "border-l-4 border-verified bg-verified/10", disputed: "border-l-4 border-disputed bg-disputed/10", hallucination: "border-l-4 border-hallucination bg-hallucination/10", neutral: "border-l-4 border-transparent", }; const VERDICT_BADGE: Record = { supports: "bg-verified/15 text-verified", refutes: "bg-hallucination/15 text-hallucination", unclear: "bg-disputed/15 text-disputed", }; const CONFIDENCE_STYLE: Record = { high: "bg-verified/15 text-verified", medium: "bg-disputed/15 text-disputed", low: "bg-hallucination/15 text-hallucination", }; function personaColor(role: string): string { const r = role.toLowerCase(); if (r.includes("verif")) return "var(--color-factcheck)"; if (r.includes("fact") || r.includes("skeptic")) return "var(--color-skeptic)"; if (r.includes("domain") || r.includes("expert")) return "var(--color-explainer)"; if (r.includes("devil") || r.includes("advocate")) return "var(--color-defender)"; return "var(--color-consensus)"; } /** Maps agent role keywords → avatar image path in /avatars/. */ function personaAvatar(agent: string, role: string): string { const r = role.toLowerCase(); if (agent === "verifier" || r.includes("verif")) return "/avatars/verifier.svg"; if (r.includes("fact") || r.includes("skeptic")) return "/avatars/skeptic.png"; if (r.includes("domain") || r.includes("expert")) return "/avatars/expert.png"; if (r.includes("devil") || r.includes("advocate")) return "/avatars/advocate.png"; return "/avatars/generator.png"; } function TypingDots() { return ( {[0, 1, 2].map((i) => ( ))} ); } function AgentIcon({ agent, role }: { agent: string; role: string }) { const r = role.toLowerCase(); const cls = "h-4 w-4"; if (agent === "verifier") return ; if (r.includes("fact") || r.includes("skeptic")) return ; if (r.includes("domain") || r.includes("expert")) return ; return ; } // ── Fireflies — ambient floating particles ────────────────────────────────────── function Fireflies({ count = 12 }: { count?: number }) { return (