# Project export: DebtLens This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: "Your codebase has debt you don't know about. DebtLens shows you exactly where." - Devpost: https://devpost.com/software/debtlens-6vl8p1 - GitHub: https://github.com/backSpaceTwice/DebtLens - Video: https://www.youtube.com/embed/7MkpEkOaMdg?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — backSpaceTwice (21 commits), Claude Opus 4.8 (17 commits) ## Devpost submission (written by the team) ### Inspiration Every engineer has inherited a codebase they didn't write. You open a file, it's 400 lines long, there are no tests, the last dependency update was two years ago, and there's a TODO from 2021 that says "fix this later." The challenge isn't finding problems. It's knowing which ones matter most, why they matter, and how to fix them. We built DebtLens to do all three. ### What it does The biggest lesson was the difference between decorating with math and building with math. We don't send raw files to Claude and ask "find debt." We compute real static signals first: severity = 0.35 * C + 0.30 * T + 0.20 * D + 0.15 * Doc The LLM reasons on top of those signals, not instead of them. Every finding is grounded in specific line numbers you can verify. We also learned that scope discipline is a feature — cutting from seven node types to four debt categories gave us the time to build auto-fix properly instead of shipping seven shallow things. ### How we built it Three layers, each independently verifiable: Static Analysis — the backend computes raw metrics per file (nesting depth, LOC, test file existence, dependency age) before touching the LLM. The LLM gets structured signals, not raw source. Schema-Constrained Extraction — Claude must cite specific line numbers to report a finding, or return nothing. Line references are validated server-side. No hallucinated findings reach the UI. Auto-Fix Pipeline — a second LLM call generates a complete file rewrite with a self-reported confidence score. Two gates run before the user sees anything: a confidence floor ( < 65\% ) drops the fix, and a syntax check (node --check, py_compile) deletes the branch on failure. On Apply, the branch pushes to GitHub and opens a pre-filled PR. On Discard, the temp directory is deleted entirely. ### Challenges we ran into JSON compliance. Claude occasionally responded to the auto-fix prompt with prose instead of JSON. Fix: strengthened system prompt plus a resilient parser that extracts the first & to last } as a fallback before throwing. Scratch repo isolation. DebtLens never has a full clone — only file content from the API. We build a scratch git repo containing just the file under fix, apply the rewrite, run the syntax gate, and push only that branch to GitHub on Apply. ESM syntax checking. node --check on ESM files in a bare scratch repo fails because Node defaults to CommonJS. Fix: inject a minimal {"type": "module" } package.json into the scratch repo before checking. Prompt specificity. Generic suggestions produce useless auto-fixes. We treated extraction quality as a hard gate - no auto-fix pipeline until every refactorSuggestion referenced exact line numbers and function names specific enough to act on without clarification. ### Accomplishments we're proud of The auto-fix pipeline working end to end on a real repo — branch isolation, confidence floor, syntax gate, and GitHub PR in one flow. That's not a demo feature, that's a production-grade safety architecture built in a hackathon timeline.Getting the LLM to reason on top of static signals rather than replacing them. Every finding cites real line numbers validated server-side. No hallucinated debt items reach the UI.Treating extraction quality as a hard gate before building auto-fix. We didn't start Step 9 until every suggestion was specific enough to act on without clarification. That discipline is why the auto-fix produces meaningful diffs instead of noise. ### What we learned Scope discipline is a feature. Cutting from seven node types to four debt categories gave us the time to build each one with real depth — and enough runway for auto-fix. The difference between decorating with math and building with math. A severity formula means nothing if the inputs aren't grounded in real computed metrics. We built the static analysis pass first precisely so the LLM had something concrete to reason about. Prompt compliance is an engineering problem, not a prompt problem. When Claude returned prose instead of JSON, the fix wasn't just a better prompt — it was a resilient parser that extracts { to } as a fallback. Robust systems don't rely on perfect model behavior. ### What's next Model flexibility. Engineers should be able to bring their own model — Claude, GPT-4o, Gemini, or a self-hosted Llama variant for teams with strict data privacy requirements. The LLM call is already isolated to a single module, so swapping the provider is an interface change, not an architectural one. A settings panel would let users paste in their own API key and select their preferred model without touching the codebase. Private repo support via GitHub OAuth, so teams can scan their actual production codebases rather than only public repos. IDE integration — a VS Code extension that surfaces debt items inline as you edit, rather than requiring a separate analysis run. Trend tracking — re-scan a repo over time and show whether the health score is improving or degrading sprint over sprint. Debt that's growing is more urgent than debt that's stable. Team dashboards — aggregate scores across multiple repos so engineering leads can see which codebases need the most attention at a glance. ## README (from the GitHub repository) # DebtLens **A technical-debt scanner for GitHub repositories.** DebtLens traverses a repo's source, runs a deterministic static-analysis pass to compute real metrics, then uses the Anthropic API to reason *on top of those signals* — not raw files — and produce a structured, line-grounded debt report. The architectural claim: **every finding is grounded in specific line numbers.** The LLM never invents debt it can't cite to a metric or a line. Out-of-bounds line refs are dropped before they ever reach the UI. --- ## What it does - **Repo health dashboard** — an overall 0–100 score plus four category scores (Complexity, Test Coverage, Dependencies, Documentation) with color bands. - **Prioritized debt list** — every finding with a severity badge, category tag, file path, and one-line summary; sortable and filterable by category. - **File drilldown** — full source with the referenced lines highlighted, next to the debt item's reasoning and a specific, actionable refactor suggestion. - **Auto-fix (Step 9)** — for a given debt item, generate a complete rewritten file, apply it to an **isolated throwaway branch**, run a **syntax gate**, and show a unified diff with a confidence score. It never touches your repo. You can then push the fix to a branch on GitHub (compare URL returned) or download the fixed file. --- ## How it works ``` GitHub URL │ ▼ github.js fetch the N most-recently-modified source files │ (N is user-chosen, default 30, capped at 50; skips │ node_modules, lockfiles, config, binaries) ▼ staticAnalysis.js per-file metrics — loc, functionCount, maxNestingDepth, │ todoCount, hasTestFile, docstringRatio, dependencyAge ▼ index.js per-category file selection (each criterion is a metric) │ ▼ llmExtractor.js Anthropic call per file → debt items, line-ref validated │ ▼ scorer.js severity formula + repo health + category scores │ ▼ server.js streams progress + final report to the React frontend (SSE) ``` **Severity formula** (`scorer.js`, weights are tunable): ``` severity = complexity·0.35 + testDebt·0.30 + depDebt·0.20 + docDebt·0.15 ``` where `complexity = norm(loc × maxNestingDepth)`, `testDebt = 0|100` by whether a test file exists, `depDebt = norm(maxDependencyAge)`, and `docDebt = (1 − docstringRatio)·100`. Overall repo health = `100 − mean(fileScores)`. --- ## Project structure ``` DebtLens/ ├── backend/ │ ├── server.js # Express server — /analyze (SSE) + auto-fix routes │ ├── index.js # Pipeline orchestration + CLI driver │ ├── github.js # GitHub API traversal (default 30 files, cap 50, in-memory cache) │ ├── staticAnalysis.js # Deterministic per-file metrics (no LLM) │ ├── llmExtractor.js # Anthropic calls + line-ref validation (4 categories) │ ├── scorer.js # Severity formula + repo/category scoring │ ├── env.js # Zero-dependency .env loader │ └── autoFix/ │ ├── fixGenerator.js # LLM call → full rewritten file (structured output) │ ├── fixApplier.js # Scratch git repo + isolated branch + syntax gate │ ├── diffBuilder.js # Unified diff via the `diff` package │ └── githubPush.js # Push fix to a GitHub branch (or download fallback) └── frontend/ └── src/ ├── App.jsx └── components/ ├── RepoInput.jsx # URL + file-count input (demo pre-loaded) ├── HealthDashboard.jsx # Overall + 4 category score cards ├── DebtList.jsx # Sortable / filterable debt list ├── FileDrilldown.jsx # Source viewer with line highlighting └── AutoFixPanel.jsx # Diff viewer + apply/discard controls ``` --- ## Getting started ### Prerequisites - **Node.js ≥ 20.6** (the backend uses the built-in `process.loadEnvFile`) - An **Anthropic API key** (uses `claude-sonnet-4-6`) - *(Optional)* a **GitHub token** — raises the API rate limit from 60 to 5000 req/hour, and is required to *push* auto-fixes to a branch ### 1. Backend ```bash cd backend npm install cp .env.example .env # then fill in ANTHROPIC_API_KEY (and optionally a token) npm run server # starts the API on http://localhost:3001 ``` `.env` keys: | Variable | Purpose | | ------------------------ | -------------------------------------------------------------- | | `ANTHROPIC_API_KEY` | Required — LLM extraction and fix generation | | `DEBTLENS_GITHUB_TOKEN` | Optional — preferred GitHub token (read for analysis, write for Apply) | | `GITHUB_TOKEN` | Optional — fallback if `DEBTLENS_GITHUB_TOKEN` is unset | > Apply-to-GitHub needs **write** access (`repo` / `public_repo` scope) and only > works on repos your token can push to (yours or a fork). For repos you don't > own, Apply falls back to downloading the fixed file. ### 2. Frontend ```bash cd frontend npm install npm run dev # Vite dev server; proxies /analyze and /api to :3001 ``` Open the printed URL, paste a public GitHub repo (pre-loaded with `expressjs/express`), pick a file count (1–50, defaults to 30), and hit **Analyze**. ### CLI (no frontend) The backend doubles as a CLI that prints the full report to the console: ```bash cd backend node index.js https://github.com/expressjs/express # or the shortcuts: npm run demo # sindresorhus/slugify npm run demo:express # expressjs/express ``` --- ## API | Method | Route | Description | | -------- | ----------------------- | --------------------------------------------------------------------------- | | `POST` | `/analyze` | `{ repoUrl, fileCount }` → SSE stream of `progress` events then a `done` event with the full report | | `POST` | `/api/autofix` | `{ file, debtItem }` → generate + apply (isolated branch) + syntax-check + diff | | `POST` | `/api/autofix/push` | `{ repoPath, branch, owner, repo, fix }` → push to a GitHub branch (or download fallback) | | `DELETE` | `/api/autofix/discard` | `{ repoPath, branch }` → tear down the temp branch + scratch repo | --- ## Auto-fix safety invariants Auto-fix is built so a bad suggestion can never corrupt your code: - The original repo is **never cloned or written to** — DebtLens only ever holds file *contents*. Each fix goes into a throwaway scratch git repo containing just the one file, on a `debtlens/autofix-{timestamp}` branch. - **No diff is shown unless the syntax check passes** (`node --check`, `python -m py_compile`, `go vet`; unchecked languages cap confidence at 70). - Fixes with `confidence < 65`, a null rewrite, or no overlap with the original line refs are **dropped server-side, with the reason shown** to the user. - **Confidence is always visible.** The temp branch is always deleted on discard. --- ## Key constraints - **File cap of 50**, with a default of 30 — adjustable in the UI, always surfaced and enforced server-side ("most recently modified files" — a defensible recency heuristic). - **Line-ref validation:** any finding referencing an out-of-bounds line is dropped and logged — never displayed. - **Rate limits:** GitHub is 60 req/hour unauthenticated; file contents are cached in memory for the session, and a token raises the limit to 5000/hour. --- ## Tech stack - **Backend:** Node.js + Express, `@anthropic-ai/sdk`, `simple-git`, `diff` - **Frontend:** React 18 + Vite - **APIs:** GitHub REST API, Anthropic API (`claude-sonnet-4-6`) ## Detected evidence (automated analysis) Indexed codebase: 27 recognized source files, 185 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - Express (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - React (technology) — detected in the code - Node.js (technology) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers ## Codebase structure (from repository index) ### Files (31 of 31) ``` .gitignore backend/.env.example backend/autoFix/diffBuilder.js backend/autoFix/fixApplier.js backend/autoFix/fixGenerator.js backend/autoFix/githubPush.js backend/env.js backend/github.js backend/index.js backend/llmExtractor.js backend/package.json backend/repoStore.js backend/scorer.js backend/server.js backend/staticAnalysis.js CLAUDE.md frontend/index.html frontend/package.json frontend/src/App.jsx frontend/src/components/AutoFixPanel.jsx frontend/src/components/DebtList.jsx frontend/src/components/FileDrilldown.jsx frontend/src/components/HealthDashboard.jsx frontend/src/components/RepoInput.jsx frontend/src/components/Sidebar.jsx frontend/src/components/TopBar.jsx frontend/src/index.css frontend/src/main.jsx frontend/src/utils.js frontend/vite.config.js README.md ``` ### Dependencies - backend/package.json: @anthropic-ai/sdk@^0.105.0, cors@^2.8.6, diff@^9.0.0, express@^4.22.2, simple-git@^3.36.0 - frontend/package.json: @vitejs/plugin-react@^4.3.1, react@^18.3.1, react-dom@^18.3.1, vite@^5.4.2 ### Recent commits (newest first) - docs: correct file-cap description, fix sidebar version string - feat: add hover tooltips to health dashboard score cards - feat: multi-repo history with caching and delete - feat: add app shell layout with topbar/sidebar, strip non-functional buttons - Merge branch 'step-9-autofix' - docs: write full README (overview, setup, API, auto-fix safety) - Merge pull request #1 from backSpaceTwice/step-9-autofix - feat: dedicated GitHub token resolver + structured PR description - feat: real "Apply fix" — push to GitHub branch or download fallback - docs: note Step 9 as-built deviations in CLAUDE.md - feat: Step 9 — auto-fix (generate → isolated branch + syntax gate → diff) - feat: Step 8 — polish (SSE loading steps, weight sliders, demo URL, error states) - feat: Step 7 — file drilldown with line highlighting - feat: Step 6 — debt list with expand, sort, filter + user file-count control - feat: Step 5 — React health dashboard + backend performance - feat: Step 4 — severity scoring and repo health scores - feat: Step 3 — extend LLM extraction to all four debt categories - feat: Step 2 — LLM extraction for complexity debt - feat: Step 1 — GitHub traversal + static analysis backend - chore: add CLAUDE.md project brief ## Key source files (fetched from GitHub, selected and truncated for size) ### CLAUDE.md ```markdown # DebtLens — Claude Code Instructions ## Project Overview DebtLens is a technical debt scanner for GitHub repositories. It traverses source files, runs a static analysis pass to compute measurable metrics, then uses the Anthropic API to reason over those signals and produce a structured debt report. **Core output:** - Repo-level health dashboard (4 category scores + overall) - Prioritized debt list with severity scores and reasoning - Per-file drilldown with line-level grounding (click a debt item → see the exact lines) **The key architectural claim:** the LLM reasons on top of real static signals, not raw files. Every finding is grounded in specific line numbers. --- ## Tech Stack - **Backend:** Node.js (Express) — GitHub API traversal, static analysis, Anthropic API calls, auto-fix orchestration - **Frontend:** React — health dashboard, debt list, file drilldown viewer, auto-fix diff viewer - **APIs:** GitHub REST API (public repos via URL, no auth required for MVP), Anthropic API (`claude-sonnet-4-6`) - **Auto-fix:** `simple-git` (Node.js git bindings) — branch creation, file patching, syntax verification, PR diff generation --- ## Project Structure ``` debtlens/ ├── backend/ │ ├── index.js # Express server entry point │ ├── github.js # GitHub API file traversal │ ├── staticAnalysis.js # Computable metrics (no LLM) │ ├── contextAssembler.js # Picks which files to send to LLM │ ├── llmExtractor.js # Anthropic API call + schema validation │ ├── scorer.js # Severity score calculation │ └── autoFix/ │ ├── fixGenerator.js # LLM call to generate the actual code fix │ ├── fixApplier.js # Writes fix to temp branch, runs syntax check │ ├── diffBuilder.js # Builds unified diff for UI display │ └── prBuilder.js # Assembles PR description with reasoning ├── frontend/ │ ├── src/ │ │ ├── App.jsx │ │ ├── components/ │ │ │ ├── RepoInput.jsx │ │ │ ├── HealthDashboard.jsx │ │ │ ├── DebtList.jsx │ │ │ ├── FileDrilldown.jsx │ │ │ └── AutoFixPanel.jsx # Diff viewer + accept/discard controls │ │ └── index.css │ └── package.json ├── backend/package.json └── CLAUDE.md ``` --- ## Build Priority Order Build in this exact order. If time runs short, stop at whatever step you're on — every step produces something that works and demonstrates the core claim. ### Step 1 — Backend: GitHub traversal + static analysis (no LLM) - Accept a public GitHub repo URL, extract `owner/repo` - Use GitHub REST API to list files, filter out `node_modules`, `.json` config, lockfiles, and binary files - Cap at the **50 most-recently-modified files** (recency heuristic — defensible, say so in UI) - For each file, compute these raw metrics: - `loc` — lines of code - `functionCount` — rough count (regex on `function`, `def`, `=>` etc.) - `maxNestingDepth` — count max indentation levels - `todoCount` — count `TODO`, `FIXME`, `HA [truncated — 16920 more characters] ``` ### frontend/package.json ``` { "name": "debtlens-frontend", "version": "0.1.0", "private": true, "type": "module", "scripts": { "dev": "vite", "build": "vite build", "preview": "vite preview" }, "dependencies": { "react": "^18.3.1", "react-dom": "^18.3.1" }, "devDependencies": { "@vitejs/plugin-react": "^4.3.1", "vite": "^5.4.2" }, "allowScripts": { "esbuild@0.21.5": true, "fsevents@2.3.3": true } } ``` ### backend/package.json ``` { "name": "debtlens-backend", "version": "0.1.0", "private": true, "type": "module", "description": "DebtLens backend — GitHub traversal, static analysis, and LLM debt extraction", "main": "index.js", "scripts": { "start": "node index.js", "server": "node server.js", "analyze": "node index.js", "demo": "node index.js https://github.com/sindresorhus/slugify", "demo:express": "node index.js https://github.com/expressjs/express" }, "engines": { "node": ">=18" }, "dependencies": { "@anthropic-ai/sdk": "^0.105.0", "cors": "^2.8.6", "diff": "^9.0.0", "express": "^4.22.2", "simple-git": "^3.36.0" } } ``` ### backend/server.js ```javascript // server.js — DebtLens Express server. // POST /analyze → text/event-stream with progress events + final result import './env.js'; import express from 'express'; import cors from 'cors'; import { analyzeRepo, extractAllDebt, runAutoFix, discardAutoFix, pushAutoFix } from './index.js'; import { scoreRepo, WEIGHTS } from './scorer.js'; import { saveResult, listRepos, getRepo, deleteRepo } from './repoStore.js'; const app = express(); app.use(cors()); app.use(express.json()); app.post('/analyze', async (req, res) => { const { repoUrl, fileCount } = req.body ?? {}; if (!repoUrl) { return res.status(400).json({ error: 'repoUrl is required' }); } const cap = Math.min(50, Math.max(1, parseInt(fileCount) || 30)); res.setHeader('Content-Type', 'text/event-stream'); res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); res.setHeader('X-Accel-Buffering', 'no'); res.flushHeaders(); function send(type, data = {}) { res.write(`data: ${JSON.stringify({ type, ...data })}\n\n`); } try { const { meta, files, fileMetrics } = await analyzeRepo(repoUrl, cap, (p) => send('progress', p) ); const debtResults = await extractAllDebt(files, fileMetrics, (p) => send('progress', p) ); send('progress', { step: 'scoring', message: 'Scoring and building report…' }); const { overallHealth, categoryScores, fileScores } = scoreRepo( fileMetrics, debtResults, WEIGHTS ); const debtPaths = new Set(debtResults.map((r) => r.file)); const fileContents = {}; for (const f of files) { if (debtPaths.has(f.path)) fileContents[f.path] = f.content; } const result = { meta: { fullName: meta.fullName, language: meta.language ?? null, fileCount: fileMetrics.length, analyzedAt: new Date().toISOString(), }, overallHealth, categoryScores, fileScores, debtResults, fileContents, }; saveResult(result); send('done', { result }); } catch (err) { console.error('Analysis error:', err.message); send('error', { message: err.message }); } res.end(); }); // ── Multi-repo history (cached analyses, no re-analyzing) ────────────────── // GET /api/repos → summary list of cached repos, most-recently-analyzed first app.get('/api/repos', (req, res) => { res.json(listRepos()); }); // GET /api/repos/:owner/:repo → full cached result for one repo app.get('/api/repos/:owner/:repo', (req, res) => { const { owner, repo } = req.params; const result = getRepo(owner, repo); if (!result) { return res.status(404).json({ error: `No cached analysis for ${owner}/${repo}` }); } res.json(result); }); // DELETE /api/repos/:owner/:repo → remove one repo from the history cache app.delete('/api/repos/:owner/:repo', (req, res) => { const { owner, repo } = req.params; const existed = deleteRepo(owner, repo); if (!existed) { return res.status(404).json({ error: `No cached analysis for ${owner}/${repo}` }); } res.json({ ok: true }); }); // ── Auto-fix (Step 9) ────────────────────────────────────────────────────── // POST /api/autofix → generate + apply (isolated branch) + syntax-check + diff app.post('/api/autofix', async (req, res) => { const { file, debtItem } = req.body ?? {}; if (!file?.path || typeof file.content !== 'string' || !debtItem) { return res .status(400) .json({ error: 'file ({ path, content }) and debtItem are required' }); } try { const result = await runAutoFix(file, debtItem); res.json(result); } catch (err) { console.error('Auto-fix error:', err.message); res.status(500).json({ status: 'error', reason: err.message }); } }); // POST /api/autofix/push → commit the fix to a branch on the real GitHub repo // (via GITHUB_TOKEN) and return a compare URL; falls back to file download. app.post('/api/autofix/push', async (req, res) => { const { repoPath, branch, owner, repo, fix } = req.body ?? {}; if (!repoPath || !branch || !owner || !repo || !fix?.file) { return res .status(400) .json({ error: 'repoPath, branch, owner, repo, and fix.file are required' }); } try { const result = await pushAutoFix({ repoPath, branch, owner, repo, fix }); if (result.status === 'error') return res.status(500).json(result); res.json(result); } catch (err) { console.error('Push error:', err.message); res.status(500).json({ status: 'error', error: err.message }); } }); // DELETE /api/autofix/discard → tear down the temp branch + scratch repo app.delete('/api/autofix/discard', async (req, res) => { const { repoPath, branch } = req.body ?? {}; if (!repoPath) { return res.status(400).json({ error: 'repoPath is required' }); } try { await discardAutoFix(repoPath, branch); res.json({ ok: true }); } catch (err) { console.error('Discard error:', err.message); res.status(500).json({ ok: false, error: err.message }); } }); const PORT = process.env.PORT ?? 3001; app.listen(PORT, () => console.log(`DebtLens backend listening on http://localhost:${PORT}`) ); ``` ### backend/index.js ```javascript // index.js — DebtLens backend entry point. // // CLI driver: // node index.js // // Step 1: traverse the repo and run the static-analysis pass (no LLM). // Step 2: complexity-debt extraction (top files by loc × maxNestingDepth). // Step 3: extend extraction to all four categories — complexity, test, // dependency, documentation — each selected by its own static-metric // criterion and validated with the same line-ref grounding. // // Later steps add severity scoring and an Express server. import './env.js'; // load .env before anything reads process.env import { getRepoFiles } from './github.js'; import { analyzeFile, analyzeDependencies, buildRepoIndex, } from './staticAnalysis.js'; import { extractComplexityDebt, extractTestDebt, extractDependencyDebt, extractDocumentationDebt, } from './llmExtractor.js'; import { scoreRepo, WEIGHTS } from './scorer.js'; import { generateFix } from './autoFix/fixGenerator.js'; import { applyGeneratedFix, discardFix } from './autoFix/fixApplier.js'; import { buildDiffForFix } from './autoFix/diffBuilder.js'; import { pushFixToGitHub } from './autoFix/githubPush.js'; const PER_CATEGORY_LIMIT = 5; /** * Run the full static-analysis pass over a repo. * Returns the file contents alongside the metrics so the LLM steps can use * both without re-fetching. */ export async function analyzeRepo(repoUrl, fileCount = 30, onProgress = null) { onProgress?.({ step: 'fetch', message: 'Fetching repository files…' }); const { meta, files, allPaths } = await getRepoFiles(repoUrl, fileCount); onProgress?.({ step: 'fetch_done', message: `${files.length} files retrieved`, fileCount: files.length }); const repoIndex = buildRepoIndex(allPaths); console.log('🧮 Running static analysis...'); onProgress?.({ step: 'analysis', message: 'Running static analysis…' }); const fileMetrics = []; for (const file of files) { const metrics = analyzeFile(file, repoIndex); // Dependency manifests get an extra async pass (npm / PyPI age lookups). const dependency = await analyzeDependencies(file); if (dependency) metrics.dependency = dependency; fileMetrics.push(metrics); } onProgress?.({ step: 'analysis_done', message: 'Static analysis complete' }); return { meta, files, fileMetrics }; } // --------------------------------------------------------------------------- // Per-category file selection — each criterion comes straight from CLAUDE.md. // Only real source files are eligible for code-level categories. // --------------------------------------------------------------------------- const isSource = (m) => m.language !== 'other'; /** Complexity: highest loc × maxNestingDepth. */ export function selectComplexityFiles(fileMetrics, limit = PER_CATEGORY_LIMIT) { return fileMetrics .filter(isSource) .map((m) => ({ m, score: m.loc * m.maxNestingDepth })) .sort((a, b) => b.score - a.score) .slice(0, limit) .map((x) => x.m); } /** Test debt: files with no corresponding test file (largest first). */ export function selectTestDebtFiles(fileMetrics, limit = PER_CATEGORY_LIMIT) { return fileMetrics .filter((m) => isSource(m) && m.hasTestFile === false) .sort((a, b) => b.loc - a.loc) .slice(0, limit); } /** Documentation debt: docstringRatio < 0.5 with functionCount > 5. */ export function selectDocumentationFiles(fileMetrics, limit = PER_CATEGORY_LIMIT) { return fileMetrics .filter((m) => isSource(m) && m.functionCount > 5 && m.docstringRatio < 0.5) .sort((a, b) => b.functionCount - a.functionCount) .slice(0, limit); } /** Dependency debt: manifests with a dependency older than ~365 days. */ export function selectDependencyFiles(fileMetrics, limit = PER_CATEGORY_LIMIT) { return fileMetrics .filter( (m) => m.dependency && (m.dependency.staleCount > 0 || m.dependency.maxDependencyAge > 365) ) .sort((a, b) => b.dependency.maxDependencyAge - a.dependency.maxDependencyAge) .slice(0, limit); } // Run at most CONCURRENCY LLM calls simultaneously to stay within rate limits // while still being much faster than fully sequential. const CONCURRENCY = 10; async function runWithConcurrency(tasks) { const results = new Array(tasks.length); let idx = 0; async function worker() { while (idx < tasks.length) { const i = idx++; results[i] = await tasks[i](); } } await Promise.all(Array.from({ length: CONCURRENCY }, worker)); return results; } /** * Run all four debt-category extractions over their selected files. * All LLM calls run in parallel (capped at CONCURRENCY) across all categories. * @returns flat array of { file, category, debtItems } (already line-validated) */ export async function extractAllDebt(files, fileMetrics, onProgress = null) { const contentByPath = new Map(files.map((f) => [f.path, f])); const jobs = [ { name: 'complexity', fn: extractComplexityDebt, files: selectComplexityFiles(fileMetrics) }, { name: 'test', fn: extractTestDebt, files: selectTestDebtFiles(fileMetrics) }, { name: 'dependency', fn: extractDependencyDebt, files: selectDependencyFiles(fileMetrics) }, { name: 'documentation', fn: extractDocumentationDebt, files: selectDocumentationFiles(fileMetrics) }, ]; const total = jobs.reduce((s, j) => s + j.files.length, 0); console.log(`\n🤖 LLM extraction — ${total} file(s) across 4 categories (concurrency ${CONCURRENCY})...`); onProgress?.({ step: 'llm', message: 'Analyzing with AI…', done: 0, total }); let completed = 0; const tasks = []; for (const job of jobs) { for (const metrics of job.files) { const file = contentByPath.get(metrics.path); if (!file) continue; tasks.push(async () => { const result = await job.fn(file, metrics); completed++; console.log(` ✓ [${job.name}] ${metrics.path} — ${result.debtItems.length} item(s)`); onProgress?. [truncated — 5639 more characters] ``` ### frontend/src/main.jsx ```javascript import React from 'react'; import ReactDOM from 'react-dom/client'; import App from './App.jsx'; import './index.css'; ReactDOM.createRoot(document.getElementById('root')).render( ); ``` ### frontend/src/App.jsx ```javascript import { useEffect, useState } from 'react'; import TopBar from './components/TopBar.jsx'; import Sidebar from './components/Sidebar.jsx'; import RepoInput from './components/RepoInput.jsx'; import HealthDashboard from './components/HealthDashboard.jsx'; import DebtList from './components/DebtList.jsx'; import FileDrilldown from './components/FileDrilldown.jsx'; const STAGES = [ { key: 'fetch', label: 'Fetching repository files' }, { key: 'analysis', label: 'Running static analysis' }, { key: 'llm', label: 'Analyzing with AI' }, { key: 'scoring', label: 'Scoring results' }, ]; function stageState(key, steps) { const seen = new Set(steps.map((s) => s.step)); const after = (k) => { const idx = STAGES.findIndex((s) => s.key === k); return STAGES.slice(idx + 1).some((s) => seen.has(s.key)); }; if (key === 'fetch') { if (seen.has('fetch_done') || after('fetch')) return 'done'; if (seen.has('fetch')) return 'active'; } if (key === 'analysis') { if (seen.has('analysis_done') || seen.has('llm') || seen.has('scoring')) return 'done'; if (seen.has('analysis')) return 'active'; } if (key === 'llm') { if (seen.has('scoring')) return 'done'; if (seen.has('llm') || seen.has('analysis_done')) return 'active'; } if (key === 'scoring') { if (seen.has('scoring')) return 'active'; } return 'pending'; } function LoadingSteps({ steps }) { const fetchDone = steps.find((s) => s.step === 'fetch_done'); const llmSteps = steps.filter((s) => s.step === 'llm'); const lastLlm = llmSteps[llmSteps.length - 1]; return (
{STAGES.map(({ key, label }) => { const state = stageState(key, steps); return (
{state === 'done' ? '✓' : state === 'active' ? '●' : '○'} {label} {key === 'fetch' && fetchDone && ( — {fetchDone.fileCount} files )} {key === 'llm' && lastLlm && state === 'active' && ( — {lastLlm.done}/{lastLlm.total} files )} {state === 'active' && ( )}
); })}
); } function classifyError(message) { if (/rate limit/i.test(message)) return 'rate-limit'; if (/private repo|not found/i.test(message)) return 'private-repo'; return 'general'; } function ErrorState({ message }) { const type = classifyError(message); const hints = { 'rate-limit': 'GitHub allows 60 unauthenticated requests per hour. Set a GITHUB_TOKEN in the backend .env to raise this to 5 000/hour, or wait for the reset time shown above.', 'private-repo': 'DebtLens only supports public repositories. Check the URL and make sure the repo is public.', }; return (
Analysis failed: {message}
{hints[type] &&
{hints[type]}
}
); } export default function App() { const [result, setResult] = useState(null); const [loading, setLoading] = useState(false); const [loadingSteps, setLoadingSteps] = useState([]); const [error, setError] = useState(null); const [selectedItem, setSelectedItem] = useState(null); const [repos, setRepos] = useState([]); async function refreshRepos() { try { const res = await fetch('/api/repos'); if (res.ok) setRepos(await res.json()); } catch { // Sidebar history is best-effort — silently skip on failure. } } useEffect(() => { refreshRepos(); }, []); async function handleSelectRepo(owner, repo) { setError(null); setSelectedItem(null); try { const res = await fetch(`/api/repos/${owner}/${repo}`); if (!res.ok) { const data = await res.json().catch(() => ({})); throw new Error(data.error ?? `HTTP ${res.status}`); } setResult(await res.json()); } catch (err) { setError(err.message); } } async function handleDeleteRepo(owner, repo) { try { await fetch(`/api/repos/${owner}/${repo}`, { method: 'DELETE' }); } catch { // Best-effort — refreshRepos() below will reflect whatever the server actually has. } if (result?.meta?.fullName === `${owner}/${repo}`) { setResult(null); setSelectedItem(null); } refreshRepos(); } async function handleAnalyze(repoUrl, fileCount) { setLoading(true); setLoadingSteps([]); setError(null); setResult(null); setSelectedItem(null); try { const res = await fetch('/analyze', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ repoUrl, fileCount }), }); if (!res.ok) { const data = await res.json().catch(() => ({})); throw new Error(data.error ?? `HTTP ${res.status}`); } const reader = res.body.getReader(); const decoder = new TextDecoder(); let buffer = ''; while (true) { const { done, value } = await reader.read(); if (done) break; buffer += decoder.decode(value, { stream: true }); const parts = buffer.split('\n\n'); buffer = parts.pop(); for (const part of parts) { if (!part.startsWith('data: ')) continue; let msg; try { msg = JSON.parse(part.slice(6)); } catch { continue; } if (msg.type === 'progress') { setLoadingSteps((prev) => [...prev, msg]); } else if (msg.type === 'done') { setResult(msg.result); refreshRepos(); } else i [truncated — 1612 more characters] ``` ### frontend/vite.config.js ```javascript import { defineConfig } from 'vite'; import react from '@vitejs/plugin-react'; export default defineConfig({ plugins: [react()], server: { proxy: { '/analyze': 'http://localhost:3001', '/api': 'http://localhost:3001', }, }, }); ``` ### frontend/index.html ```html DebtLens — Technical Debt Scanner
``` ### backend/env.js ```javascript // env.js — zero-dependency .env loader. // // Imported first by index.js so environment variables (GITHUB_TOKEN, // ANTHROPIC_API_KEY, ...) are available before any other module reads them. // Uses Node's built-in process.loadEnvFile (Node >= 20.6); the .env is // resolved relative to this file so it works no matter where you run from. import { fileURLToPath } from 'node:url'; import { dirname, join } from 'node:path'; const here = dirname(fileURLToPath(import.meta.url)); const envPath = join(here, '.env'); try { process.loadEnvFile(envPath); console.log(`🔑 Loaded environment from ${envPath}`); } catch { // No .env file (or unreadable) — that's fine, env vars may be set inline. } ``` [18 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]