# Project export: ClassLens

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: OpenAI Build Week
- Tagline: Diagnoses what an Algebra-1 class misunderstands from their wrong answers, then builds a recheck quiz that verifies the fix. 41.1% @1 measured on 716 held-out real student answers.
- Devpost: https://devpost.com/software/notdecided-6oh4uc
- GitHub: https://github.com/V-3604/ClassLens
- Demo: https://htmlpreview.github.io/?https://github.com/V-3604/ClassLens/blob/master/examples/sample_report.html
- Video: https://www.youtube.com/embed/n6QWy5xHBMY?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — Varshith (15 commits)

## Devpost submission (written by the team)

### Inspiration

Teachers get exit ticket scores back and can see who missed question 4. Working out why takes longer than the prep period they have. I wanted to build the thing that answers why, and then I wanted a number telling me how often it was right.

### What it does

Takes a CSV of a class's quiz responses. Maps each wrong answer to a named misconception from the Eedi taxonomy (2,587 labels), then aggregates a class profile ranked by how many students each misconception affects and writes a reteach plan. Then it builds a recheck quiz. GPT-5.6 writes the stem, SymPy computes the correct answer, and an executable buggy-math rule generates the distractor. Every finished item goes back through diagnosis blind. Items that fail to map to their target misconception are rejected. Output: a static HTML class report, a Markdown reteach plan, and a verified recheck quiz. How I built it Codex built the core over three days: benchmark harness, executable rules, factory, report renderer. GPT-5.6 runs at inference time for three cached misconception hypotheses, rerank and verification over a fixed local candidate pool, stem extraction, factory item generation, and reteach writing. Retrieval is local: BGE-large hybrid plus a TF-IDF arm, combined with reciprocal-rank fusion. That keeps the baseline free and lets the GPT-5.6 contribution be measured on its own. SymPy grounds every computable answer. GPT responses are cached by model, prompt, and reasoning effort, and the cache is committed, so anyone can regenerate the report and the metrics without an API key. All of this ran on gpt-5.6-luna. I picked the cheap model on purpose: the whole point is a teacher running this on every exit ticket for every class, and that only works if one class costs cents. The lift comes from the retrieval stack: three cached hypotheses, hybrid retrieval, rerank and verify. I never benchmarked terra, so I don't know whether the bigger variants do better. Challenges I ran into The retrieval-only baseline wasn't actually retrieval-only. use_rerank=False skipped the rerank call but still ran the GPT-5.6 hypothesis arm, so the number I was calling "local embeddings alone" had GPT in it. That inflates the lift, which is the entire headline claim. I added an early return so the flag kills the whole GPT path, wrote a guard test for that boundary, then replayed all 716 held-out items with CLASSLENS_OFFLINE=1 and no GPT client instantiated. It came back 54/716 @1 and 143/716 @3, matching the published 7.5% and 20.0%. The lift was clean, but I had no way to know that until I could run the baseline with the client physically absent. Accomplishments that I'm proud of Diagnosis on 716 held-out real Eedi pairs: local retrieval 7.5% @1, GPT-5.6 best arm 41.1% @1 and 55.4% @3. A +33.5-point lift, 95% CI +27.8 to +38.9. Executable rules reproduce the exact distractor a real student picked 71 times out of 258 (27.5%), with parse failures counted in the denominator. The factory accepted 10 of 33 items and rejected 23, with the rejection reason logged for each one and shown in the report. What I learned Writing the benchmark before tuning against it killed several changes that felt like wins. A k=50 candidate pool didn't beat k=25 on recall@25. A high-effort selector variant didn't move accuracy@1. Both would have shipped on intuition. Rejection has to be visible or verification is decorative. The factory only became worth anything once the 23 failures sat in the report next to the 10 successes.

### What's next

Grammar coverage is what caps distractor reproduction. 41.1% of eligible items parse, and 67.0% of those reproduce correctly, so widening the parser moves the headline without touching a single transform. After that, more Algebra-1 rules, then geometry, and an importer for LMS quiz exports.

## README (from the GitHub repository)

# ClassLens

Turn a class's wrong answers into tomorrow's lesson — and verify the fix before a student sees it.

ClassLens takes a class's responses to an Algebra-1 diagnostic quiz and maps each wrong answer to a **named misconception** (grounded in the Eedi misconception taxonomy), with **measured accuracy on held-out official Eedi diagnostic item–distractor mappings**. It then encodes the class's top misconceptions as **executable buggy-math rules**, and runs a **factory** that manufactures a **verified recheck quiz** provably targeting them. Output: a class report (static HTML), an editable reteach plan (Markdown), and the recheck quiz.

Built with **Codex** and **GPT-5.6** for OpenAI Build Week. Track: **Education**.

[![ClassLens class report — measured diagnosis quality on 716 held-out real Eedi pairs](examples/report_preview.png)](https://htmlpreview.github.io/?https://github.com/V-3604/ClassLens/blob/master/examples/sample_report.html)

> **Judge snapshot:** GPT-5.6 Luna’s selected three-hypothesis arm achieved **41.1% @1 / 55.4% @3** on n=716 real held-out Eedi pairs, a **+33.5-point audited @1 lift** over local retrieval. Executable rules achieved **41.1% grammar coverage × 67.0% in-grammar fidelity = 27.5% strict reproduction** (71/258), and the blind factory accepted **10/33 (30.3%)** items while visibly rejecting 23 attempts.

**See it with zero setup** — [view the live report](https://htmlpreview.github.io/?https://github.com/V-3604/ClassLens/blob/master/examples/sample_report.html) (no clone needed), or open the committed [`examples/sample_report.html`](examples/sample_report.html) directly. It is the pre-rendered judge-facing artifact; it does not run the pipeline or call an API.

> Status: all core CLI paths are implemented and locally tested. Retrieval has completed its held-out calibration; factory and executable-rule measurements are seeded, bounded real-Eedi calibrations.

## Why
Algebra 1 is the gatekeeper course, and teachers run the same daily loop — teach, quick-check, reteach — but cannot turn 150 exit-ticket responses into "what does this class actually misunderstand, and what do I reteach tomorrow" in a 20-minute prep. They see scores, not causes. ClassLens surfaces the causes, writes the reteach, and verifies the recheck questions actually target the diagnosed misconceptions.

## How it works
1. **Ingest** quiz questions + student responses from a CSV export.
2. **Diagnose** — for each wrong answer, retrieve candidate misconceptions from the Eedi taxonomy (local embeddings) and use GPT-5.6 to rerank/verify the best match against the specific question and answer.
3. **Ground** — verify computable items symbolically with SymPy, so correctness is checked, not guessed.
4. **Profile** — aggregate a class-level misconception profile ranked by prevalence, with the students affected by each.
5. **Flag broken questions** — where "wrong" answers reflect valid reasoning or ambiguity, flag the question, not the students.
6. **Reteach** — GPT-5.6 writes an editable plan per top misconception: a worked example, the counterexample that breaks it, targeted practice, and a recheck item.
7. **Verify the recheck** — the recheck items come from a factory: GPT-5.6 writes the stem, SymPy computes the correct answer, an executable buggy-math rule generates the distractor, and the diagnosis engine must map that distractor back to the intended misconception (blind) or the item is rejected and regenerated.

**Design principle:** ClassLens attaches to the invariant (misconceptions are curriculum-independent), not the variable (lesson plans, textbooks, formats). Bring your own questions in any format; it maps errors to a universal taxonomy. Narrow on subject, general across teachers.

## Measured results
On a held-out split of official Eedi question–wrong-answer–misconception pairs:
- Final seeded 716-item held-out Eedi evaluation, seed 47: BAAI BGE-large hybrid local retrieval reached **7.5% @1 / 20.0% @3** (95% CI @1 **5.8–9.7%**). The selected three-hypothesis + GPT-5.6 Luna rerank arm reached **41.1% @1 / 55.4% @3** (95% CIs **37.5–44.7%** / **51.8–59.0%**) with **63.8% recall@25**: a measured **+33.5-point @1 lift** with conservative 95% CI **+27.8 to +38.9 points**. The rerank-only arm is a clearly labeled supplementary n=75 decomposition (**30.7% / 41.3%** @1/@3), not the headline comparison. The earlier fixed 75-item retrieval calibration selected BGE-large hybrid plus three hypotheses (64.0% recall@25); k=50 did not improve that calibration, so the candidate pool remains 25.
- Executable-misconception distractor reproduction: **27.5% (71/258)** across the full eligible seeded official-Eedi development set, after widening only grammar recognition against the same cached 258 GPT extractions. Grammar coverage is **41.1% (106/258)** and in-grammar fidelity is **67.0% (71/106)**, so **coverage × fidelity = strict rate**. The strict rate remains the headline because parse misses are counted rather than hidden.
- Factory round-trip acceptance: **30.3% (10/33)** across three executable misconception targets; **23 attempts were visibly rejected**. Blind local retrieval hit the intended target on 10 of 26 viable attempts (38.5%) and missed it on 16, which is surfaced rather than hidden by target injection.

### Executable-rule validation (full eligible set, n=258, seed 47)

| Rule | Funnel: eligible -> in-grammar -> reproduced | Coverage | Fidelity | Strict rate |
| --- | ---: | ---: | ---: | ---: |
| same-operation-not-inverse | 41 -> 2 -> 1 | 4.9% | 50.0% | 2.4% |
| left-to-right priority | 27 -> 6 -> 5 | 22.2% | 83.3% | 18.5% |
| negative-times-negative | 24 -> 3 -> 3 | 12.5% | 100.0% | 12.5% |
| first-term-only expansion | 17 -> 7 -> 7 | 41.2% | 100.0% | 41.2% |
| multiply-by-index | 16 -> 10 -> 10 | 62.5% | 100.0% | 62.5% |
| add-fraction-denominators | 15 -> 9 -> 8 | 60.0% | 88.9% | 53.3% |
| positive-negative-is-positive | 15 -> 6 -> 4 | 40.0% | 66.7% | 26.7% |
| add-instead-of-expand | 12 -> 3 -> 1 | 25.0% | 33.3% | 8.3% |
| subtraction-is-commutative | 11 -> 4 -> 4 | 36.4% | 100.0% | 36.4% |
| double-instead-of-square | 31 -> 23 -> 13 | 74.2% | 56.5% | 41.9% |
| division-is-commutative | 17 -> 10 -> 5 | 58.8% | 50.0% | 29.4% |
| add-instead-of-multiply | 11 -> 8 -> 4 | 72.7% | 50.0% | 36.4% |
| reverse-inequality | 21 -> 15 -> 6 | 71.4% | 40.0% | 28.6% |

### Class-aggregation robustness

Class prevalence is calculated from distinct affected learners rather than repeated responses, and identical `(question, answer, explanation)` pairs are deduplicated before runtime mapping. The report keeps per-question evidence and symbolically identified slips separate from named misconceptions, so a repeated answer pattern cannot silently inflate a reteach target. The shipped class is explicitly an illustrative composition of real Eedi answers, not a claim of longitudinal student data.

## How GPT-5.6 is used
The retrieval-only baseline is local and never calls an API. GPT-5.6 writes three cached misconception hypotheses for the strongest retrieval arm, then reranks and verifies the fixed local candidate pool against each specific question and wrong answer; the held-out benchmark measures all three arms, and the runtime `diagnose` command ships that same selected best arm (three cached hypotheses + hybrid retrieval + rerank/verify), so the product runs the configuration the benchmark measured. Later runtime roles are structured stem extraction, factory-item generation, reteach generation, class summaries, and broken-question detection. Calls are cached by model, prompt, and reasoning effort; GPT-5.6 reasoning models do not accept a temperature parameter, so deterministic roles use its fixed default behavior. The default runtime model is `gpt-5.6-luna` for cost-disciplined batch diagnosis; `gpt-5.6-terra` is reserved for a small final comparison or difficult verification cases.

## How Codex was used
Codex built the core of ClassLens over a three-day se

[README truncated for size]

## Detected evidence (automated analysis)

Indexed codebase: 50 recognized source files, 303 KB.
- HTML (language) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- CSS (language) — claimed on Devpost, not found in the code
- AI coding agent: Codex — evidence: config files committed to the repository

## Codebase structure (from repository index)

### Files (59 of 59)

```
.env.example
.gitignore
AGENTS.md
ARCHITECTURE.md
data/README.md
examples/build_sample.py
examples/README.md
examples/sample_class.csv
examples/sample_questions.csv
examples/sample_report.html
examples/sample_report.md
examples/validation_funnel.json
examples/validation_summary.json
LICENSE
prompts/broken_question.md
prompts/extraction.md
prompts/hypothesize_multiple.md
prompts/hypothesize.md
prompts/item_gen.md
prompts/README.md
prompts/rerank.md
prompts/reteach.md
pyproject.toml
README.md
requirements.txt
src/classlens/__init__.py
src/classlens/__main__.py
src/classlens/benchmark.py
src/classlens/broken_questions.py
src/classlens/buggy_rules.py
src/classlens/cli.py
src/classlens/costs.py
src/classlens/eedi_extraction.py
src/classlens/factory.py
src/classlens/ingest.py
src/classlens/llm.py
src/classlens/mapper.py
src/classlens/models.py
src/classlens/prepare_eedi.py
src/classlens/profile.py
src/classlens/report_design.html
src/classlens/report.py
src/classlens/reteach.py
src/classlens/statistics.py
src/classlens/taxonomy.py
src/classlens/verify.py
tests/test_broken_questions.py
tests/test_buggy_rules.py
tests/test_costs.py
tests/test_eedi_extraction.py
tests/test_factory.py
tests/test_hypothesis.py
tests/test_recheck_quiz.py
tests/test_report_math.py
tests/test_smoke.py
tests/test_statistics.py
tests/test_verify_latex.py
tests/test_verify.py
tests/test_vertical_slice.py
```

### Dependencies

- requirements.txt: numpy@==2.5.1, openai@==2.45.0, pandas@==3.0.3, pytest@==9.1.1, python-dotenv@==1.2.2, scikit-learn@==1.9.0, sentence-transformers@==5.6.0, sympy@==1.14.0, tqdm@==4.68.4

### Recent commits (newest first)

- Replace Codex dev-log with decision summary and publish demo link
- Lead README with the rendered class report
- Label diagnosis lift in percentage points
- Port dark report design into renderer
- Lead README with judge-facing evidence
- Polish offline judge-facing report
- Broaden cached rule validation coverage
- Document offline baseline verification
- Harden clean checkout audit
- Polish cached sample report
- Polish report and complete bounded validation
- Expose factory retrieval misses and calibration results
- Maximize hybrid retrieval before full benchmark
- Improve retrieval with hypothesis ablation
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### AGENTS.md

```markdown
# ClassLens — Codex build guide

You are Codex building **ClassLens** (this repo). Self-contained; do not depend on anything outside this folder.

## What you are building
A benchmarked, self-verifying Algebra-1 misconception engine. CLI in, class report out (static HTML + editable Markdown). Full spec + data flow: `ARCHITECTURE.md`. One-line + rationale: `README.md`. It has five parts (canonical names — do not rename):
1. **diagnosis engine** — local-embedding retrieval + GPT-5.6 rerank/verify -> named misconception per wrong answer (grounded in the Eedi taxonomy).
2. **benchmark** — held-out real Eedi ablation: embedding-only vs +GPT-5.6-rerank; report accuracy@1/@3. This measures that GPT-5.6 is doing real work.
3. **executable misconceptions** — the top ~15 computable Algebra-1 misconceptions as buggy SymPy transforms; validated by whether running the rule on real Eedi stems reproduces the real human-authored distractor.
4. **factory** — GPT-5.6 generates a new item stem for a misconception; SymPy computes the correct answer; the buggy rule generates the distractor; round-trip verification (blind re-diagnosis) accepts/rejects.
5. **recheck loop + report** — one real cycle (diagnose -> verified recheck quiz -> re-diagnose), rendered to an HTML report + Markdown reteach plan.

## Rules that matter most
- **This is the primary build thread.** Build the core here so the `/feedback` Session ID represents the real work. Run `/feedback` near submission and give the user the Session ID.
- **GPT-5.6 must be meaningful at runtime, and PROVEN so.** The benchmark ablation (embedding-only vs +GPT-5.6) is the proof — build it and report the lift. GPT-5.6 roles: rerank/verify, stem->SymPy extraction, factory item generation, reteach, class summary, broken-question detection. Confirm the exact model id at https://developers.openai.com/api/docs/guides/latest-model?model=gpt-5.6
- **Three measured numbers are the point** — surface them in the report and README: (1) diagnosis accuracy@1/@3 + GPT-5.6 lift; (2) executable-misconception distractor-reproduction rate; (3) factory round-trip acceptance rate. These are real numbers on real Eedi data, not estimates.
- **Never commit an API key.** Read `OPENAI_API_KEY` from env / gitignored `.env` (`python-dotenv`). `.env.example` holds a placeholder only. This repo is public.
- **Keep a running dev-log** in `README.md` -> "How Codex was used" and "How GPT-5.6 is used": where Codex accelerated the work, key decisions, how GPT-5.6 is integrated. Required by the hackathon; update as you build, not at the end.
- **Commits: never add `Co-Authored-By` or any AI-attribution trailer.** (Documenting Codex/GPT-5.6 usage in `README.md` is separate and required — keep it.)
- **Stay in scope.** Build only the pipeline in `ARCHITECTURE.md`. Out of scope: other subjects; any GUI/web/mobile app; LMS integrations; auth; hosting; OCR; storage/dashboards; simulated-semester / longitudinal mastery tracking / material-effectiveness attribu
[truncated — 3679 more characters]
```

### ARCHITECTURE.md

```markdown
# ClassLens — architecture

CLI in, class report out (HTML + Markdown). All modules live in `src/classlens/`. Module boundaries below are a suggestion; restructure if a cleaner design emerges, but keep concerns separated and keep the **diagnosis engine reusable** — the benchmark, the factory round-trip, and the recheck cycle all call it.

## Data flow
```
                 questions.csv + responses.csv
                          │  ingest.py
                          ▼
                 [Question], [StudentResponse]
                          │  taxonomy.py  (load Eedi; build + cache local embedding index)
                          ▼
   ┌── diagnosis engine (reused everywhere) ─────────────────────────┐
   │  for each distinct (question, wrong-answer):                     │
   │    mapper.py: hybrid local top-25 candidates → GPT-5.6 rerank   │
   │    verify.py: SymPy correctness (slip vs misconception)          │
   │  → MisconceptionMatch                                            │
   └─────────────────────────────────────────────────────────────────┘
        │                         │                         │
        ▼                         ▼                         ▼
 profile.py               benchmark.py               factory.py
 (aggregate → ClassProfile) (held-out Eedi:           (GPT-5.6 stem → SymPy answer →
        │                    acc@1/@3, embed vs        buggy_rules distractor →
        ▼                    +GPT-5.6 lift)            round-trip re-diagnose → accept/reject)
 broken_questions.py                                          │
        │                                              buggy_rules.py
        ▼                                              (executable misconceptions;
 reteach.py (GPT-5.6: worked example +                  validate: reproduce real
 counterexample + practice + recheck item               Eedi distractor)
 drawn from the verified factory)                             │
        │                                                     │
        └──────────────► recheck loop (diagnose → verified recheck quiz → re-diagnose)
                          │  report.py
                          ▼
                 report.html  +  reteach.md
```

## Modules
- `models.py` — dataclasses: `Question`, `StudentResponse`, `Misconception`, `MisconceptionMatch`, `ClassProfile`, `BrokenQuestionFlag`, `ReteachPlan`, plus new: `BuggyRule`, `RuleValidationResult`, `GeneratedItem`, `FactoryResult`, `BenchmarkResult`. Extend as needed.
- `llm.py` — thin GPT-5.6 client wrapper: `complete()` for reasoning/generation. Reads `OPENAI_API_KEY`. Disk cache keyed by input hash. `temperature=0` where determinism matters. Confirm model id from the GPT-5.6 docs.
- `ingest.py` — parse questions + responses CSV into dataclasses. MC (chosen option) is the reliable core; accept an optional free-response/explanation column.
- `taxonomy.py` — load the Eedi misconception list; compute + cache **local** embeddings (`.npy`); `retrieve(query, k)` re
[truncated — 5113 more characters]
```

### requirements.txt

```
numpy==2.5.1
openai==2.45.0
pandas==3.0.3
pytest==9.1.1
python-dotenv==1.2.2
scikit-learn==1.9.0
sentence-transformers==5.6.0
sympy==1.14.0
tqdm==4.68.4

```

### pyproject.toml

```
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"

[project]
name = "classlens"
version = "0.0.0"
description = "Class-level Algebra-1 misconception diagnosis and reteach planning (OpenAI Build Week)."
requires-python = ">=3.10"

[tool.setuptools.packages.find]
where = ["src"]

[tool.pytest.ini_options]
pythonpath = ["src"]

```

### src/classlens/cli.py

```python
"""ClassLens CLI.

Run:
    python -m classlens diagnose \
        --responses examples/sample_class.csv \
        --questions examples/sample_questions.csv \
        --out report.html
    python -m classlens bench
    python -m classlens validate-rules
    python -m classlens factory --misconception <id> --n 5

Every API-touching command honors ``--limit`` and ``--dry-run``. See ``ARCHITECTURE.md`` for
the module map and ``AGENTS.md`` for the definition of done.
"""

from __future__ import annotations

import argparse
import json
import os
import random
from dataclasses import asdict
from pathlib import Path

from .buggy_rules import registry, validate, validation_buckets
from .broken_questions import fallback_review_flag, flag_broken, verify_broken_flag
from .costs import summarize_usage
from .eedi_extraction import extract_item
from .factory import generate_items
from .ingest import load_questions, load_responses
from .benchmark import run_benchmark, run_retrieval_ablation
from .mapper import map_responses
from .models import BenchmarkResult, Misconception, ReteachPlan
from .profile import build_profile, reteach_priority
from .report import render_report
from .reteach import generate_reteach
from .statistics import conservative_lift_interval, wilson_interval
from .taxonomy import Taxonomy


def _seed_everything(seed: int) -> None:
    random.seed(seed)
    try:
        import numpy as np

        np.random.seed(seed)
    except ImportError:
        pass


def cmd_diagnose(args: argparse.Namespace) -> int:
    _seed_everything(args.seed)
    questions = {question.question_id: question for question in load_questions(args.questions)}
    responses = load_responses(args.responses)
    if args.limit is not None:
        responses = responses[: args.limit]
    unknown_question_ids = sorted({response.question_id for response in responses} - set(questions))
    if unknown_question_ids:
        raise ValueError(
            "Responses refer to question IDs not present in the questions CSV: "
            + ", ".join(unknown_question_ids)
        )
    taxonomy_path = Path("data/misconception_mapping.csv")
    if not taxonomy_path.exists():
        taxonomy_path = args.questions.parent / "misconceptions.csv"
    taxonomy = Taxonomy.load(taxonomy_path)
    distinct_pairs = len(
        {(response.question_id, response.chosen_answer, response.explanation) for response in responses}
    )
    # The GPT path runs the benchmark-selected best arm: one three-hypothesis batch plus one
    # rerank per distinct wrong answer; correct answers and slips short-circuit locally.
    estimated_calls = (
        1
        if args.local_only and args.gpt_reteach
        else 0
        if args.local_only
        else distinct_pairs * 2
    )
    offline = os.environ.get("CLASSLENS_OFFLINE") == "1"
    if offline:
        estimated_calls = 0
    if args.dry_run:
        mode = "cache-only, $0 live API" if offline else f"up to {estimated_calls} GPT-5.6 calls"
        print(f"Dry run: {len(responses)} responses, {len(taxonomy.misconceptions)} taxonomy labels, {mode}.")
        return 0
    if args.local_only:
        print(
            "Estimated GPT-5.6 calls: up to "
            f"{estimated_calls} cached reteach-plan generation; local retrieval and SymPy checks are free."
        )
    elif offline:
        # A spend estimate is meaningless once the offline guard has zeroed it out; report the
        # cached work being replayed instead, which is what makes the run reproducible.
        print(
            f"Replaying cached GPT-5.6 responses ({distinct_pairs} hypothesis batches + "
            f"{distinct_pairs} reranks, plus reteach): deterministic, $0."
        )
    else:
        print(
            f"Estimated GPT-5.6 calls: up to {estimated_calls} best-arm calls "
            f"({distinct_pairs} hypothesis batches + {distinct_pairs} reranks) plus one reteach "
            "and any candidate broken-question reviews; cached matches cost $0 on rerun."
        )
    matches = map_responses(responses, questions, taxonomy, use_rerank=not args.local_only)
    n_students = len({response.student_id for response in responses})
    profile = build_profile(matches, n_students)
    candidate_flags = [
        (question, candidate)
        for question_id, question in questions.items()
        if (candidate := flag_broken(question, profile.per_question.get(question_id, []))) is not None
    ]
    confirmed_flags = (
        []
        if args.local_only
        else [
            confirmed
            for question, candidate in candidate_flags
            if (confirmed := verify_broken_flag(question, profile.per_question[question.question_id], candidate)) is not None
        ]
    )
    if not confirmed_flags:
        fallback = fallback_review_flag(questions, profile.per_question)
        if fallback is not None:
            confirmed_flags = [fallback]
    reteach = []
    excluded_question_ids = {flag.question_id for flag in confirmed_flags}
    ranked_reteach, _ = reteach_priority(profile, excluded_question_ids)
    if ranked_reteach:
        top_misconception = ranked_reteach[0][0]
        verified_recheck = _first_verified_recheck(top_misconception.misconception_id)
        if args.local_only and not args.gpt_reteach:
            reteach = [_offline_reteach(top_misconception, verified_recheck)]
        else:
            reteach = [generate_reteach(top_misconception, recheck_item=verified_recheck)]
    recheck_quiz = _load_recheck_quiz()
    render_report(
        profile=profile,
        reteach=reteach,
        recheck_quiz_md=recheck_quiz,
        benchmark=_load_benchmark(),
        out_html=args.out,
        broken_questions=confirmed_flags,
        questions=questions,
        misconceptions={item.misconception_id: item for item in taxonomy.misconceptions},
        local_only=args.local_only,
        gpt_reteach=args.gpt_reteach,
        validation_summary=_load_validation_summary(),
        factory_summary=_load
[truncated — 23653 more characters]
```

### tests/test_verify_latex.py

```python
"""The symbolic verifier must safely decline unconverted LaTeX."""

from __future__ import annotations

from classlens.models import Question, StudentResponse
from classlens.verify import is_answer_wrong


def test_symbolic_grounding_declines_latex_without_crashing() -> None:
    question = Question("q1", "Compute.", "A", {"A": r"\( \frac{1}{2} \)", "B": r"\( 6 \)"})
    response = StudentResponse("s1", "q1", "B")
    assert is_answer_wrong(response, question) is None

```

### tests/test_statistics.py

```python
"""Tests for the benchmark confidence-interval helpers."""

from classlens.statistics import conservative_lift_interval, wilson_interval


def test_wilson_interval_bounds_a_proportion() -> None:
    lower, upper = wilson_interval(35, 100)
    assert lower < 0.35 < upper
    assert round(lower, 3) == 0.264
    assert round(upper, 3) == 0.447


def test_conservative_lift_interval_excludes_zero_for_clear_lift() -> None:
    lower, upper = conservative_lift_interval(70, 20, 100)
    assert lower > 0
    assert upper > lower

```

### tests/test_eedi_extraction.py

```python
"""Tests for cached Eedi extraction parsing."""

from __future__ import annotations

from classlens.eedi_extraction import extract_item


def test_extraction_preserves_label_and_parses_model_json(monkeypatch: object) -> None:
    monkeypatch.setattr(
        "classlens.eedi_extraction.complete",
        lambda *args, **kwargs: '{"expression":"2+3*4","distractor":"20"}',
    )
    extracted = extract_item(
        {"misconception_id": "1507", "question_text": "Evaluate.", "wrong_answer": "20"}
    )
    assert extracted == {"misconception_id": "1507", "expression": "2+3*4", "distractor": "20"}

```

### tests/test_broken_questions.py

```python
"""Tests for conservative broken-question detection."""

from __future__ import annotations

from classlens.broken_questions import flag_broken
from classlens.models import Misconception, MisconceptionMatch, Question


def test_distributed_low_confidence_matches_flag_question() -> None:
    question = Question("q1", "Which?", "A")
    matches = [
        MisconceptionMatch("s1", "q1", "B", Misconception("1", "one"), 0.2),
        MisconceptionMatch("s2", "q1", "C", Misconception("2", "two"), 0.3),
        MisconceptionMatch("s3", "q1", "D", Misconception("3", "three"), 0.4),
    ]
    assert flag_broken(question, matches) is not None

```

### tests/test_smoke.py

```python
"""Smoke tests: the package imports and the CLI parser builds.

Real logic tests are added in Codex alongside each pipeline stage.
"""

from classlens.cli import build_parser


def test_parser_builds() -> None:
    parser = build_parser()
    args = parser.parse_args(
        ["diagnose", "--responses", "r.csv", "--questions", "q.csv"]
    )
    assert args.command == "diagnose"
    assert str(args.out) == "report.html"


def test_subcommands_build() -> None:
    parser = build_parser()
    for cmd in ("bench", "validate-rules"):
        assert parser.parse_args([cmd]).command == cmd
    factory_args = parser.parse_args(["factory", "--misconception", "m1"])
    assert factory_args.command == "factory"
    assert factory_args.n == 5

```

[38 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]