narcore

A self-improving undercover narcotics investigator for social media.

A Browserbase agent infiltrates logged-in Instagram and negotiates a real deal in a live war room, and every bust it closes is written into Redis memory + a vector corpus that makes the next operation smarter. All on one Redis.

Semantic detection · autonomous undercover negotiation · memory that compounds

▶️ Watch the 3-minute demo

---

⚡ The idea

Drug advertising left the dark web years ago. It's on Instagram, X, TikTok, and Telegram now, hidden in plain sight behind coded language, emojis, misspellings, and acronyms. The instant a keyword filter learns a term, the seller rotates to a new one. This is semantic drift, and static blocklists are always one step behind it.

Narcore closes the loop. It detects coded ads by meaning (not keywords), lets an analyst teach it new slang with one click, and then sends an autonomous Browserbase operative undercover into the seller's DMs to confirm a deal and a meeting location. Every confirmed bust feeds two Redis systems (a vector corpus and an Iris agent memory), so the next operation starts smarter.

It gets better at catching dealers every time it catches one.

---

🎯 The problem

Sellers don't write "selling oxycodone." They write:

"restockd 🔵 m30z, hmu for PUFF 🍃, cashapp ready, signal me 📲"

• Coded vocabulary that mutates weekly (blues, m30, perc, zaza, plug…).
• Emoji & symbol substitution (🍃 💊 🔵 👃) that slips past text filters.
• Hand-offs to encrypted apps (telegram, signal) to move the deal off-platform.

A blocklist can only catch what it already knows. By the time a human adds a new term, the seller has invented three more. You cannot keyword-match your way out of an adversarial, drifting vocabulary. You have to match meaning, and you have to keep learning.

---

🔁 How narcore works: one loop, one Redis

Narcore isn't a pile of features. It's a single self-reinforcing flywheel:

                ┌─────────────────────────── ONE REDIS ───────────────────────────┐
                │                                                                  │
  Instagram ───▶│   vector corpus (detection)      ◀── R1 writes the seller's     │
 (Browserbase)  │   agent memory (Redis Iris)      ◀── R2 pins each confirmed bust │
       ▲        │   operation state + live streams                                 │
       │        └──────────────────────────────────────────────────────────────-──┘
       │                          ▲                              │
       │   B2 consistent identity │                              │ R2 recall primes
       │                          │                              ▼ the next operation
       └─────────  the operative (Stagehand · B3 self-healing DM-open)  ───────────┘

  THE LOOP:  Browserbase produces field intel ──▶ Redis learns ──▶ the next op is smarter

The end-to-end workflow

#	Stage	What happens
1	Detect	A Browserbase + Stagehand fleet scrapes posts → each is embedded (nomic, 768-d) → a Redis vector KNN finds the nearest known drug-slang vector → semantic score + corroborating heuristics → flagged if risk ≥ 0.7 (shown as 70 on the UI's 0–100 scale).
2	Analyst learning loop	An analyst clicks Approve on a real ad → the caption is re-embedded and added to the corpus (corpus:approved:*). Future paraphrases of that term now flag automatically.
3	Operative negotiation	Analyst clicks Engage → a Browserbase operative opens the seller's DM and negotiates as a buyer, confirming Deal ✓ and Location ✓ turn by turn in a live war room.
4	Field-intel learning (R1)	On a confirmed bust, Claude extracts the slang the seller actually used → embedded → corpus:field:* in the same vector index → pending posts re-scored → a live stream:field-intel ticker fires.
5	Lead summary & export	One click produces a court-ready report: handle, platform, detected code-words, risk breakdown, the matched term, and an LLM-drafted outreach email, exportable as .md/.txt.
6	Cross-operation memory (R2)	The bust is pinned to Redis Iris agent memory. Before the next operation, the operative recalls what worked against similar sellers and walks in already primed.

In one breath: scrape → score by meaning → analyst approves → system learns → paraphrase now flags → operative DMs the seller → deal + location confirmed → field slang learned → next operative is smarter.

---

🖼️ Screenshots

Command Center: the ranked lead queue beside a live undercover operative working a DM in real time.

Fleet / War Room (/agents): parallel Browserbase browsers scanning Instagram hashtags, each in a live-view tile	Lead detail: the risk breakdown, nearest known term, and the heuristic hits behind every flag

---

🏗️ Architecture

                         ┌──────────────────────────────────────────────┐
                         │           Next.js 16 (App Router)            │
   Browser ◀────────────▶│  Frontend: dashboard · war room · feed ·     │
                         │            operations · memory · drift viz   │
                         │  Backend:  Route Handlers  (/api/*)          │
                         └───────┬───────────────┬───────────────┬──────┘
                                 │               │               │
                  embed(caption) │      KNN /    │   negotiate / │
                                 ▼      state    ▼   extract     ▼
                  ┌──────────────────┐  ┌──────────────────┐  ┌─────────────────────┐
                  │  Embedding side- │  │   Redis Stack    │  │  Browserbase +       │
                  │  car (llama.cpp) │  │  ▸ idx:corpus    │  │  Stagehand           │
                  │  nomic-v2-moe    │  │  ▸ idx:posts     │  │  ▸ scraper fleet     │
                  │  768-d /v1/embed │  │  ▸ op:* state    │  │  ▸ undercover op     │
                  └──────────────────┘  │  ▸ stream:field  │  └──────────┬──────────┘
                                        │  ▸ Iris memory   │             │
                                        └──────────────────┘   Anthropic │ Claude
                                                                  (brain) ▼
                                                          negotiate · extract slang · draft outreach

Data flow (one line):

scraper ─► POST /api/ingest ─► embed caption ─► Redis vector KNN ─► score ─► store post
dashboard (SWR, polls ~3s) ◄── GET /api/posts ◄── Redis
analyst Approve ─► re-embed ─► add vector to corpus     ← the learning loop
operative confirms ─► extract slang ─► corpus:field:* (R1) + pin Iris memory (R2)

Everything routes through Redis. The scraper is a pure HTTP client. It knows nothing but POST /api/ingest, the same front door every data source uses.

---

🔴 Redis: vectors, memory, state, and streams

Redis here is not a cache. It's the vector search engine, the agent's long-term memory, the live event bus, and the system of record, all in one box. The corpus and the agent memory even share one embedding family (nomic, 768-d), so "similar slang" and "similar tactics" live in the same vector space.

R1: the detector learns from every bust

When an operation is confirmed, narcore reads the slang the seller actually used and teaches it to the detector, live.

1. One Claude call extracts the coded terms from the seller's messages.
2. A provenance gate keeps a term only if Claude is confident and the quote is genuinely in the seller's own words, so the agent can never teach itself made-up words (no self-poisoning).
3. Survivors are embedded and written as corpus:field:* vectors into the same idx:corpus index the detector already searches, instantly raising the risk score of matching posts.
4. Pending posts are re-scored, and a Redis Stream (stream:field-intel) announces "operative learned 'X' → N posts re-flagged" to the war-room ticker.

✅ Proof: 15/15 live checks. Learning one term raised a pending post's risk from 11.9 to 82.5 (0–100 scale) and flipped it to flagged. See src/lib/field-intel.ts.

R2: the operative remembers across operations (Redis Iris / Agent Memory)

The operative has long-term episodic memory, powered by the official Redis Agent Memory Server (Iris) running on the same Redis.

• Recall (turn 0): narcore semantically searches memory and injects the top hits into the negotiator's prompt as PRIOR_INTEL, e.g. "opener that worked," "this seller responds to casual texts."
• Pin (on confirm): a durable episodic memory is written, covering drug, meeting spot, opener, and turn count.
• Cross-seller learning: a lesson learned against an oxycodone dealer can be recalled for a different seller of a different drug, because memory is indexed by meaning, not by handle.
• One embedding family, one box, no external key. Iris reuses our local nomic embedder.
• Fail-open: if the memory server is down, recall returns nothing and the operative runs exactly as before. Memory is an amplifier, never a dependency.

✅ Proof: 7/7 live checks: pin → worker indexes → semantic recall returns it for a different seller, and fail-open verified with the server down. See src/lib/agent-memory.ts.

🛠️ Engineering note: Iris embeds via LiteLLM, which sends a null field the local embedder rejects. We root-caused it and added a tiny null-stripping proxy (infra/embedding-proxy/) so Iris can reuse our embedder: one embedding family, zero external cost.

What Redis is doing (data structures)

Capability	Redis feature	Keys	Notes
Real-time detection	RediSearch vector KNN (FLAT · COSINE · FLOAT32 · 768-d)	idx:corpus over corpus:{seed,approved,field}:*	Exact NN. Corpus is small, so FLAT is fast and accurate.
Post store / feed	RediSearch (numeric + tag)	idx:posts over post:*	Query by risk, platform, approval status.
Operation state	Hashes + Lists	op:{id}, op:{id}:messages	Negotiation survives across requests; transcript is append-only.
Learning ticker	Streams (XADD / XREVRANGE, MAXLEN ~200)	stream:field-intel	Live war-room feed of what the system just learned.
Self-pruning corpus	TTL, refreshed on every match	corpus:approved:*, corpus:field:*	14-day sliding window; terms that keep catching posts stay alive, stale ones auto-evict. No cron job.
Agent memory	RedisVL semantic index (Iris)	operative:episodic:*	Cross-operation recall, namespace-isolated.

Why Redis specifically: sub-millisecond KNN at scrape time · a native append-only event stream for the live ticker · zero-cron TTL eviction that keeps the corpus fresh · semantic memory that persists across restarts and is searched by meaning · and one dependency holding vectors, state, memory, and events together.

---

🟦 Browserbase: an undercover agent, not just a scraper

Narcore doesn't merely read Instagram. It infiltrates it, with a consistent, checkpointed, self-improving operative running from law-enforcement-style infrastructure. Browserbase is what makes that realistic in a live run without external CAPTCHA APIs or manual proxy rotation.

The operative

A single keep-alive Browserbase session driven by Stagehand with a Claude brain:

1. Navigates to the seller's profile, handles checkpoints/login walls.
2. Opens the DM thread, then loops: extract() the thread → classify sender → negotiate() (one Claude call → structured message) → sendMessage() (typed & verified to land) → waitForReply().
3. Updates Deal ✓ / Location ✓ each turn; on confirmation it triggers R1 (learn the slang) and R2 (pin the memory).

See scraper/operative-agent.ts and src/lib/operative-brain.ts.

The War Room

A fleet of parallel cloud browsers (scale it to your Browserbase plan's concurrency) scans real Instagram hashtags at once, each streamed live into the UI, each advancing independently. A stuck agent shows Checkpoint/Blocked with a Take over link (interactive live view) and never blocks the others. A break-glass fixture keeps things moving if a live session stalls.

Browserbase features in play

Feature	Why it matters here
Residential proxies (geolocation-targeted)	Instagram checkpoints datacenter IPs instantly; a residential egress in the account's country looks human.
Persistent logged-in contexts	One manual pnpm ig:login, then every session starts already authenticated, the #1 defense against login checkpoints.
Session keep-alive + REQUEST_RELEASE	Bridges the gap between creating the session (to render the live view) and Stagehand attaching to it.
Automatic CAPTCHA solving (solveCaptchas)	Instagram's checkpoints are solved, not dodged, with no external CAPTCHA service.
Live-view CDP stream	Read-only, embeddable browser feed → the war-room tiles and the operation view. Anyone can watch the automation in real time.
Verified Browsers · advanced stealth · OS pinning	Defeat headless detection. (Enterprise-gated; guarded so a misconfig can't break session creation. See B2.)

B2: one consistent browser identity

Every session (the one-time login and every later run) is created through a single chokepoint (createIgSession() in src/lib/browserbase.ts), so Instagram sees an identical fingerprint (OS, region, proxy country, viewport, CAPTCHA solving). pnpm ig:login snapshots that fingerprint into Redis; runs verify they still match and surface an "identity match" badge in the war room.

✅ Proof: 14/14 live checks. It also caught a real Browserbase 400 (a non-Linux OS needs the Enterprise plan), now guarded so a misconfig can't break session creation. See src/lib/session-identity.ts.

B3: self-healing DM-open

The single most fragile step (finding and opening the DM through Instagram's shifting UI) can be delegated to a one-line stagehand.agent() (DOM mode) that figures out its own path past popups and layout changes. It reuses our existing model + key (no extra cost), and our own /direct/t/… check decides success, not the agent's say-so. Off by default behind OPERATIVE_DM_OPEN_STRATEGY=agent: a deliberate choice about when to hand over autonomy, with the auditable "send message" loop kept deterministic.

Why Browserbase over local Playwright: local Playwright is a headless datacenter browser with no proxy, no persisted login, no CAPTCHA solving, and no embeddable live view, which are exactly the four things that get you checkpointed on real Instagram before you start.

---

🧰 Tech stack

Layer	Technology
Frontend	Next.js 16.2.9 (App Router) · React 19.2.4 · TypeScript 5 · Tailwind CSS v4 (tokens-only, OKLCH) · shadcn/ui (Base UI) · SWR ^2.4 · Sonner
Backend	Next.js Route Handlers (/api/*), no separate server · Zod ^4 validation
Vectors + state	Redis Stack (RediSearch + RedisVL) · redis (node-redis) ^6
Embeddings	nomic-embed-text-v2-moe (768-d) via self-hosted llama.cpp (OpenAI-compatible /v1/embeddings); hosted/mock fallbacks
Browser automation	Browserbase @browserbasehq/sdk ^2.10 · @browserbasehq/stagehand ^3.6
LLM	Anthropic Claude (@anthropic-ai/sdk ^0.105): operative brain, slang extraction, outreach drafting
Agent memory	Redis Agent Memory Server (Iris) + worker, via docker-compose
Viz	umap-js (2D projection of the corpus)

---

🚀 Quickstart

Prerequisites: Docker Desktop (WSL2), Node + pnpm. Windows: in PowerShell use curl.exe (not curl) for POSTs, or run them in Git Bash. GET URLs open in a browser.

# 1. Install
pnpm install

# 2. One-time: fetch the embedding model (~490 MB, git-ignored, bind-mounted into the container)
mkdir -p infra/embedding/models
curl -L -o infra/embedding/models/nomic-embed-text-v2-moe-q8_0.gguf \
  https://huggingface.co/ggml-org/Nomic-Embed-Text-V2-GGUF/resolve/main/nomic-embed-text-v2-moe-q8_0.gguf

Create .env.local in the repo root:

REDIS_URL=redis://localhost:6379
EMBEDDING_MODE=auto
EMBEDDING_API_URL=http://localhost:8080/v1/embeddings
RISK_THRESHOLD=0.7

# Optional: live Browserbase scraping / the undercover operative
BROWSERBASE_API_KEY=your_key
BROWSERBASE_PROJECT_ID=your_project_id
# BROWSERBASE_CONTEXT_IDS=ctx_a,ctx_b   # from `pnpm ig:login`

# Optional: real LLM outreach, negotiation, and field-intel (else a template/mock is used)
ANTHROPIC_API_KEY=your_key

# 3. Bring up Redis + embedder (+ the Iris agent-memory stack: api / worker / proxy)
docker compose up -d

# 4. Run the app
pnpm dev                                          # http://localhost:3000

# 5. Seed the detection corpus through the server (uses the same real embeddings the API uses)
curl.exe -X POST http://localhost:3000/api/seed   # -> {"loaded":45,"skipped":0}

# 6. (For the R2 memory walkthrough) pre-seed operative memory so the first op shows "Prior intel"
pnpm seed:memories

Open http://localhost:3000/dashboard. Health check: GET /api/health returns {"ok":true,"redis":true,"embeddings":true,"corpusSize":45,...,"modelVersion":"nomic-embed-text-v2-moe@768"}.

Full bring-up, the Browserbase login flow (pnpm ig:login), troubleshooting, and a concepts deep-dive live in RUNBOOK.md.

---

🎬 Try it in 60 seconds

The headline loop (Browserbase × Redis):

1. Open /dashboard and click Engage on a flagged demo lead.
2. Watch the war room: the Browserbase operative opens the DM and negotiates; Deal ✓ and Location ✓ light up.
3. On confirm, the field-intel ticker fires (stream:field-intel) and the bust is pinned to Iris memory.
4. Start the next operation: the war room shows a "Prior intel used" card. It's already smarter.

The detection + learning mini-walkthrough (no Browserbase needed):

pnpm post-mock                                   # a coded ad appears, scored, in /dashboard
# flip Approve on a flagged row  (corpus:approved:* appears in RedisInsight @ :8001)
pnpm post-mock                                   # ingest a paraphrase -> it now scores higher
curl.exe -X POST http://localhost:3000/api/rescore   # re-score pending posts vs the grown corpus

Offline-safe scrape (always works): POST /api/scrape -d '{"live":false}'.

---

✅ Verify it yourself

Every deepening was verified end-to-end against live infrastructure (real Redis, the local nomic embedder, Claude, Browserbase), not just type-checked:

npx tsx scripts/test-field-intel.ts     # R1: field-intel learning loop   (15 checks)
npx tsx scripts/test-agent-memory.ts    # R2: Iris cross-operation recall  (7 checks)
npx tsx scripts/test-identity.ts        # B2: consistent session identity (14 checks)

---

🗂️ Project structure

gov-dr-ai/
├── src/
│   ├── app/                    # Next.js App Router
│   │   ├── (dashboard)/        # dashboard · agents (war room) · operations · command · memory
│   │   ├── feed/               # synthetic social feed (the scrape target for dev/demo)
│   │   ├── semantic-drift/     # 2D corpus visualization, live
│   │   └── api/                # Route Handlers: ingest · posts · corpus · rescore ·
│   │                           #   field-intel · memory · operations · agents · seed · scrape
│   ├── components/             # dashboard · operations · landing · ui (shadcn)
│   └── lib/
│       ├── redis.ts repo.ts            # Redis client, key schemas, KNN, posts/corpus CRUD
│       ├── field-intel.ts              # R1: learn slang from confirmed busts
│       ├── agent-memory.ts             # R2: Iris recall + pin
│       ├── browserbase.ts session-identity.ts   # B2: session chokepoint + fingerprint drift
│       ├── operative-brain.ts operative-allowlist.ts
│       └── scoring.ts embeddings.ts model.ts    # risk = semantic + heuristics
├── scraper/                    # Stagehand flows: operative-agent.ts, scrape.ts, fixtures.ts
├── scripts/                    # seed · ig:login · ig:verify · test-{field-intel,agent-memory,identity}
├── infra/                      # embedding (llama.cpp) + embedding-proxy (Iris null-strip)
├── data/                       # seed-terms.json (45 DEA/SAMHSA terms) · mock-feed.json
└── docker-compose.yml          # Redis Stack + embedder + Iris (api/worker/proxy)

Deeper docs: SPEC.md (frozen contracts) · OPERATIVE.md (the operative + all four deepenings) · NARCORE-UPGRADES.md (the Redis + Browserbase story) · RUNBOOK.md (bring-up + concepts) · brand.md (design system).

---

🛡️ Responsible use & safety

Narcore is a demonstration of detection and investigative tooling for authorized Trust & Safety and law-enforcement contexts. It is built with safeguards first:

• Synthetic data by default. The dashboard scrapes a built-in synthetic /feed of coded + benign decoy posts (clearly labeled test data). The "real Instagram" mode exists to prove the architecture, not to surveil the public.
• Demo-adversary accounts only. The undercover operative negotiates only with accounts the team controls, and only handles explicitly on an allowlist (enforced in src/lib/operative-allowlist.ts). It will not engage an arbitrary stranger.
• No real transactions, ever. The operative confirms intent and a meeting location for an investigative lead summary. It never sends money, never completes a purchase, and never arranges a real hand-off.
• Throwaway accounts, never personal. Automating real Instagram violates IG's Terms of Service and risks bans, so Browserbase logins must use disposable accounts you own.
• Human-in-the-loop. The corpus only learns from an analyst's explicit Approve, and field-intel is provenance-gated so the system can't teach itself fabricated terms. Every flag is explainable (which term matched, which heuristics fired) and exportable as a court-ready summary.
• Privacy posture. Self-hosted embeddings (no caption ever leaves your box for a third-party embedder); Redis is the single source of truth; secrets stay in .env.local (never committed).

This is investigative tooling, not an enforcement weapon. The goal is to give analysts an explainable, ranked, self-improving queue, and to keep a human in every consequential decision.

---

🧭 Roadmap

• Multi-platform ingestion adapters (X, Telegram, TikTok) behind the same /api/ingest contract.
• Approximate vector index (HNSW) for million-scale corpora.
• Operative tactic A/B testing scored by close-rate, pulled from Iris memory.
• Analyst feedback on outreach drafts to fine-tune the summary voice.

---

🙏 Built with

Browserbase + Stagehand · Redis Stack & the Agent Memory Server (Iris) · Anthropic Claude · Nomic Embed · Next.js / Vercel.