TruthLens

🌍

Inspiration

Over 52% of U.S. adults get news from social media, where misinformation spreads rapidly (Pew Research). Political content today is shaped by polarization, AI-generated misinformation, and narrative framing. Most tools try to declare what’s “true” or “false.” TruthLens reveals narrative structure instead. We quantify: Cross-source semantic agreement Narrative divergence Emotional intensity Source diversity 🧠 What It Does Paste a headline or URL and TruthLens: Scrapes parallel coverage across publishers Generates 768-dimensional semantic embeddings Clusters articles using Elasticsearch kNN Computes a transparent confidence score Visualizes narrative groups and bias distribution The dashboard includes: ⚙️ How We Built It 1️⃣ Bright Data SERP scraping Article HTML extraction Parallel coverage retrieval 2️⃣ Jina Embeddings (v3) 768-dimensional vectors Semantic representation of full article text 3️⃣ Elasticsearch (Elastic Cloud) dense_vector storage kNN vector search Similarity scoring Greedy clustering Metric aggregation 4️⃣ Deterministic Scoring Fully transparent. No opaque LLM reasoning. 🚧 Challenges Choosing the right vector similarity threshold Handling inconsistent article HTML structures Designing a neutral, defensible scoring formula Avoiding political labeling while preserving usefulness 🏆 Accomplishments Built a full ingest → embed → index → cluster → score → visualize pipeline Deep integration with Elasticsearch vector search Real-time parallel media coverage analysis Transparent, reproducible scoring system Production-ready dashboard UI 📚 What We Learned Vector search is powerful for narrative detection Agreement across independent sources is measurable Emotional intensity often correlates with divergence Deterministic AI builds trust 🚀 What’s Next 🌐 World heatmap of coverage origins 📈 Historical divergence tracking 🔔 Narrative shift detection 🧩 Chrome extension overlay 🔧 Full Technology List Frontend Next.js (App Router) React Tailwind CSS Lucide Icons Backend Next.js API Routes TypeScript Search & Storage Elasticsearch (Elastic Cloud) dense_vector kNN search Vector similarity scoring Index mappings Google Cloud Aggregations Embeddings Jina Embeddings v3 (768 dimensions) Data Acquisition Bright Data API SERP API Web Scraper API Analysis & Scoring Custom sentiment analysis logic Greedy threshold clustering algorithm Deterministic confidence scoring formula Elastic Cloud