# Project export: Clarity This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: Prompt observability in two lines of code. Gain instant visibility into usage, costs, and performance of your LLM applications. - Devpost: https://devpost.com/software/clarity-yem8pd - GitHub: https://github.com/akshatdotcom/clarity/ - Team: 2 GitHub contributor(s) — Akshat Shah (57 commits), akhilbz (26 commits) ## Devpost submission (written by the team) ### Inspiration "Our AI-powered customer support system was failing 40% of the time. We had no idea why." During a late-night debugging session, we realized something terrifying: our production AI was silently failing on thousands of customer emails. No error logs. No metrics. No way to debug. Just angry customers and a bleeding bank account. Traditional monitoring tools weren't built for LLM prompts. You can't just check CPU usage or error rates- you need to see: Why did GPT-5-nano classify this email wrong? Why are we spending $14,400/month on AI? Which prompt version is actually better? Why is latency spiking at 3 AM? We looked for a solution. Langsmith was too complex. Weights & Biases wasn't designed for production. Existing tools required hours of setup and still didn't answer our questions. So we built Clarity: Prompt observability that just works. ### What it does Clarity is a 2-line integration that gives you complete visibility into your LLM applications: For Developers For Everyone Else A user-friendly dashboard that shows: Real-time request monitoring - See every LLM call as it happens Automatic cost tracking - Know exactly what you're spending ($0.48 per email? Too much!) Instant replay - Re-run any request with different models/prompts Prompt versioning - Track performance across v1, v2, v3... Deep debugging - Full request/response logs with token breakdowns Performance analytics - Latency trends, success rates, model comparison Mock Demo We built SmartMail - a customer support AI that looks perfect... until it doesn't: Try billing email: "I was charged twice" → Works flawlessly Try technical email: "App keeps crashing" → Classification Failed Reveal: This isn't a broken demo- it's a real AI system failing 40% of the time Without Clarity: Developers have no idea why it's failing With Clarity: Instant debugging, root cause analysis, and fix deployment ### How we built it Architecture A full-stack observability platform in 48 hours: 1. Clarity SDK (@clarity/node) TypeScript SDK with strict mode for bulletproof types OpenAI wrapper - Intercepts chat.completions.create() Anthropic wrapper - Intercepts messages.create() Smart batching - Queues logs, flushes every 5 seconds Cost calculator - Hardcoded pricing for GPT-5, GPT-4o, Claude Opus/Sonnet/Haiku Auto-detection - Reads app name from package.json, environment from NODE_ENV Graceful shutdown - Flushes queue on process exit Zero dependencies - Just node-fetch for API calls 2. Web Dashboard (Next.js + React) Real-time log viewer - Server-sent events for live updates Cost analytics - Charts showing spend over time Replay engine - Re-run requests with different parameters Filtering system - By prompt ID, environment, status, date range Prompt comparison - Side-by-side v1 vs v2 metrics 3. SmartMail Demo (Next.js) Intentionally broken classifier (v1.2 fails on technical emails) Multi-model support - GPT-4o-mini, GPT-4o, Claude Sonnet Cost tracking - Shows real costs per email classification Integration showcase - Demonstrates Clarity SDK in action Tech Stack SDK: TypeScript 5.0, Node.js 20, Jest (31 passing tests) Web: Next.js 16, React, Tailwind CSS, Shadcn UI Demo: Next.js, OpenAI SDK 6.7.0, Anthropic SDK 0.67.0 Infra: Neon, PostgreSQL, Vercel (planned) Key Technical Achievements 1. Perfect TypeScript DX 2. Non-blocking Logging Logs never block your LLM calls Background queue with retry logic (max 3 attempts) Errors logged but never thrown Your AI keeps working even if Clarity is down 3. Smart Cost Calculation Model normalization: gpt-4o-2024-08-06 → gpt-4o Cached token support: (uncached × $2.50) + (cached × $1.25) Per-request cost tracking GPT-5 ready (estimated pricing included) 4. Prompt Version Management ### Challenges we ran into 1. TypeScript Type Constraints Problem: Wrapping OpenAI/Anthropic clients broke type inference Solution: Generic wrappers + Object.defineProperty 2. The Streaming Problem OpenAI and Anthropic both support streaming responses. Our wrappers needed to handle both: 3. Cost Calculation Edge Cases Cached tokens: OpenAI's prompt caching reduces costs Model versioning: Handle gpt-4o-2024-08-06 as gpt-4o 4. Race Conditions in Logging Problem: Process could exit before logs flushed Solution: Graceful shutdown handlers 5. The "Demo Must Fail" Paradox SmartMail needed to fail convincingly without looking like our code was broken: Made v1.2 intentionally buggy (catches wrong keywords) Added clear UI states for "Classification Failed" Created realistic error messages Built in fallback behavior (shows error, doesn't crash) 6. SDK Version Compatibility Upgraded to latest SDKs mid-hackathon: OpenAI 4.0.0 → 6.7.0 (major breaking changes) Anthropic 0.20.0 → 0.67.0 (new message format) Had to refactor wrappers for new APIs All 31 tests still passing ### Accomplishments we're proud of 1. The 2-Line Integration We obsessed over developer experience: Most observability tools require: Installing 5+ packages Configuring YAML files Adding instrumentation to every function Learning a complex API Clarity just works. 2. Zero TypeScript Errors Verified with: tsc --noEmit Clean build 31/31 unit tests passing Demo app compiles Full IntelliSense support 3. Production-Ready Cost Tracking Hardcoded pricing for 9 models across 2 providers: OpenAI: GPT-5, GPT-5-turbo, GPT-4o, GPT-4o-mini, GPT-4-turbo, GPT-4, GPT-3.5-turbo Anthropic: Claude Opus 4, Claude Sonnet 4, Claude Sonnet 3.5, Claude Haiku 3.5 Real-world accuracy: 4. The Clarity Demo Approach Built a demo that tells a story: Show perfect AI behavior → Audience relaxed Show failure → Audience thinks "oh no, bug!" Reveal it's intentional → Mind blown Switch to Clarity dashboard → Show the solution Debug in real-time → Prove it works This demo strategy makes Clarity's value instantly obvious. 5. Smart Defaults That Actually Work 6. Comprehensive Documentation Main README with quick start SDK README with full API docs Demo app README with setup guide Integration summary with test results Completion checklist (100% done!) ### What we learned 1. TypeScript Generics Are Powerful Going from this: To this: 2. Developer Experience > Features We cut streaming support to focus on making the basic integration perfect: 2 lines vs 20 lines Zero config vs complex setup Auto-detection vs manual specification Type-safe vs error-prone The result: Clarity is easier to integrate than any competitor. 3. Observability Isn't Just Logging Users don't just want logs—they want answers: "Here are 10,000 request logs" "Your v1.2 classifier fails 40% of the time on technical emails" "Your v1.2 classifier fails 40% of the time on technical emails" "Total cost: $14,400/month" "Total cost: $14,400/month" "You're using GPT-4o for classification. Switch to GPT-4o-mini and save $11,000/month" "You're using GPT-4o for classification. Switch to GPT-4o-mini and save $11,000/month" "Request failed with status 400" "Request failed with status 400" "Input exceeded max tokens. Truncate to <4096 tokens" "Input exceeded max tokens. Truncate to <4096 tokens" 4. The Power of "Just Works" Every time we asked "should this be configurable?" we chose "no": App ID? Auto-detect from package.json Environment? Map from NODE_ENV Batching? 5 seconds is always right Shutdown? Handle automatically Less configuration = more usage. 5. Testing Prevents Disasters Mid-hackathon SDK upgrade could have broken everything: OpenAI 4.0 → 6.7 (major version jump) Anthropic 0.20 → 0.67 (3x version jump) But our 31 unit tests caught every breaking change: 6. Demos Should Tell Stories SmartMail isn't just a tech demo—it's a story: Setup: "Here's a working AI system" Conflict: "Oh no, it's failing!" Crisis: "40% of emails are being mishandled" Resolution: "Clarity shows us exactly why" Happy Ending: "Fixed in minutes, saving thousands" Stories > feature lists. 7. Rapid AI Integration Every developer we talked to said: "We're spending thousands on OpenAI" "We have no idea where the money goes" "Our AI fails randomly" "We can't debug it" This isn't a nice-to-have. This is a must-have. ### What's next Streaming Support Handle Server-Sent Events from OpenAI/Anthropic Log streaming tokens in real-time Calculate costs for partial responses Streaming Support Handle Server-Sent Events from OpenAI/Anthropic Log streaming tokens in real-time Calculate costs for partial responses More Providers Google Gemini wrapper AWS Bedrock support Cohere integration Mistral AI support More Providers Google Gemini wrapper AWS Bedrock support Cohere integration Mistral AI support Dashboard v2 Real PostgreSQL backend (currently mock data) User authentication Team collaboration features API key management Dashboard v2 Real PostgreSQL backend (currently mock data) User authentication Team collaboration features API key management Short Term (1-2 Months) Advanced Analytics Cost forecasting: "At this rate, you'll spend $50K next month" Anomaly detection: "Success rate dropped 20% in last hour" Model recommendations: "Switch to GPT-4o-mini for 80% cost savings" Advanced Analytics Cost forecasting: "At this rate, you'll spend $50K next month" Anomaly detection: "Success rate dropped 20% in last hour" Model recommendations: "Switch to GPT-4o-mini for 80% cost savings" Prompt Optimization A/B testing framework Statistical significance testing Automatic rollback on regression Gradual rollout (10% → 50% → 100%) Prompt Optimization A/B testing framework Statistical significance testing Automatic rollback on regression Gradual rollout (10% → 50% → 100%) Alerts & Notifications Slack integration Email alerts PagerDuty integration Custom webhooks Alerts & Notifications Slack integration Email alerts PagerDuty integration Custom webhooks Medium Term (3-6 Months) Python SDK from clarity import init, wrap_openai init(api_key=os.getenv('CLARITY_API_KEY')) client = wrap_openai(OpenAI()) Python SDK Browser SDK // Works in Next.js, React, Vue, vanilla JS import { wrapOpenAI } from '@clarity/browser'; Browser SDK Evaluation Framework Define test cases Run bulk evaluations Compare model performance Track quality metrics over time Evaluation Framework Define test cases Run bulk evaluations Compare model performance Track quality metrics over time Long Term (6-12 Months) Enterprise Features SSO / SAML authentication Role-based access control Audit logs SOC 2 compliance Enterprise Features SSO / SAML authentication Role-based access control Audit logs SOC 2 compliance Self-Hosted Option Docker deployment Kubernetes helm charts On-premise installation Air-gapped environments Self-Hosted Option Docker deployment Kubernetes helm charts On-premise installation Air-gapped environments AI Insights Automatic prompt improvement suggestions Cost optimization recommendations Quality regression detection Anomaly explanations AI Insights Automatic prompt improvement suggestions Cost optimization recommendations Quality regression detection Anomaly explanations The Vision Clarity becomes the default way to build with LLMs. Just like: Sentry for error tracking Datadog for infrastructure monitoring Stripe for payments Clarity for Prompt observability. Every AI application, from day one, integrates Clarity. Because flying blind isn't an option anymore. Try It Yourself SmartMail Demo Try these emails: "I was charged twice" (works) "App keeps crashing" (fails) Clarity Dashboard SDK Integration Impact For Developers: Debug AI failures in seconds (not days) Ship with confidence Optimize costs without guesswork For Businesses: 40% cost reduction through model optimization 95% success rate (up from 60%) Happy customers who get correct responses For The Market: $50B+ AI market 90% lack observability Early mover advantage Massive TAM Built With TypeScript & Node.js Next.js & React OpenAI SDK 6.7.0 Anthropic SDK 0.67.0 Tailwind CSS & Shadcn UI Jest for testing ## README (from the GitHub repository) # Clarity **Simple Prompt observability that just works.** Clarity gives you instant visibility into your LLM applications with a 2-line integration. See every request, understand your costs, and optimize your prompts—without the complexity. ## Why Clarity? Building with LLMs is like flying blind. When costs spike or quality drops, you're left guessing. Clarity makes your LLM application as debuggable as traditional software. - **2-line setup** - Wrap your OpenAI/Anthropic client and you're done - **Instant visibility** - Every request automatically logged with full context - **Cost tracking** - See exactly where your money goes, per prompt - **Smart defaults** - Zero configuration, just works - **Beautiful UI** - No learning curve, intuitive from day one ## Quick Start ### 1. Install the SDK ```bash npm install @clarity/node ``` ### 2. Initialize and wrap your client ```typescript import { init, wrapOpenAI } from "@clarity/node"; import OpenAI from "openai"; // Initialize once at app startup init({ apiKey: process.env.CLARITY_API_KEY, }); // Wrap your OpenAI client const openai = wrapOpenAI(new OpenAI(), { promptId: "email-classifier", promptVersion: "v2", }); // Use normally - all calls are automatically logged! const response = await openai.chat.completions.create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Classify this email..." }], }); ``` ### 3. View your logs Head to [clarity.dev/dashboard](https://clarity.dev/dashboard) to see your requests in real-time. That's it! 🎉 ## Features ### 📊 Complete Request Logging - Full input/output capture - Token counts and costs - Latency tracking - Error monitoring - Automatic metadata tagging ### 💰 Cost Analytics - Real-time cost calculation - Cost per prompt breakdown - Cost trends over time - Model comparison - Usage by environment (dev/staging/prod) ### 🔄 Replay & Compare - Re-run any request with different parameters - Compare outputs side-by-side - Test different models instantly - Optimize based on real data ### 🔍 Smart Filtering - Filter by prompt ID - Filter by environment - Filter by status (success/error) - Date range selection - Search across all logs ### 🏷️ Prompt Management - Tag prompts with IDs and versions - Track prompt performance - Compare version metrics - Organize by use case ## Development This is a monorepo containing the SDK, web dashboard, and demo app. ### Setup ```bash # Install dependencies npm install # Build the SDK cd packages/sdk-node && npm run build # Run the web dashboard cd packages/web && npm run dev ``` ### Code Quality We maintain strict code quality standards: - **TypeScript** with strict mode enabled - **ESLint** for catching unused imports, variables, and code quality issues - **Prettier** for consistent formatting - **Type checking** on all packages - **GitHub Actions** for automated checks on all PRs Before committing: ```bash npm run format # Format code npm run lint:fix # Fix linting issues npm run type-check # Type check all packages ``` See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines. ## Supported Providers **OpenAI** - ✅ GPT-5 - ✅ GPT-4o, GPT-4o-mini - ✅ GPT-4, GPT-4-turbo - ✅ GPT-3.5-turbo **Anthropic** - ✅ Claude Opus 4 - ✅ Claude Sonnet 4, Sonnet 3.5 - ✅ Claude Haiku 3.5 **Coming Soon** - 🔜 Google Gemini - 🔜 AWS Bedrock ## Smart Defaults Clarity auto-detects everything it can: - **App ID**: Reads from your `package.json` name - **Environment**: Detects from `NODE_ENV` (development → dev, staging → staging, production → prod) - **Batching**: Automatically batches logs for efficiency - **Graceful shutdown**: Flushes logs before process exit ## Configuration The SDK works with minimal configuration, but you can customize: ```typescript init({ apiKey: process.env.CLARITY_API_KEY, appId: "my-custom-app-name", // Optional: override auto-detected app name environment: "prod", // Optional: override auto-detected environment enabled: process.env.NODE_ENV !== "test", // Optional: disable in tests flushInterval: 5000, // Optional: batch flush interval in ms }); ``` ### Wrap Options ```typescript wrapOpenAI(client, { promptId: "email-classifier", // Identify this prompt promptVersion: "v2", // Track versions sessionId: conversationId, // Group related requests route: "/api/classify", // API endpoint name tags: ["production", "important"], // Custom tags metadata: { userId: "123" }, // Custom metadata }); ``` ## Demo App Check out our comprehensive demo app in [`demo-app/`](demo-app/) for complete examples: ```bash cd demo-app npm install cp .env.example .env # Add your API keys npm run dev # Run combined demo npm run openai # OpenAI examples only npm run anthropic # Anthropic examples only ``` The demo showcases: - ✅ 2-line integration - ✅ OpenAI and Anthropic usage - ✅ Multiple models (GPT-4o, Claude, etc.) - ✅ Multi-turn conversations - ✅ Error handling - ✅ Custom metadata and tags - ✅ Cost tracking - ✅ Perfect TypeScript support ## Examples ### Basic Usage ```typescript import { init, wrapOpenAI } from "@clarity/node"; import OpenAI from "openai"; init({ apiKey: process.env.CLARITY_API_KEY }); const openai = wrapOpenAI(new OpenAI(), { promptId: "chat-bot", promptVersion: "v1", }); const response = await openai.chat.completions.create({ model: "gpt-4o", messages: [ { role: "system", content: "You are a helpful assistant." }, { role: "user", content: "Hello!" }, ], }); ``` ### Multi-turn Conversations ```typescript // Group conversation turns with sessionId const openai = wrapOpenAI(new OpenAI(), { promptId: "support-chat", sessionId: conversationId, // Same ID for entire conversation }); // Each turn is logged and grouped for (const turn of conversationTurns) { const response = await openai.chat.completions.create({ model: "gpt-4o", messages: conversationHistory, }); } ``` ### Environment-specific Tracking ```typescript // Development init({ apiKey: process.env.CLARITY_API_KEY, environment: "dev", // Automatically tagged }); // Production init({ apiKey: process.env.CLARITY_API_KEY, environment: "prod", }); // Filter by environment in the dashboard ``` ### Using with Anthropic ```typescript import { init, wrapAnthropic } from "@clarity/node"; import Anthropic from "@anthropic-ai/sdk"; init({ apiKey: process.env.CLARITY_API_KEY }); const anthropic = wrapAnthropic(new Anthropic(), { promptId: "content-generator", promptVersion: "v3", }); const response = await anthropic.messages.create({ model: "claude-sonnet-4", max_tokens: 1024, messages: [{ role: "user", content: "Write a blog post about AI." }], }); ``` ## Dashboard Features ### Request Logs - Real-time log streaming - Detailed request/response inspection - Copy prompts for debugging - Filter and search across all logs ### Analytics - Cost breakdown by prompt - Usage by environment - Request trends over time - Model performance comparison ### Replay - Re-run any request - Test different models - Compare outputs side-by-side - Optimize based on real data ## API Keys Get your API key from the [Clarity dashboard](https://clarity.dev/dashboard/api-keys). Store it securely in your environment variables: ```bash CLARITY_API_KEY=obs_your_key_here ``` ## Documentation - [Quick Start Guide](https://clarity.dev/docs/quickstart) - [API Reference](https://clarity.dev/docs/api) - [Examples](https://clarity.dev/docs/examples) ## Roadmap - [x] OpenAI support - [x] Anthropic support - [x] Cost tracking - [x] Replay feature - [x] Prompt versioning - [ ] Usage analytics (in progress) - [ ] Advanced search (in progress) - [ ] Python SDK - [ ] Alerts and notifications - [ ] Team collaboration - [ ] Google Gemini support - [ ] AWS Bedrock support ## Contributing We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details. ## Support - **Documentation**: [clarity [README truncated for size] ## Detected evidence (automated analysis) Indexed codebase: 110 recognized source files, 514 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - React (technology) — detected in the code - SQL (language) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (120 of 147) ``` .editorconfig .eslintignore .eslintrc.json .github/workflows/code-quality.yml .gitignore .prettierignore .prettierrc.json .vscode/extensions.json .vscode/settings.json CONTRIBUTING.md demo-app/.env.example demo-app/.gitignore demo-app/eslint.config.mjs demo-app/package.json demo-app/README.md demo-app/src/anthropic-demo.ts demo-app/src/index.ts demo-app/src/openai-demo.ts demo-app/src/type-check-demo.ts demo-app/tsconfig.json package.json packages/sdk-demo/.gitignore packages/sdk-demo/eslint.config.mjs packages/sdk-demo/package.json packages/sdk-demo/README.md packages/sdk-demo/src/examples/chatbot.ts packages/sdk-demo/src/examples/comparison.ts packages/sdk-demo/src/examples/simple.ts packages/sdk-demo/src/index.ts packages/sdk-demo/tsconfig.json packages/sdk-node/.gitignore packages/sdk-node/.npmignore packages/sdk-node/eslint.config.mjs packages/sdk-node/jest.config.js packages/sdk-node/package.json packages/sdk-node/README.md packages/sdk-node/src/costs.ts packages/sdk-node/src/index.ts packages/sdk-node/src/observatory.ts packages/sdk-node/src/types.ts packages/sdk-node/src/utils.ts packages/sdk-node/src/wrappers/anthropic.ts packages/sdk-node/src/wrappers/openai.ts packages/sdk-node/tests/costs.test.ts packages/sdk-node/tsconfig.json packages/smartmail-demo/.gitignore packages/smartmail-demo/env.example packages/smartmail-demo/eslint.config.mjs packages/smartmail-demo/next.config.ts packages/smartmail-demo/package.json packages/smartmail-demo/postcss.config.mjs packages/smartmail-demo/README.md packages/smartmail-demo/src/app/api/classify/route.ts packages/smartmail-demo/src/app/api/respond/route.ts packages/smartmail-demo/src/app/api/triage/route.ts packages/smartmail-demo/src/app/globals.css packages/smartmail-demo/src/app/layout.tsx packages/smartmail-demo/src/app/page.tsx packages/smartmail-demo/src/components/ClassificationResult.tsx packages/smartmail-demo/src/components/EmailUpload.tsx packages/smartmail-demo/src/components/ResponsePreview.tsx packages/smartmail-demo/src/lib/models.ts packages/smartmail-demo/src/lib/prompts.ts packages/smartmail-demo/tsconfig.json packages/web/.gitignore packages/web/API_README.md packages/web/components.json packages/web/drizzle.config.ts packages/web/drizzle/0000_sleepy_doctor_doom.sql packages/web/drizzle/meta/_journal.json packages/web/drizzle/meta/0000_snapshot.json packages/web/eslint.config.mjs packages/web/migrations/001_initial_schema.sql packages/web/next.config.ts packages/web/package.json packages/web/postcss.config.mjs packages/web/README.md packages/web/scripts/generate-test-key.ts packages/web/src/app/api/v1/api-keys/[id]/route.ts packages/web/src/app/api/v1/api-keys/route.ts packages/web/src/app/api/v1/dashboards/[id]/route.ts packages/web/src/app/api/v1/dashboards/route.ts packages/web/src/app/api/v1/dashboards/utils.ts packages/web/src/app/api/v1/logs/[id]/replay/route.ts packages/web/src/app/api/v1/logs/[id]/route.ts packages/web/src/app/api/v1/logs/batch/route.ts packages/web/src/app/api/v1/logs/route.ts packages/web/src/app/api/v1/models/list/route.ts packages/web/src/app/api/v1/prompts/list/route.ts packages/web/src/app/api/v1/stats/route.ts packages/web/src/app/api/webhooks/clerk/route.ts packages/web/src/app/dashboard/api-keys/page.tsx packages/web/src/app/dashboard/layout.tsx packages/web/src/app/dashboard/page.tsx packages/web/src/app/docs/layout.tsx packages/web/src/app/docs/page.tsx packages/web/src/app/docs/quickstart/page.tsx packages/web/src/app/globals.css packages/web/src/app/layout.tsx packages/web/src/app/logs/layout.tsx packages/web/src/app/logs/page.tsx packages/web/src/app/page.tsx packages/web/src/app/sign-in/[[...sign-in]]/page.tsx packages/web/src/app/sign-up/[[...sign-up]]/page.tsx packages/web/src/components/api-key-create-modal.tsx packages/web/src/components/brand-link.tsx packages/web/src/components/code-block.tsx packages/web/src/components/dashboard-header.tsx packages/web/src/components/dashboard-preview.tsx packages/web/src/components/dashboard-switcher.tsx packages/web/src/components/log-detail-drawer.tsx packages/web/src/components/log-detail-panel.tsx packages/web/src/components/log-filters.tsx packages/web/src/components/logs-table.tsx packages/web/src/components/recent-logs-sidebar.tsx packages/web/src/components/save-dashboard-modal.tsx packages/web/src/components/stats-cards.tsx packages/web/src/components/ui/badge.tsx packages/web/src/components/ui/button.tsx packages/web/src/components/ui/calendar.tsx [27 more files omitted for size] ``` ### Dependencies - demo-app/package.json: @anthropic-ai/sdk@^0.67.0, @clarity/node@file:../packages/sdk-node, @types/node@^20.0.0, dotenv@^16.0.0, openai@^6.7.0, tsx@^4.7.0, typescript@^5.0.0, typescript-eslint@^8.46.2 - package.json: @typescript-eslint/eslint-plugin@^8.0.0, @typescript-eslint/parser@^8.0.0, eslint@^9.0.0, eslint-plugin-unused-imports@^4.1.0, prettier@^3.0.0, typescript@^5.0.0 - packages/sdk-demo/package.json: @anthropic-ai/sdk@^0.32.1, @clarity/node@file:../sdk-node, @types/node@^22.10.2, dotenv@^16.4.7, openai@^4.77.0, tsx@^4.19.2, typescript@^5.7.2, typescript-eslint@^8.46.2 - packages/sdk-node/package.json: @anthropic-ai/sdk@^0.67.0, @anthropic-ai/sdk@>=0.20.0, @types/jest@^29.5.0, @types/node@^20.0.0, @types/node-fetch@^2.6.11, dotenv@^17.2.3, jest@^29.5.0, node-fetch@^2.7.0, openai@^6.7.0, openai@>=4.0.0, ts-jest@^29.1.0, typescript@^5.0.0, typescript-eslint@^8.46.2 - packages/smartmail-demo/package.json: @anthropic-ai/sdk@^0.67.0, @clarity/node@^0.1.0, @tailwindcss/postcss@^4, @types/node@^20, @types/react@^19, @types/react-dom@^19, babel-plugin-react-compiler@1.0.0, eslint@^9, eslint-config-next@16.0.0, next@16.0.0, openai@^6.7.0, react@19.2.0, react-dom@19.2.0, tailwindcss@^4, typescript@^5 - packages/web/package.json: @anthropic-ai/sdk@^0.67.0, @clerk/nextjs@^6.34.0, @neondatabase/serverless@^1.0.2, @radix-ui/react-dialog@^1.1.15, @radix-ui/react-dropdown-menu@^2.1.16, @radix-ui/react-label@^2.1.7, @radix-ui/react-popover@^1.1.15, @radix-ui/react-select@^2.2.6, @radix-ui/react-separator@^1.1.7, @radix-ui/react-slot@^1.2.3, @radix-ui/react-tabs@^1.1.13, @tailwindcss/postcss@^4, @tanstack/react-table@^8.21.3, @types/bcrypt@^6.0.0, @types/cls-hooked@^4.3.9, @types/node@^20, @types/react@^19, @types/react-dom@^19, @types/react-syntax-highlighter@^15.5.13, @types/uuid@^10.0.0, babel-plugin-react-compiler@1.0.0, bcrypt@^6.0.0, class-variance-authority@^0.7.1, cls-hooked@^4.2.2, clsx@^2.1.1, date-fns@^4.1.0, date-fns-tz@^3.2.0, drizzle-kit@^0.31.5, drizzle-orm@^0.44.7, eslint@^9, eslint-config-next@16.0.0, lucide-react@^0.548.0, next@16.0.0, openai@^6.7.0, react@19.2.0, react-day-picker@^9.11.1, react-dom@19.2.0, react-intersection-observer@^9.16.0, react-syntax-highlighter@^16.0.0, recharts@^3.3.0, refractor@^5.0.0, sonner@^2.0.7, svix@^1.80.0, tailwind-merge@^3.3.1, tailwindcss@^4, tw-animate-css@^1.4.0, typescript@^5, uuid@^10.0.0, winston@^3.18.3, winston-daily-rotate-file@^5.0.0 ### Recent commits (newest first) - Update README.md - Ensured most of the doc links are user-auth-free - Completed view overlapping for filtered view screens - Fixed View All Activity screen and ensured to fit within screen while addressing for mobile view - fix table suspense padding - Merge branch 'main' of github.com:akshatdotcom/clarity - fix replay feature - Merge branch 'main' of https://github.com/akshatdotcom/clarity - Improved text and stylistic elements to table - highlight time filter - Merge branch 'main' of https://github.com/akshatdotcom/clarity - filter works great - fix horizontal sizing issue for log detail panel - Add error code - Merge branch 'main' of https://github.com/akshatdotcom/clarity - Refined filter universality improvement; enables filter to remainstate universally independent of switching saved dashboards - add smartmail demo app - Cleared logs and dashboard tabs on navbar; removed 24h, 7d, and 30d options from filter options - Merge branch 'main' of https://github.com/akshatdotcom/clarity - Completed full refinement of dashboard page with integrated logging, filtration, and custom filtered dashboard insights ## Key source files (fetched from GitHub, selected and truncated for size) ### CONTRIBUTING.md ```markdown # Contributing to Clarity Thank you for your interest in contributing! This document outlines the code quality standards and workflow for this project. ## Code Quality Standards We use several tools to maintain consistent, high-quality code: ### 1. **TypeScript** - All code must be TypeScript - Type safety is mandatory - Use `tsc --noEmit` to check types before committing ### 2. **ESLint** - Catches unused imports, variables, and code quality issues - Configures TypeScript-specific linting rules - Run `npm run lint` to check your code - Run `npm run lint:fix` to automatically fix issues ### 3. **Prettier** - Formats all code consistently - Run `npm run format` to format your code - Code is automatically formatted on save in VS Code ### 4. **EditorConfig** - Ensures consistent file encoding, line endings, and indentation ## Development Workflow ### Getting Started 1. Clone the repository 2. Install dependencies: `npm install` 3. Install recommended VS Code extensions (they'll be suggested automatically) ### Before Committing Run these commands to ensure your code meets our standards: ```bash # Format your code npm run format # Fix linting issues npm run lint:fix # Type check npm run type-check ``` ### Pre-commit Checklist - [ ] Code is formatted with Prettier - [ ] No ESLint errors - [ ] No TypeScript errors - [ ] All tests pass (if applicable) - [ ] Code builds successfully ## VS Code Setup The project includes VS Code settings that will: - ✅ Automatically format on save - ✅ Fix ESLint issues on save - ✅ Organize imports on save - ✅ Show real-time linting errors Install the recommended extensions when prompted. ## GitHub Actions All pull requests are automatically checked by GitHub Actions: - **Linting**: Ensures no ESLint errors - **Formatting**: Ensures code is properly formatted - **Type Checking**: Ensures TypeScript compiles - **Build**: Ensures all packages build successfully PRs that fail these checks will not be merged. ## Code Style Guidelines ### Naming Conventions - Use camelCase for variables and functions - Use PascalCase for classes and components - Use UPPER_SNAKE_CASE for constants ### Unused Code - Never commit unused imports or variables - Remove dead code immediately - Use ESLint to catch these issues ### Type Safety - Avoid `any` type - use proper types instead - Use type inference where appropriate - Export types that are used in multiple places ### File Organization - Keep files focused and small - One main export per file when possible - Group related functionality ## Questions? If you have questions about the code quality standards or workflow, please open an issue or reach out to the maintainers. ``` ### packages/web/API_README.md ```markdown # Clarity API Backend Documentation ## Overview This is the complete API backend for Clarity, an LLM observability platform. The backend handles log ingestion from the Node.js SDK, provides query endpoints for the dashboard, and supports features like log replay and analytics. ## Architecture - **Framework**: Next.js 16 (App Router) - **Database**: Neon Postgres (serverless) - **Authentication**: Clerk (session-based for dashboard, API keys for SDK) - **Language**: TypeScript (strict mode) ## Getting Started ### 1. Environment Variables Create a `.env.local` file in `packages/web/` with the following variables: ```bash # Database DATABASE_URL=your_neon_postgres_connection_string # Clerk Authentication NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_test_... CLERK_SECRET_KEY=sk_test_... CLERK_WEBHOOK_SECRET=whsec_... # LLM Providers (for replay feature) OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... ``` ### 2. Database Setup Run the migration to create the database schema: ```bash # Connect to your Neon database and run the migration psql $DATABASE_URL -f migrations/001_initial_schema.sql ``` This creates three tables: - **users**: Synced from Clerk authentication - **api_keys**: API keys for SDK authentication (bcrypt hashed) - **logs**: LLM request/response logs with full metadata ### 3. Clerk Webhook Setup Configure Clerk to send webhook events to your API: 1. Go to Clerk Dashboard → Webhooks 2. Add endpoint: `https://your-domain.com/api/webhooks/clerk` 3. Subscribe to events: `user.created`, `user.updated`, `user.deleted` 4. Copy the signing secret to `CLERK_WEBHOOK_SECRET` ### 4. Start Development Server ```bash cd packages/web npm run dev ``` The API will be available at `http://localhost:3000/api/v1/` ## API Endpoints ### Authentication **Two authentication methods:** 1. **API Key** (for SDK): Use `Bearer obs_xxxxx` in Authorization header 2. **Clerk Session** (for dashboard): Automatic via middleware ### Logs #### POST /api/v1/logs/batch Batch insert logs from SDK. **Auth**: API Key **Request**: ```json { "logs": [ { "id": "uuid", "created_at": "2025-01-24T10:00:00Z", "app_id": "my-app", "environment": "prod", "provider": "openai", "model": "gpt-4o", "input_text": "[{\"role\":\"user\",\"content\":\"Hello\"}]", "output_text": "Hi there!", "input_tokens": 10, "output_tokens": 5, "latency_ms": 1250, "cost_usd": 0.000375, "status": "success" } ] } ``` **Response**: ```json { "success": true, "inserted": 1 } ``` #### GET /api/v1/logs List logs with filters and pagination. **Auth**: Clerk Session **Query Parameters**: - `promptId`: Filter by prompt ID - `environment`: Filter by environment (dev/staging/prod) - `status`: Filter by status (success/error) - `startDate`: Start date (ISO 8601) - `endDate`: End date (ISO 8601) - `page`: Page number (default: 1) - `limit`: Items per page (default: 50, max: 100) **Example**: ``` [truncated — 6861 more characters] ``` ### package.json ``` { "name": "clarity-monorepo", "version": "0.1.0", "private": true, "scripts": { "lint": "npm run lint --workspaces", "lint:fix": "npm run lint:fix --workspaces", "format": "prettier --write \"**/*.{ts,tsx,js,jsx,json,md,mdx,css,html,yaml,yml}\"", "format:check": "prettier --check \"**/*.{ts,tsx,js,jsx,json,md,mdx,css,html,yaml,yml}\"", "type-check": "tsc --noEmit --project packages/web/tsconfig.json && tsc --noEmit --project packages/sdk-node/tsconfig.json && tsc --noEmit --project demo-app/tsconfig.json", "clean": "npm run clean --workspaces && rm -rf node_modules" }, "workspaces": [ "packages/*", "demo-app" ], "devDependencies": { "@typescript-eslint/eslint-plugin": "^8.0.0", "@typescript-eslint/parser": "^8.0.0", "eslint": "^9.0.0", "eslint-plugin-unused-imports": "^4.1.0", "prettier": "^3.0.0", "typescript": "^5.0.0" }, "engines": { "node": ">=18.0.0", "npm": ">=9.0.0" } } ``` ### demo-app/package.json ``` { "name": "clarity-demo-app", "version": "1.0.0", "description": "Demo app to test Clarity SDK integration", "type": "module", "scripts": { "dev": "tsx src/index.ts", "openai": "tsx src/openai-demo.ts", "anthropic": "tsx src/anthropic-demo.ts", "lint": "eslint src --ext .ts", "lint:fix": "eslint src --ext .ts --fix", "format": "prettier --write \"**/*.{ts,js,json,md}\"", "type-check": "tsc --noEmit" }, "keywords": [], "author": "", "license": "MIT", "dependencies": { "@anthropic-ai/sdk": "^0.67.0", "@clarity/node": "file:../packages/sdk-node", "dotenv": "^16.0.0", "openai": "^6.7.0" }, "devDependencies": { "@types/node": "^20.0.0", "tsx": "^4.7.0", "typescript": "^5.0.0", "typescript-eslint": "^8.46.2" } } ``` ### packages/smartmail-demo/package.json ``` { "name": "smartmail-demo", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev --webpack", "build": "next build --webpack", "start": "next start", "lint": "eslint" }, "dependencies": { "@anthropic-ai/sdk": "^0.67.0", "@clarity/node": "^0.1.0", "next": "16.0.0", "openai": "^6.7.0", "react": "19.2.0", "react-dom": "19.2.0" }, "devDependencies": { "@tailwindcss/postcss": "^4", "@types/node": "^20", "@types/react": "^19", "@types/react-dom": "^19", "babel-plugin-react-compiler": "1.0.0", "eslint": "^9", "eslint-config-next": "16.0.0", "tailwindcss": "^4", "typescript": "^5" } } ``` ### packages/sdk-demo/package.json ``` { "name": "@clarity/sdk-demo", "version": "1.0.0", "description": "Demo application showcasing Clarity SDK with OpenAI and Anthropic", "private": true, "type": "module", "scripts": { "dev": "tsx src/index.ts", "build": "tsc", "start": "node dist/index.js", "demo:simple": "tsx src/examples/simple.ts", "demo:chatbot": "tsx src/examples/chatbot.ts", "demo:comparison": "tsx src/examples/comparison.ts", "lint": "eslint src --ext .ts", "lint:fix": "eslint src --ext .ts --fix", "format": "prettier --write \"**/*.{ts,js,json,md}\"", "type-check": "tsc --noEmit" }, "dependencies": { "@anthropic-ai/sdk": "^0.32.1", "@clarity/node": "file:../sdk-node", "dotenv": "^16.4.7", "openai": "^4.77.0" }, "devDependencies": { "@types/node": "^22.10.2", "tsx": "^4.19.2", "typescript": "^5.7.2", "typescript-eslint": "^8.46.2" }, "engines": { "node": ">=18.0.0" } } ``` ### packages/sdk-node/package.json ``` { "name": "@clarity/node", "version": "0.1.0", "description": "Node.js SDK for Clarity - Simple LLM observability", "main": "dist/index.js", "types": "dist/index.d.ts", "scripts": { "build": "tsc", "test": "jest", "lint": "eslint src --ext .ts", "lint:fix": "eslint src --ext .ts --fix", "format": "prettier --write \"**/*.{ts,js,json,md}\"", "type-check": "tsc --noEmit", "prepublishOnly": "npm run build" }, "keywords": [ "llm", "observability", "openai", "anthropic", "monitoring", "logging" ], "author": "", "license": "MIT", "dependencies": { "dotenv": "^17.2.3", "node-fetch": "^2.7.0" }, "peerDependencies": { "@anthropic-ai/sdk": ">=0.20.0", "openai": ">=4.0.0" }, "peerDependenciesMeta": { "openai": { "optional": true }, "@anthropic-ai/sdk": { "optional": true } }, "devDependencies": { "@anthropic-ai/sdk": "^0.67.0", "@types/jest": "^29.5.0", "@types/node": "^20.0.0", "@types/node-fetch": "^2.6.11", "jest": "^29.5.0", "openai": "^6.7.0", "ts-jest": "^29.1.0", "typescript": "^5.0.0", "typescript-eslint": "^8.46.2" }, "files": [ "dist" ] } ``` ### packages/web/package.json ``` { "name": "web", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev --webpack", "build": "next build --webpack", "start": "next start", "lint": "eslint . --ext .ts,.tsx", "lint:fix": "eslint . --ext .ts,.tsx --fix", "format": "prettier --write \"**/*.{ts,tsx,js,jsx,json,md,css}\"", "type-check": "tsc --noEmit", "db:generate": "drizzle-kit generate", "db:migrate": "drizzle-kit migrate", "db:push": "drizzle-kit push", "db:studio": "drizzle-kit studio" }, "dependencies": { "@anthropic-ai/sdk": "^0.67.0", "@clerk/nextjs": "^6.34.0", "@neondatabase/serverless": "^1.0.2", "@radix-ui/react-dialog": "^1.1.15", "@radix-ui/react-dropdown-menu": "^2.1.16", "@radix-ui/react-label": "^2.1.7", "@radix-ui/react-popover": "^1.1.15", "@radix-ui/react-select": "^2.2.6", "@radix-ui/react-separator": "^1.1.7", "@radix-ui/react-slot": "^1.2.3", "@radix-ui/react-tabs": "^1.1.13", "@tanstack/react-table": "^8.21.3", "@types/react-syntax-highlighter": "^15.5.13", "@types/uuid": "^10.0.0", "bcrypt": "^6.0.0", "class-variance-authority": "^0.7.1", "cls-hooked": "^4.2.2", "clsx": "^2.1.1", "date-fns": "^4.1.0", "date-fns-tz": "^3.2.0", "drizzle-orm": "^0.44.7", "lucide-react": "^0.548.0", "next": "16.0.0", "openai": "^6.7.0", "react": "19.2.0", "react-day-picker": "^9.11.1", "react-dom": "19.2.0", "react-intersection-observer": "^9.16.0", "react-syntax-highlighter": "^16.0.0", "recharts": "^3.3.0", "refractor": "^5.0.0", "sonner": "^2.0.7", "svix": "^1.80.0", "tailwind-merge": "^3.3.1", "uuid": "^10.0.0", "winston": "^3.18.3", "winston-daily-rotate-file": "^5.0.0" }, "devDependencies": { "@tailwindcss/postcss": "^4", "@types/bcrypt": "^6.0.0", "@types/cls-hooked": "^4.3.9", "@types/node": "^20", "@types/react": "^19", "@types/react-dom": "^19", "babel-plugin-react-compiler": "1.0.0", "drizzle-kit": "^0.31.5", "eslint": "^9", "eslint-config-next": "16.0.0", "tailwindcss": "^4", "tw-animate-css": "^1.4.0", "typescript": "^5" } } ``` ### demo-app/src/index.ts ```typescript import { init, wrapOpenAI, wrapAnthropic } from "@clarity/node"; import OpenAI from "openai"; import Anthropic from "@anthropic-ai/sdk"; import { config } from "dotenv"; // Load environment variables config(); async function main() { console.log("🚀 Clarity SDK Demo - Combined Example\n"); // Initialize Clarity once init({ apiKey: process.env.CLARITY_API_KEY || "demo-api-key", endpoint: process.env.CLARITY_ENDPOINT, }); console.log("✅ Clarity initialized"); console.log("📦 App ID: auto-detected from package.json"); console.log("🌍 Environment: auto-detected from NODE_ENV\n"); // Test OpenAI if (process.env.OPENAI_API_KEY) { console.log("--- OpenAI Test ---"); const openai = wrapOpenAI(new OpenAI(), { promptId: "combined-demo", tags: ["openai"], }); const response = await openai.chat.completions.create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Say hi in 5 words or less!" }], max_tokens: 20, }); console.log("OpenAI:", response.choices[0]?.message?.content); console.log(`Tokens: ${response.usage?.total_tokens}\n`); } else { console.log("⚠️ Skipping OpenAI (no API key)\n"); } // Test Anthropic if (process.env.ANTHROPIC_API_KEY) { console.log("--- Anthropic Test ---"); const anthropic = wrapAnthropic(new Anthropic(), { promptId: "combined-demo", tags: ["anthropic"], }); const message = await anthropic.messages.create({ model: "claude-3-5-haiku-20241022", max_tokens: 20, messages: [{ role: "user", content: "Say hi in 5 words or less!" }], }); const text = message.content[0].type === "text" ? message.content[0].text : ""; console.log("Anthropic:", text); console.log( `Tokens: ${message.usage.input_tokens + message.usage.output_tokens}\n` ); } else { console.log("⚠️ Skipping Anthropic (no API key)\n"); } console.log("✨ All tests complete!"); console.log("\n⏳ Waiting for logs to flush..."); await new Promise((resolve) => setTimeout(resolve, 6000)); console.log("✅ Done!"); } main().catch(console.error); ``` ### packages/sdk-node/src/index.ts ```typescript /** * Clarity Node.js SDK * * Simple 2-line integration for LLM observability. * Automatically logs all OpenAI and Anthropic requests with full context. */ // Export main initialization function export { init } from "./observatory"; // Export wrapper functions export { wrapOpenAI } from "./wrappers/openai"; export { wrapAnthropic } from "./wrappers/anthropic"; // Export types for TypeScript users export type { ClarityConfig, WrapperOptions, Environment, Provider, LogStatus, LogEntry, LogError, TokenUsage, ModelPricing, } from "./types"; // Export cost calculation utilities (for advanced users) export { calculateOpenAICost, calculateAnthropicCost, getOpenAIPricing, getAnthropicPricing, } from "./costs"; ``` [100 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]