# Project export: Senya This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: Making music something you can feel, see, and sign. - Devpost: https://devpost.com/software/senya - GitHub: https://github.com/Deeksha-Vaidyanathan/senya - Demo: https://docs.google.com/presentation/d/1jKFJWTeVdbkPNoNfudgQinBf5Az1CdM60-hINnNjjjI/edit?usp=sharing - Result: winner (Most Creative Use of Pika) - Team: 3 GitHub contributor(s) — Tejas Mundhe (2 commits), nithila-sadheesh (2 commits), Claude Sonnet 4.6 (1 commits) ## Devpost submission (written by the team) ### Inspiration On February 12, 2023, the Super Bowl Halftime Show featuring Rihanna delivered a record 113 million viewers. But it wasn't her performance that captured the attention of most — it was her ASL interpreter. Justine Miles made the performance accessible to the Deaf and hard-of-hearing community, and in doing so went viral, exposing millions of hearing viewers to the expressiveness of American Sign Language for the first time. That moment crystallized something for us: ASL interpretation of music shouldn't be a once-a-year spectacle reserved for the biggest stages. Every song, for every person, should be able to sign. Senya was born from the idea that the gap between a song and its ASL performance could be closed by software, making music a shared experience rather than a divided one. ### What it does Senya transforms any song into a synchronized ASL music video. You give it lyrics (or just an audio file) and the original song, and it returns a complete video of a signer performing the lyrics in American Sign Language, timed to the music, with karaoke-style captions burned in. Under the hood it transcribes the lyrics with timestamps, translates English into ASL gloss, fetches real ASL signing clips for each sign, stretches them to match the song's pacing, stitches them into one continuous performance, overlays the original music, and adds synced captions, producing a shareable video that makes any track accessible and engaging for the Deaf and hard-of-hearing community. ### How we built it Senya is a seven-stage Python pipeline. Stage 1 uses OpenAI Whisper (or a text parser) to turn lyrics into timed words. Stage 2 uses Anthropic's Claude (claude-sonnet-4-6) to translate English into ASL gloss, since ASL has its own grammar and word order rather than a one-to-one mapping from English. Stage 3 resolves each gloss token to a real signing clip by fetching GIFs on demand from Lifeprint (Dr. Bill Vicars' ASL University), converting them to MP4 with Pillow and OpenCV while preserving each frame's native timing, and caching the resulting CDN URLs so no sign is ever fetched twice. Stages 4 through 7 run entirely on Pika's REST API: generate_reference_video takes the stitched clips and overlays them onto the Pika avatar to give a realistic concert-feel, edit_speed adjusts clip timing at the phrase level, edit_concat stitches clips together (batched to respect the 24-clip limit), edit_audio_mix overlays the original song, and add_captions burns in synced karaoke lyrics. The final output is a single Pika CDN URL. ### Challenges we ran into The biggest challenge was getting Pika to sign accurately at all. Our first approach was to fine-tune a model on MLASL, a dictionary of English words mapped to ASL videos that is widely used in research. That quickly proved unworkable: Pika is closed source, so we had no access to the model weights and no way to fine-tune. We pivoted to Pika's generate-reference-video tool, using Lifeprint's videos as the reference. Because Lifeprint features a consistent signer against a consistent background, Pika was able to reproduce the signs accurately and place that signing within a concert-style environment, which was exactly the result we were after. From there, the focus shifted to stitching the generated clips into one continuous performance. We made strong progress getting portions of the song working, and refined our approach to timing and tempo to keep the signing synchronized to the music across the track. ### Accomplishments we're proud of We're proud that Senya uses real ASL rather than synthetic avatars or approximations that the Deaf community has long criticized. We built an on-demand architecture that requires no multi-gigabyte dataset download: each sign is fetched the first time it's needed and cached forever after, so the system gets faster the more you use it. And we got an end-to-end pipeline working across two AI APIs and a video editing API, turning a raw song into a captioned, music-synced ASL performance with a single command. ### What we learned We learned that ASL is a full language with its own grammar, not a signed transcription of English — which is why the gloss translation step matters so much and why naive word-for-word mapping fails. We got hands-on with the realities of programmatic video editing: frame timing, aspect ratios, concat limits, and audio mixing each carry constraints that shape the whole architecture. And we learned that accessibility engineering is full of trade-offs between sign authenticity and coverage, musical timing and sign readability, where the right answer is usually a thoughtful fallback rather than a perfect solution. ### What's next Next we want to move from stitched real-sign clips toward smoother transitions between signs, since signing is continuous rather than a sequence of discrete words. We'd like to expand sign coverage beyond Lifeprint by integrating additional datasets (like How2Sign) to shrink the fingerspelling fallback rate. We're also interested in capturing ASL's non-manual markers, which are facial expression and body movement carry grammatical and emotional meaning that static clip-stitching loses. Longer term: real-time signing for live performances, support for sign languages beyond ASL, and a community feedback loop so Deaf signers can help improve translation quality. ## README (from the GitHub repository) No README available. ## Detected evidence (automated analysis) Indexed codebase: 59 recognized source files, 934 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - AI coding agent: Claude Code — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (67 of 67) ``` .agents/skills/app-sizzle/references/liquid-glass.md .agents/skills/app-sizzle/references/template-a-cinematic.md .agents/skills/app-sizzle/SKILL.md .agents/skills/app-store-screens/references/default-layout.md .agents/skills/app-store-screens/references/layout-archetypes.md .agents/skills/app-store-screens/references/render-pipeline.md .agents/skills/app-store-screens/SKILL.md .agents/skills/baseball-trend/SKILL.md .agents/skills/build-a-brand/references/brand-directions.md .agents/skills/build-a-brand/references/brand-guidelines.md .agents/skills/build-a-brand/references/brand-identity.md .agents/skills/build-a-brand/references/brand-md-template.md .agents/skills/build-a-brand/SKILL.md .agents/skills/content-director/formats/dance.md .agents/skills/content-director/formats/duet.md .agents/skills/content-director/formats/pov.md .agents/skills/content-director/formats/talking.md .agents/skills/content-director/formats/teleprompter.html .agents/skills/content-director/formats/teleprompter.md .agents/skills/content-director/README.md .agents/skills/content-director/SKILL.md .agents/skills/explainer/SKILL.md .agents/skills/fix-my-look/SKILL.md .agents/skills/founder-product-video/references/ops-notes.md .agents/skills/founder-product-video/SKILL.md .agents/skills/kiss-cam/SKILL.md .agents/skills/language-swap/references/language-coverage.md .agents/skills/language-swap/SKILL.md .agents/skills/persona-builder/references/aesthetic-prompts.md .agents/skills/persona-builder/references/persona-md-template.md .agents/skills/persona-builder/references/templates/mood-board-no-header.template.html .agents/skills/persona-builder/references/templates/mood-board.template.html .agents/skills/persona-builder/references/templates/pdf-about.template.html .agents/skills/persona-builder/references/templates/pdf-content-categories.template.html .agents/skills/persona-builder/references/templates/pdf-cover.template.html .agents/skills/persona-builder/references/templates/pdf-do-dont.template.html .agents/skills/persona-builder/references/templates/pdf-hooks-dm.template.html .agents/skills/persona-builder/references/templates/pdf-moodboard.template.html .agents/skills/persona-builder/references/templates/pdf-next-steps.template.html .agents/skills/persona-builder/references/templates/pdf-shared-head.template.html .agents/skills/persona-builder/references/templates/pdf-voice-mode.template.html .agents/skills/persona-builder/SKILL.md .agents/skills/podcast/SKILL.md .agents/skills/ugc-ads/SKILL.md .agents/skills/viral-hook/SKILL.md .gitignore backend/.env backend/.env.example backend/dictionary.py backend/gloss.py backend/local_ffmpeg.py backend/main.py backend/pika_client.py backend/pipeline.py backend/requirements.txt dictionary/avatar.json dictionary/index.json frontend/app.js frontend/index.html frontend/style.css planning.md scripts/download_signs.py scripts/generate_avatar.py scripts/generate_dict.py scripts/requirements.txt scripts/word_list.py skills-lock.json ``` ### Dependencies - backend/requirements.txt: fastapi@==0.115.0, python-dotenv@==1.0.1, python-multipart@==0.0.9, uvicorn[standard]@==0.30.6 - scripts/requirements.txt: anthropic@==0.34.2, beautifulsoup4@==4.12.3, python-dotenv@==1.0.1, requests@==2.32.3 ### Recent commits (newest first) - added video upload option - Add local ffmpeg pipeline, transcript bypass, and gitignore - Added optional text vid transcript feature - Dictionary method - planning.md ## Key source files (fetched from GitHub, selected and truncated for size) ### planning.md ```markdown # SignAI — ASL Avatar Hackathon Project ## Pika Track | Team of 3 --- ## The Idea As people watch videos, an AI avatar appears alongside the video performing the content in American Sign Language (ASL). This helps deaf users who prefer sign language over captions follow along naturally. --- ## Core Strategy: Pre-generated Sign Dictionary + Runtime Assembly Pika can't generate accurate ASL on the fly from a text prompt alone — but we work around this by **pre-generating a library of correct sign videos** using real ASL reference footage, then assembling them at runtime. ### Full Pipeline ``` Input Video │ ▼ [Pika: transcribe_audio] │ ▼ Text → ASL Gloss tokens (via Claude API)/ │ ▼ Dictionary Lookup (word → Pika-generated sign clip) │ ▼ [Pika: edit_concat] → assembled signing video │ ▼ [Pika: edit_pip] → final video with avatar in corner ``` --- ## Step 1: Build the Sign Dictionary (Do This Before the Hackathon) This is the most important prep work. **Source material:** Download free ASL sign videos from public dictionaries: - [Handspeak](https://www.handspeak.com) - [ASLU / LifePrint](https://www.lifeprint.com) - [ASL University](https://www.asluniversity.com) Target **300–500 of the most common English words** to start. **Use Pika's `generate_reference_video`** — pass the real ASL video as a motion reference and generate a consistent-looking avatar doing the same sign. Pika doesn't need to *know* ASL — it just copies the motion from reference footage onto the avatar. ``` real_asl_video["hello"] → generate_reference_video( reference=real_asl_video, prompt="person signing hello in ASL, clean background" ) → avatar_sign["hello"].mp4 ``` Store all generated clips in a hosted asset store (Supabase or S3). That's the dictionary. --- ## Step 2: Create a Consistent Avatar Character Before building the dictionary, design the avatar once: 1. Use Pika's `generate_image` to create a base character — consistent appearance, neutral background, clear hands 2. Use that same image as the base for **every** `generate_reference_video` call 3. Consistency is critical so `edit_concat` doesn't look jarring between signs --- ## Step 3: Gloss Translation at Runtime ASL is not word-for-word English — it has its own grammar called **ASL gloss**. **Recommended approach:** Use the Claude API to convert the transcript to ASL gloss order before dictionary lookup. This is ~10 lines of code and makes the output significantly more accurate. Example prompt: ``` Convert this English sentence to ASL gloss order (no articles, topic-comment structure): "The dog ran through the field" → "DOG RUN THROUGH FIELD" ``` **Fallback (simpler):** Signed Exact English (SEE) — sign each English word in order. Not true ASL grammar, but visually understandable and totally shippable for a demo. --- ## Step 4: Runtime Assembly with Pika ```python # Pseudocode transcript = pika.transcribe_audio(video) gloss_token [truncated — 2731 more characters] ``` ### .agents/skills/viral-hook/SKILL.md ```markdown --- name: viral-hook description: >- Prepend a 4s viral hook + optional designed title to a user's video, then hard-cut into the real clip. An attention-grabbing event erupts into the user's OWN scene; the title is rendered in-scene by Seedance. One MCP call does the whole render. Requires Pika MCP. Triggers: "add a viral hook", "disruption hook on my video", "hook + title on my clip", "viral hook with typography", "scroll-stopper intro". argument-hint: "[video path or URL] [optional title line]" required-capabilities: - mcp__plugin_pika_pika__upload_asset - mcp__plugin_pika_pika__analyze_media - mcp__plugin_pika_pika__generate_viral_hook - mcp__plugin_pika_pika__task_status --- # Viral Hook Prepends an extreme, no-dialogue ~4s hook to a user's video — an attention-grabbing event that erupts into the user's own scene, with an optional designed title burned into the lower third — then hard-cuts into the untouched clip. `mcp__plugin_pika_pika__generate_viral_hook` does the whole render in one call and returns BOTH the hook and the stitched final. This skill's job is the creative judgment the tool can't make: read the scene, pick a hook action from the menu, and write a title. ## Hook rules (govern what you WRITE as `hook_action`) 1. **Certain to grab attention, fast.** Extreme and unmissable inside ~2s — over plausibility or relevance to the video's topic. 2. **Erupts into the user's OWN scene.** Same person, location, lighting — the tool anchors on the best visual anchor frame: 0s when usable, otherwise the first detectable face frame. Describe the event entering THAT space, never a different setting. 3. **Use the scene's real geometry.** Enter through a doorway with depth, a wall behind the subject, headroom above. 4. **No dialogue.** Voice-free (only ambient / impact SFX) — don't write spoken lines into `hook_action`. ## Hook super-category menu Four super-categories. **Rotate** super-category per run on the same clip/session — without rotation, regenerates collapse to vehicle / explosion / creature. Archetypes are starting points, not a closed list; invent freely within the category. ### A. Destructive intrusion — external force violently enters and damages the scene | Archetype | What happens | Scene affordance | |---|---|---| | Incoming vehicle | car / truck / bus smashes through a wall or doorway and barrels at camera | background depth or a back wall | | Flash flood / wave | a wall bursts and water surges down toward camera | corridor / back wall | | Structural collapse | ceiling / wall / shelf caves in, debris rains down | headroom / walls | | Explosion / blast | fireball erupts behind them, shockwave + debris rush forward | a background surface | | Creature attack | animal / monster lunges from off-frame or bursts in | open off-frame edge | | Absurd intruder | dinosaur, elephant, UFO, giant hand enters behind them | background space | | Natural disaster | tornado / meteor / lightning / sinkhole | [truncated — 6089 more characters] ``` ### scripts/requirements.txt ``` requests==2.32.3 beautifulsoup4==4.12.3 anthropic==0.34.2 python-dotenv==1.0.1 ``` ### backend/requirements.txt ``` fastapi==0.115.0 uvicorn[standard]==0.30.6 python-multipart==0.0.9 python-dotenv==1.0.1 ``` ### backend/main.py ```python """ SignAI — FastAPI backend Endpoints: POST /process — process a video URL POST /process/upload — upload a video file and process it GET /dictionary/stats — dictionary coverage stats GET /health — health check """ import os from contextlib import asynccontextmanager from fastapi import FastAPI, HTTPException, UploadFile, File, Form from fastapi.middleware.cors import CORSMiddleware from fastapi.staticfiles import StaticFiles from pydantic import BaseModel from dotenv import load_dotenv import dictionary as dict_ from pipeline import process_video load_dotenv() @asynccontextmanager async def lifespan(app: FastAPI): dict_.load() yield app = FastAPI(title="SignAI", lifespan=lifespan) app.add_middleware( CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"], ) frontend_path = os.path.join(os.path.dirname(__file__), "..", "frontend") outputs_path = os.path.join(frontend_path, "outputs") os.makedirs(outputs_path, exist_ok=True) if os.path.exists(frontend_path): app.mount("/outputs", StaticFiles(directory=outputs_path), name="outputs") app.mount("/app", StaticFiles(directory=frontend_path, html=True), name="frontend") class ProcessRequest(BaseModel): video_url: str pip_position: str = "bottom-right" transcript: str | None = None @app.get("/health") def health(): return {"status": "ok"} @app.get("/dictionary/stats") def dictionary_stats(): return dict_.stats() @app.post("/process") def process_url(req: ProcessRequest): try: return process_video(req.video_url, req.pip_position, req.transcript or None) except ValueError as e: raise HTTPException(status_code=422, detail=str(e)) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) @app.post("/process/upload") def process_upload( file: UploadFile = File(...), pip_position: str = "bottom-right", transcript: str | None = Form(None), ): try: file_bytes = file.file.read() return process_video( video_url="", pip_position=pip_position, transcript_override=transcript or None, video_bytes=file_bytes, video_filename=file.filename or "upload.mp4", ) except ValueError as e: raise HTTPException(status_code=422, detail=str(e)) except Exception as e: raise HTTPException(status_code=500, detail=str(e)) ``` ### frontend/app.js ```javascript const API = "http://localhost:8000"; // Tab switching document.querySelectorAll(".tab").forEach(tab => { tab.addEventListener("click", () => { document.querySelectorAll(".tab").forEach(t => t.classList.remove("active")); document.querySelectorAll(".tab-content").forEach(c => c.classList.remove("active")); tab.classList.add("active"); document.getElementById(`tab-${tab.dataset.tab}`).classList.add("active"); }); }); // Drag-and-drop on upload area const uploadLabel = document.getElementById("upload-label"); const uploadText = document.getElementById("upload-text"); const fileInput = document.getElementById("video-file"); uploadLabel.addEventListener("dragover", e => { e.preventDefault(); uploadLabel.classList.add("drag-over"); }); uploadLabel.addEventListener("dragleave", () => uploadLabel.classList.remove("drag-over")); uploadLabel.addEventListener("drop", e => { e.preventDefault(); uploadLabel.classList.remove("drag-over"); const file = e.dataTransfer.files[0]; if (file) { fileInput.files = e.dataTransfer.files; uploadText.textContent = file.name; } }); fileInput.addEventListener("change", () => { if (fileInput.files[0]) uploadText.textContent = fileInput.files[0].name; }); // Submit document.getElementById("submit-btn").addEventListener("click", async () => { const activeTab = document.querySelector(".tab.active").dataset.tab; const position = document.getElementById("pip-position").value; const transcript = document.getElementById("transcript-input").value.trim() || null; if (activeTab === "url") { const url = document.getElementById("video-url").value.trim(); if (!url) return alert("Please enter a video URL."); await runPipeline(() => processUrl(url, position, transcript)); } else { const file = fileInput.files[0]; if (!file) return alert("Please select a video file."); await runPipeline(() => processUpload(file, position, transcript)); } }); document.getElementById("retry-btn").addEventListener("click", () => { hide("error-box"); hide("result-section"); }); async function processUrl(url, position, transcript) { const resp = await fetch(`${API}/process`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ video_url: url, pip_position: position, transcript }), }); if (!resp.ok) throw new Error((await resp.json()).detail || "Server error"); return resp.json(); } async function processUpload(file, position, transcript) { const form = new FormData(); form.append("file", file); if (transcript) form.append("transcript", transcript); const resp = await fetch(`${API}/process/upload?pip_position=${position}`, { method: "POST", body: form, }); if (!resp.ok) throw new Error((await resp.json()).detail || "Server error"); return resp.json(); } async function runPipeline(fn) { const btn = document.getElementById("submit-btn"); btn.disabled = true; show("result-section"); show("loader"); hide("output"); hide("error-box"); const steps = [ "Adding captions to source video...", "Converting to ASL gloss...", "Looking up sign clips...", "Overlaying ASL avatar...", ]; let stepIdx = 0; const loaderText = document.getElementById("loader-text"); loaderText.textContent = steps[0]; const ticker = setInterval(() => { stepIdx = Math.min(stepIdx + 1, steps.length - 1); loaderText.textContent = steps[stepIdx]; }, 8000); try { const result = await fn(); clearInterval(ticker); showResult(result); } catch (err) { clearInterval(ticker); hide("loader"); show("error-box"); document.getElementById("error-text").textContent = err.message; } finally { btn.disabled = false; } } function showResult(result) { hide("loader"); show("output"); const video = document.getElementById("output-video"); video.src = result.output_url; video.load(); document.getElementById("meta-transcript").textContent = result.transcript; document.getElementById("meta-gloss").textContent = result.gloss.join(" "); document.getElementById("meta-coverage").textContent = `${result.signs_found} / ${result.signs_total} signs matched` + (result.missing_signs.length ? ` (missing: ${result.missing_signs.join(", ")})` : ""); const dl = document.getElementById("download-btn"); dl.href = result.output_url; } function show(id) { document.getElementById(id).classList.remove("hidden"); } function hide(id) { document.getElementById(id).classList.add("hidden"); } ``` ### backend/dictionary.py ```python """In-memory dictionary loaded from index.json at startup.""" import json from pathlib import Path INDEX_PATH = Path(__file__).parent.parent / "dictionary" / "index.json" SOURCE_DIR = Path(__file__).parent.parent / "dictionary" / "source" _index: dict = {} def load(): global _index if INDEX_PATH.exists(): _index = json.loads(INDEX_PATH.read_text()) print(f"Dictionary loaded: {len(_index)} words, " f"{sum(1 for v in _index.values() if v.get('pika_asset_url'))} with generated clips") def lookup(word: str) -> str | None: """Return the Pika asset URL for a word, or None if not in dictionary.""" entry = _index.get(word.lower()) if entry and entry.get("pika_asset_url"): return entry["pika_asset_url"] return None def local_lookup(word: str) -> str | None: """Return the local file path for a word's source clip, or None.""" entry = _index.get(word.lower()) if not entry or not entry.get("source"): return None local_path = SOURCE_DIR / Path(entry["source"]).name if local_path.exists(): return str(local_path) return None def stats() -> dict: total = len(_index) generated = sum(1 for v in _index.values() if v.get("pika_asset_url")) verified = sum(1 for v in _index.values() if v.get("verified")) return {"total": total, "generated": generated, "verified": verified} ``` ### scripts/generate_avatar.py ```python """ Script 2a: Generate the base avatar character image via Pika MCP (claude CLI). Run this ONCE before generate_dict.py. Saves the avatar URL to dictionary/avatar.json. """ import json import subprocess from pathlib import Path AVATAR_PATH = Path(__file__).parent.parent / "dictionary" / "avatar.json" MCP_CONFIG = json.dumps({ "mcpServers": { "pika-mcp": {"type": "url", "url": "https://mcp.pika.me/api/mcp"} } }) PROMPT = ( "Use the Pika generate_image tool to create: " "A friendly person from the shoulders up, facing the camera directly, " "clean light gray background, neutral expression, arms and hands visible, " "wearing a dark solid-color shirt for hand contrast, photorealistic, " "soft studio lighting, no text, no watermarks. " "Return the image URL." ) SCHEMA = json.dumps({ "type": "object", "properties": {"url": {"type": "string"}}, "required": ["url"] }) def main(): if AVATAR_PATH.exists(): existing = json.loads(AVATAR_PATH.read_text()) print(f"Avatar already exists: {existing['url']}") print("Delete dictionary/avatar.json to regenerate.") return print("Generating avatar character...") result = subprocess.run( ["claude", "-p", PROMPT, "--output-format", "json", "--json-schema", SCHEMA, "--mcp-config", MCP_CONFIG, "--allowedTools", "mcp__pika-mcp__*"], capture_output=True, text=True, timeout=120 ) if result.returncode != 0: print(f"Error: {result.stderr}") return outer = json.loads(result.stdout) url = outer["structured_output"]["url"] AVATAR_PATH.write_text(json.dumps({"url": url}, indent=2)) print(f"Avatar saved: {url}") if __name__ == "__main__": main() ``` ### backend/gloss.py ```python """ Convert English transcript to ASL gloss tokens. Uses claude CLI (no API key needed — runs in local Claude Code session). Falls back to rule-based if claude CLI is unavailable. """ import os import re import json import subprocess _STRIP = { "a", "an", "the", "is", "are", "was", "were", "am", "be", "been", "being", "do", "does", "did", "will", "would", "could", "should", "shall", "may", "might", "must", "have", "has", "had", "of", "to", "in", "on", "at", "by", "for", "with", "about", "into", "through", "during", "it", "its", } def _rule_based(transcript: str) -> list[str]: words = re.sub(r"[^a-zA-Z\s'-]", "", transcript).split() return [w.upper() for w in words if w.lower() not in _STRIP] def _claude_gloss(transcript: str) -> list[str]: prompt = ( "Convert this English text to ASL gloss notation. " "Rules: remove articles (a/an/the), use topic-comment structure, " "present tense only, CAPITALIZE all tokens. " "Return only the gloss tokens as a JSON array of strings.\n\n" f"Text: {transcript}" ) schema = json.dumps({ "type": "object", "properties": { "tokens": {"type": "array", "items": {"type": "string"}} }, "required": ["tokens"] }) result = subprocess.run( ["claude", "-p", prompt, "--output-format", "json", "--json-schema", schema], capture_output=True, text=True, timeout=60 ) if result.returncode != 0: raise RuntimeError(result.stderr) outer = json.loads(result.stdout) return [t.strip().upper() for t in outer["structured_output"]["tokens"] if t.strip()] def to_asl_gloss(transcript: str) -> list[str]: try: return _claude_gloss(transcript) except Exception as e: print(f" Claude gloss failed ({e}), using rule-based fallback") return _rule_based(transcript) ``` ### scripts/word_list.py ```python # 150 highest-value ASL signs for the hackathon demo # Trimmed from 300 to fit within 5000 Pika credits # (~25 credits/clip × 150 = 3750, leaving ~1250 for avatar + runtime demos) WORD_LIST = [ # Greetings / social (high demo value) "hello", "goodbye", "please", "thank-you", "sorry", "excuse-me", "yes", "no", # Core verbs (most common in any sentence) "go", "come", "see", "know", "want", "need", "help", "work", "eat", "drink", "sleep", "walk", "run", "stop", "start", "finish", "sit", "stand", "give", "take", "like", "love", "understand", "learn", "ask", "tell", "show", "find", "call", "meet", "think", "wait", "open", "close", "read", "write", "play", "buy", # Pronouns (essential for any sentence) "i", "you", "he", "she", "we", "they", "me", "my", "your", "this", "that", "here", "there", # Common nouns (broad coverage) "person", "man", "woman", "family", "friend", "mother", "father", "sister", "brother", "home", "school", "store", "car", "food", "water", "money", "time", "day", "week", "year", "morning", "night", "dog", "cat", "book", "phone", "doctor", # Adjectives (most expressive) "good", "bad", "big", "small", "hot", "cold", "fast", "slow", "happy", "sad", "angry", "sick", "tired", "hungry", "old", "new", "easy", "hard", "right", "wrong", "late", "early", # Numbers (1-10 only) "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten", # Colors (most common) "red", "blue", "green", "yellow", "black", "white", # Time (high utility) "today", "tomorrow", "yesterday", "now", "later", "always", "never", "before", "after", # Question words (critical for any conversation) "what", "where", "when", "who", "why", "how", # Useful standalone signs "maybe", "again", "more", "enough", "all", "nothing", "because", "but", ] # Deduplicated and lowercased WORD_LIST = sorted(set(w.lower() for w in WORD_LIST)) ``` [44 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]