Speak Baby

Inspiration

The inspiration for Boop came from our parents telling us about the late nights they had raising us. We kept wondering whether they were avoidable. During early interviews and informal surveys with new parents, we kept hearing the same frustrations: "At 3AM, I don't want to guess if she's hungry or cold." "In the initial months, it felt like my baby and I spoke different languages." One parent told us: "The hardest part isn't crying. It's not knowing what the crying means." This was the sentence that shaped Boop's mission.

What it does

Boop is a multimodal AI platform that combines audio intelligence, computer vision, conversational AI, and generative media to empower parents. Baby Cry Decoder This feature allows parents to record their baby's cry and receive real-time predictions of the cause — hungry, needs a diaper change, needs to be burped, tired, etc. The baby's cry is modelled as a time-series waveform, and differences in the dominant frequency bands, harmonic intensity, and duration patterns determine the cry cause. Our ML pipeline captures audio, filters noise, generates spectrograms, and extracts MFCC features, which are then fed into a CNN + LSTM ensemble classifier. The CNN detects frequency patterns from spectrogram images while the LSTM captures temporal dependencies across cry sequences, with the final prediction combining both outputs via a weighted ensemble. We optimized the trained model and converted it to ONNX for low-latency mobile inference, delivering near real-time predictions designed for exhausted parents at 2AM. AI Assistant (Personalized Pediatric Chatbot) Boop's AI Assistant is a pediatric companion that knows your baby's unique health story. Powered by Anthropic Claude Sonnet 4.5 and leveraging our multi-turn agent architecture, the assistant maintains context across all your baby's data: cry patterns showing increased hunger, diaper logs indicating digestive changes, and daily survey responses tracking sleep regression. Intelligent Daily Check-Ins: Boop's agent learns and adapts. Using Claude's autonomous agent capabilities, the system asks standard questions (feeding amounts, wet diapers, sleep quality) then generates intelligent follow-ups based on previous responses. After each survey, parents receive a comprehensive summary that includes: Daily Health Summary – AI-generated narrative of baby's day with flagged concerns Comparison Analysis – Using Browserbase automation, Boop scrapes trusted pediatric sources (AAP, CDC growth charts, Mayo Clinic) for age-specific norms and compares baby's metrics Curated Research – For any flagged concerns (reflux, sleep issues, feeding difficulties), Browserbase automatically finds and surfaces relevant, evidence-based articles from trusted sources The daily check data directly feeds into standardized pediatrician reports for use at your visits (SOAP, Well-child Visit). Diaper Diary (Digestive Health Monitoring) The Diaper Health Tracker allows parents to upload diaper images to monitor their baby's bowel health over time. Using Computer Vision, the color distribution, texture, and frequency of abnormal signals over the last week flags potential concerns such as dehydration, constipation, or diarrhea. If a trend has been noticed for some time, it recommends a check-in with the doctor. Baby Cam Monitor (BoopDanger) We offer a feature that gives notifications of environmental risks in real time after integrating with a live cam. Our model continuously detects the baby's position, alerting parents if the distance between the baby and a dangerous object is below a threshold. There's also a comprehensive log of the baby's instances around dangerous objects. Baby's Adventures (AI Memory Collage) Boop generates a "Spotify Wrapped"-style baby recap utilizing collages from baby cam photos, mood classification from facial analysis, and music generated to match by vibe from Suno. Product Safety Check (EWG Scan) Boop's Product Safety Check feature empowers parents with instant, evidence-based safety information. Simply scan a product's barcode or search by name, and Boop retrieves comprehensive safety ratings from trusted sources including the Environmental Working Group (EWG) database.

How we built it

Core Stack Python – Machine learning & backend logic Java – Android components ONNX Runtime – Model deployment Cursor IDE – Full-stack development Anthropic Claude Sonnet 4.5 – AI Assistant ChatGPT 5.2 – Used for stool classification OpenEvidence – Research validation Suno – Generative music Development Workflow We built everything inside Cursor, including model training, ONNX conversion, API wrapping, backend logic, and iterative debugging. System Architecture Boop is modular — each service communicates through internal APIs to create a unified AI system: Audio ML service – Cry decoder Vision ML service – Diaper + safety analysis Chat intelligence layer – Claude-powered assistant Generative media layer – Suno music generation Structured health engine – Report builder

Challenges we ran into

We found that the frequencies of cries stemming from the same cause vary significantly across infants. There are also limited labelled datasets for infant health. We also had to account for noise in home and outdoor environments.

Accomplishments we're proud of

Parents reporting feeling less anxious and having a good time staying up all night with each other.

What we learned

Empowering parents and encouraging them plays a role in making a better society. Design needs to be created keeping the tired user in mind. It is very difficult to resist DoorDashing the Melt.

What's next

Expanded safety detection features Continuous video monitor screen capture and detection iOS expansion Clinical validation partnerships and HIPAA compliance