Project Info
This project did not submit a demo video on Devpost.
Problem Statement Many home cooks struggle with multitasking and managing their time effectively while preparing meals, which often leads to overcooked or undercooked food. College students and individuals with ADHD may also experience difficulties with procrastination and maintaining focus in the kitchen. CHOP CHOP aims to alleviate these challenges by providing a fun and interactive cooking experience that enhances coordination and multitasking abilities. Solution Overview CHOP CHOP is a hands-free cooking assistant that leverages Google Gemini for generative AI and optimized advice, alongside gesture control through the Leap Motion hand tracking sensor. By providing structured cooking tasks and leveraging AI to generate detailed task lists, our project helps home cooks manage their time and streamline the cooking process. Key Features Gesture Control with Leap Motion Users can navigate tasks hands-free by utilizing gestures detected by the Leap Motion sensor: Swipe Gesture: Move to the next task or fast-forward. Tap Gesture: Add more time to the current task. Clap Gesture: Pause or stop the timer. Gesture Control with Leap Motion Users can navigate tasks hands-free by utilizing gestures detected by the Leap Motion sensor: Swipe Gesture: Move to the next task or fast-forward. Tap Gesture: Add more time to the current task. Clap Gesture: Pause or stop the timer. Swipe Gesture: Move to the next task or fast-forward. Tap Gesture: Add more time to the current task. Clap Gesture: Pause or stop the timer. AI-Powered Task Generation Prompt Engineering: Users can input specific cooking prompts (e.g., “I want to cook a medium-rare sirloin steak with unchopped onions and chopped broccoli”). Detailed Task Breakdown: The AI generates a structured list of cooking steps, including time estimates and coordination of overlapping tasks. AI-Powered Task Generation Prompt Engineering: Users can input specific cooking prompts (e.g., “I want to cook a medium-rare sirloin steak with unchopped onions and chopped broccoli”). Detailed Task Breakdown: The AI generates a structured list of cooking steps, including time estimates and coordination of overlapping tasks. Interactive Task List Each task in the list features a "Start" button with a countdown timer that dynamically updates based on user interaction (start, pause, fast-forward). Users can easily visualize their progress through the cooking process. Interactive Task List Each task in the list features a "Start" button with a countdown timer that dynamically updates based on user interaction (start, pause, fast-forward). Users can easily visualize their progress through the cooking process. How It Works User Input: Users provide a cooking prompt, and the AI generates a detailed task list tailored to their recipe. Task Management: Users can interact with the task list, starting timers and navigating tasks with gestures. Technology Stack Frontend: Developed with Next.js to manage routing and user interactions efficiently. Backend: Integrates Google Gemini AI capabilities for prompt engineering and task generation. Gesture Control: Utilizes the Leap Motion SDK for accurate gesture recognition. Future Developments Enhanced AI Integration: Further refinement of AI prompts to improve task generation accuracy and efficiency. Expanded Voice Command Features: Adding more voice functionalities for seamless kitchen assistance. User Feedback Mechanism: Implementing a system for users to provide feedback on tasks and improve the AI's learning process. Conclusion CHOP CHOP is designed to revolutionize the cooking experience for home cooks by combining gesture control, AI-driven task management, and voice commands. By addressing the challenges of multitasking and time management, we aim to enhance the overall quality of cooking and make meal preparation more enjoyable for everyone.
CHOP CHOP
Overview
CHOP CHOP is a hands-free, AI-powered cooking assistant designed to help home cooks, students, and those with ADHD manage multiple tasks in the kitchen more efficiently. By utilizing voice commands and gesture controls, CHOP CHOP ensures a smooth cooking experience, eliminating the risk of overcooking or undercooking meals.
The project integrates multiple hardware and software components, including Leap Motion for gesture control, Google Home for voice assistance, and a web-based interface for managing cooking tasks.
Problem Statement
Many people, especially college students and individuals with ADHD, struggle with multitasking and maintaining focus while cooking. This can lead to improperly cooked food and a less enjoyable cooking experience. CHOP CHOP solves this issue by providing an engaging, hands-free challenge that helps users stay organized and focused in the kitchen.
Features
- Voice Commands: Using Google Home and the Google Assistant SDK, CHOP CHOP allows you to control tasks and timers via voice commands.
- Gesture Control: Leap Motion Hand Tracking Sensor lets you move through the task list, pause timers, or add extra time with hand gestures.
- AI Task Generation: Using Google Gemini, CHOP CHOP can generate detailed cooking instructions based on user input and optimize task coordination for an efficient cooking experience.
- Dynamic Timer System: Each cooking task includes a "Start" button that triggers a countdown timer. Users can fast-forward, pause, or adjust the timer dynamically.
Components
Hardware:
- Leap Motion Hand Tracking Sensor: Enables hands-free control via gestures.
- Google Home Mini: Allows for voice-activated control using the Google Assistant SDK.
- Raspberry Pi: Connects the Leap Motion sensor to the web application and controls external signals (e.g., LEDs or sounds).
Software:
- Google Assistant SDK: Provides voice interaction for controlling tasks.
- Google Gemini: AI model used to generate structured task lists based on user input.
- Next.js: Web framework for building the front-end interface and managing routing.
- UI Library: Utilizes NextUI or similar libraries for simplified, responsive styling.
Prompt Engineering for Task Generation
CHOP CHOP uses prompt engineering to generate detailed, step-by-step cooking instructions.
Example Input:
"I want to cook a medium-rare sirloin steak and stir-fry unchopped onions with chopped and washed broccoli."
Example Output:
- Heat the pan on medium heat for 2 minutes and chop the onions.
- Season the steak with salt and pepper, and place it on the pan.
- Flip the steak after 4 minutes, then stir-fry onions in a separate pan for 3 minutes.
- Add broccoli to the stir-fry after 2 minutes and continue stirring for 3 more minutes.
The instructions are structured to optimize timing and minimize idle time.
Gesture Controls
- Swipe Gesture: Move on to the next task or fast-forward.
- Tap Gesture: Add more time to the current task.
- Clap Gesture: Pause or stop the timer.
The Leap Motion SDK communicates with the Raspberry Pi to process these gestures and control the app.
Next Steps
- Leap Motion Gesture Control: Map gestures to actions in the app using the Leap Motion SDK.
- AI Task Generation: Fine-tune prompts to produce efficient and detailed cooking instructions.
- Timer & Task Flow: Build the task list UI, integrate timers, and add functionality to fast-forward or extend time.
- Google Home Integration: Set up voice commands for starting tasks and controlling the cooking process.
- Hardware Integration: Use Raspberry Pi to manage communication between Leap Motion, external signals, and the app.
How to Run
- Clone the repository.
git clone https://github.com/drew-tom1/calhacks2024.git - Install dependencies.
npm install - Set development + environment variables.
- Run development server.
npm run dev
Analysis
View
Metric
- 19
- 12
- 2
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- ExpressIn code
- Google GeminiIn code
- JavaScriptIn code
- Next.jsIn code
- ReactIn code
- Tailwind CSSIn code
- TypeScriptIn code
- PythonClaimed
8 of 9 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
17 KB
Source files
15
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
tthacks24/chopchop
28 files · 848 KB · @ 313985b
Structure
Interface
8 files · 29%Screens, components and styles rendered to the user.
API & routing
5 files · 18%Request entry points: routes, handlers and controllers.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- JavaScript54%
- Markdown23%
- TypeScript21%
- CSS2%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
package.json
npm · 19- @clerk/nextjs
- @google/generative-ai
- @nextui-org/react
- @tanstack/react-query
- express
- framer-motion
- next
- react
- react-dom
- +10 more
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.