# Project export: Health Receipt This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2024 - Tagline: Unlocking the Secrets of Your Plate, One Bite at a Time! - Devpost: https://devpost.com/software/health-receipt - GitHub: https://github.com/chrehall68/health-receipt - Team: 3 GitHub contributor(s) — chrehall68 (26 commits), Atishay Jain (1 commits), sravangorati2001 (1 commits) ## Devpost submission (written by the team) ### Inspiration and Beginning The journey of developing our food transparency app began with a simple yet powerful idea: to empower consumers with the knowledge they need to make informed decisions about the food they buy and consume. Inspired by the vision of Open Food Facts, we were driven by the belief that everyone should have access to transparent information about the products they consume daily. This vision resonated deeply with our team, igniting a passion to contribute to a more transparent and sustainable food system. ### What it does Health Receipt is a revolutionary app designed to provide consumers with comprehensive information about the food products they purchase. Its core functionalities include: OCR-Based Receipt Scanning: Using optical character recognition (OCR) technology, Health Receipt scans grocery store receipts to extract detailed information about the food items listed. Nutrition Information: The app retrieves nutrition facts for each scanned product, including calorie count, macronutrient breakdown, and ingredient list. Eco-Scores and Packaging Information: Health Receipt evaluates the environmental impact of each product by analyzing packaging materials and carbon footprint, providing users with eco-scores to guide sustainable purchasing decisions. Allergen Warnings: The app identifies potential allergens present in scanned products, helping users with dietary restrictions make safe choices. Recipe Creation: Health Receipt offers a unique feature that allows users to generate recipes based on the ingredients scanned from their receipts, promoting creative and healthy meal planning. ### How we built it Typescript: We chose TypeScript for its strong typing system and scalability, enabling us to write cleaner and more maintainable code. Bun: The backend of Health Receipt is powered by Bun, providing a fast and efficient server environment for processing user requests and handling data. npm: We utilized npm, the package manager for JavaScript, to integrate various libraries and dependencies into our project, streamlining development and ensuring compatibility. Convex: Leveraging the Convex framework, we implemented features such as text search, server functions, and real-time updates, enhancing the performance and functionality of our app's backend. Google Vision API: For OCR functionality, we integrated the Google Vision API, which enabled us to extract text data from grocery store receipts with high accuracy and reliability. Together AI: Collaborating with Together AI, we incorporated innovative features like recipe creation into Health Receipt, enriching the user experience and adding value to the app. ### Challenges we ran into Throughout the hackathon, we encountered numerous challenges that tested our perseverance and problem-solving skills. From overcoming technical hurdles to refining user interfaces for optimal usability, each obstacle presented an opportunity for growth and innovation. One of our significant challenges was ensuring the accuracy and reliability of OCR data extraction, requiring meticulous fine-tuning and validation processes. With each hurdle overcame, we emerged stronger and more resilient, fueled by our passion for catalyzing positive change in the food industry. ### Accomplishments we're proud of Our integration with the Open Food Facts database has enabled us to contribute to the broader movement for food transparency. By leveraging this vast repository of product data, we're facilitating greater transparency and accountability within the food industry. ### What we learned Through the development of Health Receipt, we gained invaluable insights into the importance of teamwork and collaboration. Our team dynamics played a crucial role in driving the project forward, with each member contributing their unique strengths and expertise. We learned to leverage individual strengths to overcome challenges and capitalize on opportunities, fostering a supportive and cohesive working environment. Additionally, we became more attuned to recognizing and addressing our team's weaknesses, embracing a culture of continuous improvement and open communication. Ultimately, our journey with Health Receipt reinforced the significance of teamwork in achieving shared goals and delivering impactful solutions to real-world problems. ## README (from the GitHub repository) # Health-Receipt Understanding how healthy your shopping list is is now as easy as simply scanning your receipt! Simply open the app, scan your Walmart receipt, and start understanding what's going into your body. From essential nutrients to harmful chemicals, our application details them all in an easily-understandable way. Get started today! ## Team This application was built with ❤️ by * Eliot H. * Atishay J. * Sravan K. * Rohit ## Data Flow For developers, here's how everything works. When you scan your receipt, our OCR model extracts what products you bought and securely sends that data to our Convex backend, which then looks up the products from Walmart, retrieves the information about the foods from our database, and then sends that data back to our frontend, which displays it. ## Getting Started Developing! * First, make sure to run `bun install` (or `npm install`) in order to install all necessary packages * Next, run the backend by running `npx convex dev` * Then, install Expo Go on the mobile device that you will be viewing the frontend * Finally, run `bun start` (or `npm start`) in order to start the frontend. Open Expo Go on your device and scan the QR code, and then you'll see the application there! ## Detected evidence (automated analysis) Indexed codebase: 39 recognized source files, 76 KB. - JavaScript (language) — detected in the code - React (technology) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (45 of 45) ``` .gitignore app.json app/_layout.tsx app/(tabs)/_layout.tsx app/(tabs)/history.tsx app/(tabs)/home.tsx app/(tabs)/scan.tsx app/+html.tsx app/creds.ts app/hooks.ts app/index.tsx app/login.tsx app/main.tsx app/modal.tsx app/order/[orderId].tsx app/order/about.tsx app/store.ts babel.config.js components/__tests__/StyledText-test.js components/ItemCard.tsx components/OrderDisplay.tsx components/StyledCard.tsx components/StyledText.tsx components/Themed.tsx components/useClientOnlyValue.ts components/useClientOnlyValue.web.ts components/useColorScheme.ts components/useColorScheme.web.ts constants/Colors.ts constants/values.ts convex/_generated/api.d.ts convex/_generated/api.js convex/_generated/dataModel.d.ts convex/_generated/server.d.ts convex/_generated/server.js convex/item.ts convex/openfoodfacts.ts convex/README.md convex/schema.ts convex/togetherai.ts convex/tsconfig.json package.json README.md sample_receipt.jsonl tsconfig.json ``` ### Dependencies - package.json: @babel/core@^7.20.0, @expo/vector-icons@^14.0.0, @google-cloud/vision@^4.1.0, @react-navigation/native@^6.0.2, @reduxjs/toolkit@^2.2.1, @types/react@~18.2.45, convex@^1.9.0, expo@~50.0.7, expo-camera@^14.0.5, expo-file-system@^16.0.6, expo-font@~11.10.3, expo-image-picker@^14.7.1, expo-linking@^6.2.2, expo-router@~3.4.7, expo-splash-screen@~0.26.4, expo-status-bar@~1.11.1, expo-system-ui@~2.9.3, expo-web-browser@~12.8.2, jest@^29.2.1, jest-expo@~50.0.2, react@18.2.0, react-dom@^18.2.0, react-native@0.73.4, react-native-circular-progress@^1.3.9, react-native-dotenv@^3.4.10, react-native-fs@^2.20.0, react-native-get-random-values@^1.10.0, react-native-image-base64@^0.1.4, react-native-paper@^5.12.3, react-native-safe-area-context@^4.8.2, react-native-screens@~3.29.0, react-native-uuid@^2.0.1, react-native-vector-icons@^10.0.3, react-native-web@~0.19.6, react-redux@^9.1.0, react-test-renderer@18.2.0, recharts@^2.12.0, types@^0.1.1, typescript@^5.1.3 ### Recent commits (newest first) - Merge pull request #7 from chrehall68/ui-patches - ui patches - Merge pull request #6 from chrehall68/db-conns - add recipe connection - add about the fields page - hook up data pipeline for scanning receipts - Merge pull request #5 from chrehall68/icons - Merge branch 'main' into icons - finish updating atishay's script - Merge pull request #4 from Atishaysjain/ocr - Added ocr file with score prediction, updated model for food items and no text photo detection - update readme, add more icons and visuals - together ai changes - add icons to nutrition facts - Merge pull request #3 from chrehall68/openfoodapi - add nutrition fact table and allergen display - fix schema - Merge pull request #2 from chrehall68/login - fix navigation bug - login ## Key source files (fetched from GitHub, selected and truncated for size) ### package.json ``` { "name": "health-receipt", "main": "expo-router/entry", "version": "1.0.0", "scripts": { "start": "expo start", "android": "expo start --android", "ios": "expo start --ios", "web": "expo start --web", "test": "jest --watchAll" }, "jest": { "preset": "jest-expo" }, "dependencies": { "@expo/vector-icons": "^14.0.0", "@google-cloud/vision": "^4.1.0", "@react-navigation/native": "^6.0.2", "@reduxjs/toolkit": "^2.2.1", "convex": "^1.9.0", "expo": "~50.0.7", "expo-camera": "^14.0.5", "expo-file-system": "^16.0.6", "expo-font": "~11.10.3", "expo-image-picker": "^14.7.1", "expo-linking": "^6.2.2", "expo-router": "~3.4.7", "expo-splash-screen": "~0.26.4", "expo-status-bar": "~1.11.1", "expo-system-ui": "~2.9.3", "expo-web-browser": "~12.8.2", "react": "18.2.0", "react-dom": "^18.2.0", "react-native": "0.73.4", "react-native-circular-progress": "^1.3.9", "react-native-dotenv": "^3.4.10", "react-native-fs": "^2.20.0", "react-native-get-random-values": "^1.10.0", "react-native-image-base64": "^0.1.4", "react-native-paper": "^5.12.3", "react-native-safe-area-context": "^4.8.2", "react-native-screens": "~3.29.0", "react-native-uuid": "^2.0.1", "react-native-vector-icons": "^10.0.3", "react-native-web": "~0.19.6", "react-redux": "^9.1.0", "recharts": "^2.12.0", "types": "^0.1.1" }, "devDependencies": { "@babel/core": "^7.20.0", "@types/react": "~18.2.45", "jest": "^29.2.1", "jest-expo": "~50.0.2", "react-test-renderer": "18.2.0", "typescript": "^5.1.3" }, "private": true } ``` ### app/main.tsx ```typescript import { Title } from "@/components/StyledText"; import { Stack, router } from "expo-router"; import { View } from "react-native"; import { Button } from "react-native-paper"; export default function Index() { return Health-Receipt } ``` ### app/index.tsx ```typescript import { Title } from "@/components/StyledText"; import { Stack, router } from "expo-router"; import { View } from "react-native"; import { Button } from "react-native-paper"; export default function Index() { return Health-Receipt } ``` ### app/_layout.tsx ```typescript import FontAwesome from '@expo/vector-icons/FontAwesome'; import { DarkTheme, DefaultTheme, ThemeProvider } from '@react-navigation/native'; import { useFonts } from 'expo-font'; import { Stack } from 'expo-router'; import * as SplashScreen from 'expo-splash-screen'; import { useEffect, useState } from 'react'; import { useColorScheme } from '@/components/useColorScheme'; import { ConvexProvider, ConvexReactClient } from "convex/react"; import "react-native-get-random-values"; import { PaperProvider } from 'react-native-paper'; import { Provider } from 'react-redux' import { store } from './store'; const convex = new ConvexReactClient(process.env.CONVEX_URL!, { unsavedChangesWarning: false, }); export { // Catch any errors thrown by the Layout component. ErrorBoundary, } from 'expo-router'; export const unstable_settings = { // Ensure that reloading on `/modal` keeps a back button present. initialRouteName: '(tabs)', }; // Prevent the splash screen from auto-hiding before asset loading is complete. SplashScreen.preventAutoHideAsync(); export default function RootLayout() { const [loaded, error] = useFonts({ SpaceMono: require('../assets/fonts/SpaceMono-Regular.ttf'), ...FontAwesome.font, }); // Expo Router uses Error Boundaries to catch errors in the navigation tree. useEffect(() => { if (error) throw error; }, [error]); useEffect(() => { if (loaded) { SplashScreen.hideAsync(); } }, [loaded]); if (!loaded) { return null; } return ; } function RootLayoutNav() { const colorScheme = useColorScheme(); return ( ); } ``` ### app/(tabs)/_layout.tsx ```typescript import React from 'react'; import FontAwesome from '@expo/vector-icons/FontAwesome'; import { Link, Tabs } from 'expo-router'; import { Pressable } from 'react-native'; import Colors from '@/constants/Colors'; import { useColorScheme } from '@/components/useColorScheme'; import { useClientOnlyValue } from '@/components/useClientOnlyValue'; // You can explore the built-in icon families and icons on the web at https://icons.expo.fyi/ function TabBarIcon(props: { name: React.ComponentProps['name']; color: string; }) { return ; } export default function TabLayout() { const colorScheme = useColorScheme(); return ( , headerRight: () => ( {({ pressed }) => ( )} ), }} /> , headerRight: () => ( {({ pressed }) => ( )} ), }} /> , headerRight: () => ( {({ pressed }) => ( )} ), }} /> ); } ``` ### convex/_generated/server.js ```javascript /* eslint-disable */ /** * Generated utilities for implementing server-side Convex query and mutation functions. * * THIS CODE IS AUTOMATICALLY GENERATED. * * Generated by convex@1.9.0. * To regenerate, run `npx convex dev`. * @module */ import { actionGeneric, httpActionGeneric, queryGeneric, mutationGeneric, internalActionGeneric, internalMutationGeneric, internalQueryGeneric, } from "convex/server"; /** * Define a query in this Convex app's public API. * * This function will be allowed to read your Convex database and will be accessible from the client. * * @param func - The query function. It receives a {@link QueryCtx} as its first argument. * @returns The wrapped query. Include this as an `export` to name it and make it accessible. */ export const query = queryGeneric; /** * Define a query that is only accessible from other Convex functions (but not from the client). * * This function will be allowed to read from your Convex database. It will not be accessible from the client. * * @param func - The query function. It receives a {@link QueryCtx} as its first argument. * @returns The wrapped query. Include this as an `export` to name it and make it accessible. */ export const internalQuery = internalQueryGeneric; /** * Define a mutation in this Convex app's public API. * * This function will be allowed to modify your Convex database and will be accessible from the client. * * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument. * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible. */ export const mutation = mutationGeneric; /** * Define a mutation that is only accessible from other Convex functions (but not from the client). * * This function will be allowed to modify your Convex database. It will not be accessible from the client. * * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument. * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible. */ export const internalMutation = internalMutationGeneric; /** * Define an action in this Convex app's public API. * * An action is a function which can execute any JavaScript code, including non-deterministic * code and code with side-effects, like calling third-party services. * They can be run in Convex's JavaScript environment or in Node.js using the "use node" directive. * They can interact with the database indirectly by calling queries and mutations using the {@link ActionCtx}. * * @param func - The action. It receives an {@link ActionCtx} as its first argument. * @returns The wrapped action. Include this as an `export` to name it and make it accessible. */ export const action = actionGeneric; /** * Define an action that is only accessible from other Convex functions (but not from the client). * * @param func - The function. It receives an {@link ActionCtx} as its first argument. * @returns The wrapped function. Include this as an `export` to name it and make it accessible. */ export const internalAction = internalActionGeneric; /** * Define a Convex HTTP action. * * @param func - The function. It receives an {@link ActionCtx} as its first argument, and a `Request` object * as its second. * @returns The wrapped endpoint function. Route a URL path to this function in `convex/http.js`. */ export const httpAction = httpActionGeneric; ``` ### babel.config.js ```javascript module.exports = function (api) { api.cache(true); return { presets: ['babel-preset-expo'], plugins: ["module:react-native-dotenv"], env: { production: { plugins: ['react-native-paper/babel'] } } }; }; ``` ### components/useColorScheme.ts ```typescript export { useColorScheme } from 'react-native'; ``` ### constants/values.ts ```typescript export const small = 4; export const medium = 8; export const large = 16; export const lxl = 24; export const xl = 32; ``` ### components/useClientOnlyValue.ts ```typescript // This function is web-only as native doesn't currently support server (or build-time) rendering. export function useClientOnlyValue(server: S, client: C): S | C { return client; } ``` [28 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]