# Project export: AI Skincare Consultant

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: An AI guide that helps you navigate through personalized skincare, ingredient insights, and the complexities of the beauty industry.
- Devpost: https://devpost.com/software/ai-skincare-consultant
- GitHub: https://github.com/h-yuan12/faceai
- Demo: https://faceai-gelv.onrender.com/
- Video: https://www.youtube.com/embed/TtpVJY_3CWk?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — h-yuan12 (3 commits)

## Devpost submission (written by the team)

### Inspiration

Skincare is hard. Within the ever-expanding beauty industry, it is difficult to keep track of various products and ingredients that are safe to use. For those who are just starting, it can be even harder to navigate the plethora of options and understand what truly benefits their skin. Additionally, personal skin concerns can vary greatly, making it challenging to find tailored solutions without professional guidance. FaceAI Skincare Assistant was inspired by the need to democratize skincare knowledge, making personalized skincare accessible to everyone through the power of artificial intelligence.

### What it does

AI Skincare Consultant is a web application designed to simplify and personalize the skincare journey. By uploading a selfie, users can receive an analysis of their skin, identifying issues such as acne, oiliness, pigmentation, and wrinkles. The application then generates personalized skincare routines tailored to the user's unique skin profile, complete with product recommendations and detailed ingredient insights. Additionally, users can interact with the AI assistant to refine their preferences and gain in-depth knowledge about skincare ingredients, learning about the ingredients' usage and safety, ensuring informed and effective skincare choices.

### How we built it

The frontend is built with React while OpenCV.js handles image analysis to detect various skin conditions. The backend uses Firebase Cloud Functions and the Gemini 1.5 Flash model to process data and generate personalized skincare recommendations. Vertex AI is utilized for advanced machine learning capabilities, ensuring accurate and reliable analyses. For ingredient data, we use Cosmify's api, which has comprehensive data for various ingredients used in skincare products.

### Challenges we ran into

Image Analysis: It was difficult to get image analysis that was accurate. Skin conditions can often be difficult to identify and it was clear that there were still inaccuracies in our evaluations. As such, we had users confirm the conclusions that the AI drew in case they were incorrect. Personalization Complexity: Creatin personalized skincare routines involves considering a multitude of factors, including skin type, concerns, sensitivities, and user preferences, which added complexity to the recommendations.

### Accomplishments we're proud of

We are proud that we were able to leverage various technologies to create a web app that could potentially help many people in their skincare journeys.

### What we learned

Throughout our development process, we learned much about Gemini's capabilities and weaknesses. Although it is certainly powerful, there are aspects in image analysis where it fails, which was important to our project. Also, we were able to learn more about the development process, constantly finding ways to improve on our ideas.

### What's next

I would like to continue in the development of the skincare consultant, implementing various new features that can help users including Routine Progress Tracking Expanded Skin Condition Detection Direct Skincare Product Integration Enhanced Personalization Community Features

## README (from the GitHub repository)

# FaceAI Skincare Assistant

Let AI guide you through personalized skincare, ingredient insights, and the complexities of the beauty industry.

## Table of Contents

- [Introduction](#introduction)
- [Features](#features)
- [Live Demo](#live-demo)
- [Installation](#installation)
- [Usage](#usage)
- [Technologies Used](#technologies-used)
- [Contact](#contact)

## Introduction

AI Skincare Consultant is a web application designed to simplify and personalize the skincare journey. By uploading a selfie, users can receive an analysis of their skin, identifying issues such as acne, oiliness, pigmentation, and wrinkles. The application then generates personalized skincare routines tailored to the user's unique skin profile, complete with product recommendations and detailed ingredient insights. Additionally, users can interact with the AI assistant to refine their preferences and gain in-depth knowledge about skincare ingredients, learning about the ingredients' usage and safety, ensuring informed and effective skincare choices.

## Features

- **Image Analysis**: Upload a selfie to get a comprehensive analysis of your skin, including acne detection, oiliness/dryness levels, pigmentation assessment, and wrinkle identification using OpenCV.js.
- **Personalized Skincare Routines**: Receive AI-generated skincare routines tailored to your specific skin needs.
- **Ingredient Insights**: Explore detailed information about recommended skincare ingredients, including benefits and safety ratings based on EWG (Environmental Working Group) data.
- **Interactive Chat Interface**: Engage with the AI assistant to refine your skincare routine preferences and specifications.
- **Ingredient Details Modal**: Click on any ingredient to view in-depth details, including properties and safety information.
- **Responsive Design**: Accessible and user-friendly across various devices and screen sizes.

## Live Demo

Experience the application in action without any setup at https://faceai-gelv.onrender.com

## Installation

To run FaceAI Skincare Assistant locally, follow these steps:

### 1. Clone the Repository

git clone https://github.com/yourusername/faceai-skincare-assistant.git
cd faceai-skincare-assistant

### 2. Install Dependencies
Ensure you have Node.js installed. Then, install the required packages:
npm install


### 3. Start the Application
npm start
The application will run in development mode. Open http://localhost:3000 to view it in your browser.

### Usage
1. Upload Your Image
Click on the upload box to select a selfie from your device.

2. Review Skin Analysis
The AI will display detected skin conditions such as acne severity, oiliness, pigmentation, and wrinkles.

3. Confirm Traits
After reviewing, click "Confirm Traits" to proceed.

4. Specify Preferences
Provide any additional specifications or preferences for your skincare routine.

5. Receive Personalized Routine
The AI will generate a detailed skincare routine tailored to your needs, including product recommendations and ingredient insights.

6. Explore Ingredients
Click on any ingredient in the sidebar to view detailed information about its properties and safety ratings.

### Technologies Used
Frontend:
- React
- OpenCV.js
- Firebase
- Vertex AI
- Axios

Backend:
- Firebase Cloud Functions
- Gemini AI Model

Others:
- CSS3
- HTML5

### Contact
For any inquiries or feedback, please reach out to:

Email: harry_yuan@berkeley.edu

## Detected evidence (automated analysis)

Indexed codebase: 14 recognized source files, 61 KB.
- CSS (language) — detected in the code
- Express (technology) — detected in the code
- Firebase (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- React (technology) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (23 of 23)

```
.firebaserc
.gitignore
firebase.json
functions/.eslintrc.js
functions/.gitignore
functions/index.js
functions/package.json
package.json
public/index.html
public/manifest.json
public/robots.txt
README.md
src/App.css
src/App.js
src/App.test.js
src/components/data/all_ingredients.json
src/components/UploadPage.css
src/components/UploadPage.js
src/firebase.js
src/index.css
src/index.js
src/reportWebVitals.js
src/setupTests.js
```

### Dependencies

- functions/package.json: @babel/plugin-proposal-private-property-in-object@^7.21.11, axios@^1.4.0, cors@^2.8.5, eslint@^8.15.0, eslint-config-google@^0.14.0, firebase-admin@^12.1.0, firebase-functions@^6.0.1, firebase-functions-test@^3.1.0
- package.json: @google-ai/generativelanguage@^2.7.0, @google-cloud/storage@^7.13.0, @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, axios@^1.7.7, busboy@^1.6.0, cors@^2.8.5, express@^4.21.1, firebase@^10.14.1, react@^18.3.1, react-circular-progressbar@^2.1.0, react-dom@^18.3.1, react-router-dom@^6.27.0, react-scripts@5.0.1, web-vitals@^2.1.4

### Recent commits (newest first)

- changed cosmify
- changed cosmify
- changed cosmify
- changed cosmify
- changed cosmify
- readme update
- readme update
- updated README
- finished v1
- first commit
- Initialize project using Create React App

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "faceai",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@google-ai/generativelanguage": "^2.7.0",
    "@google-cloud/storage": "^7.13.0",
    "@testing-library/jest-dom": "^5.17.0",
    "@testing-library/react": "^13.4.0",
    "@testing-library/user-event": "^13.5.0",
    "axios": "^1.7.7",
    "busboy": "^1.6.0",
    "cors": "^2.8.5",
    "express": "^4.21.1",
    "firebase": "^10.14.1",
    "react": "^18.3.1",
    "react-circular-progressbar": "^2.1.0",
    "react-dom": "^18.3.1",
    "react-router-dom": "^6.27.0",
    "react-scripts": "5.0.1",
    "web-vitals": "^2.1.4"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

```

### functions/package.json

```
{
  "name": "functions",
  "description": "Cloud Functions for Firebase",
  "scripts": {
    "lint": "eslint .",
    "serve": "firebase emulators:start --only functions",
    "shell": "firebase functions:shell",
    "start": "npm run shell",
    "deploy": "firebase deploy --only functions",
    "logs": "firebase functions:log"
  },
  "engines": {
    "node": "18"
  },
  "main": "index.js",
  "dependencies": {
    "firebase-admin": "^12.1.0",
    "firebase-functions": "^6.0.1",
    "axios": "^1.4.0",              
    "cors": "^2.8.5"        
  },
  "devDependencies": {
    "@babel/plugin-proposal-private-property-in-object": "^7.21.11",
    "eslint": "^8.15.0",
    "eslint-config-google": "^0.14.0",
    "firebase-functions-test": "^3.1.0"
  },
  "private": true
}

```

### src/App.js

```javascript
// src/App.js
import React from 'react';
import { BrowserRouter as Router, Routes, Route, Navigate } from 'react-router-dom';
import UploadPage from './components/UploadPage';
import './App.css';

function App() {
  return (
    <Router>
      <Routes>
        <Route path="/upload" element={<UploadPage />} />
        {/* Catch-all route that redirects to /upload */}
        <Route path="*" element={<Navigate to="/upload" />} />
      </Routes>
    </Router>
  );
}

export default App;

```

### src/index.js

```javascript
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

```

### functions/index.js

```javascript
// functions/index.js

const {onRequest} = require("firebase-functions/v2/https");
const logger = require("firebase-functions/logger");
const axios = require("axios");
const cors = require("cors")({origin: true});

// Create and deploy your first functions
// https://firebase.google.com/docs/functions/get-started

// Function to analyze ingredient list via Cosmily API
exports.analyzeIngredientList = onRequest(async (req, res) => {
  // Handle CORS
  await new Promise((resolve) => {
    cors(req, res, resolve);
  });

  // Only allow POST requests
  if (req.method !== "POST") {
    res.status(405).json({message: "Method Not Allowed. Use POST."});
    return;
  }

  const {ingredients, ingredientGroup} = req.body;

  // Basic validation
  if (!ingredients || !ingredientGroup) {
    res.status(400).json({
      message: "Missing 'ingredients' or 'ingredientGroup' in request body.",
    });
    return;
  }

  try {
    // Fetch the access token from environment variables
    const ACCESS_TOKEN = "";

    // if (!ACCESS_TOKEN) {
    //   logger.error(
    //       "COSMILY_ACCESS_TOKEN is not set in environment variables.",
    //   );
    //   res.status(500).json({message: "Server configuration error."});
    //   return;
    // }

    // Make the request to Cosmily API
    const cosmilyResponse = await axios.post(
        "https://api.cosmily.com/api/v1/analyze/ingredient_list",
        {
          ingredients,
          ingredientGroup,
        },
        {
          headers: {
            "Content-Type": "application/json",
            "Authorization": `Bearer ${ACCESS_TOKEN}`,
          },
        },
    );

    // Forward the response from Cosmily API to the frontend
    res.status(cosmilyResponse.status).json(cosmilyResponse.data);
  } catch (error) {
    logger.error("Error communicating with Cosmily API:", error.message);

    if (error.response) {
      res.status(error.response.status).json({
        message: error.response.data.message || "Error from Cosmily API",
      });
    } else if (error.request) {
      // The request was made, but no response was received
      res.status(500).json({message: "No response from Cosmily API."});
    } else {
      // Something happened in setting up the request
      res.status(500).json({message:
        "Error setting up request to Cosmily API."});
    }
  }
});

```

### src/setupTests.js

```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

```

### src/App.test.js

```javascript
import { render, screen } from '@testing-library/react';
import App from './App';

test('renders learn react link', () => {
  render(<App />);
  const linkElement = screen.getByText(/learn react/i);
  expect(linkElement).toBeInTheDocument();
});

```

### src/App.css

```css
.App {
  text-align: center;
  padding: 50px;
}

input[type="file"] {
  margin: 20px 0;
}

button {
  padding: 10px 20px;
  background-color: #67CAC7;
  color: #fff;
  border: none;
  border-radius: 5px;
  cursor: pointer;
}

progress {
  width: 100%;
  margin: 20px 0;
}

```

### src/reportWebVitals.js

```javascript
const reportWebVitals = onPerfEntry => {
  if (onPerfEntry && onPerfEntry instanceof Function) {
    import('web-vitals').then(({ getCLS, getFID, getFCP, getLCP, getTTFB }) => {
      getCLS(onPerfEntry);
      getFID(onPerfEntry);
      getFCP(onPerfEntry);
      getLCP(onPerfEntry);
      getTTFB(onPerfEntry);
    });
  }
};

export default reportWebVitals;

```

### src/index.css

```css
body {
  margin: 0;
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
    'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
    sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

code {
  font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',
    monospace;
}

```

[5 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]