# Project export: VibeS

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 10.0
- Tagline: Given an image, generates a curated playlist based on the user's preferences that matches the VibeS of that image.
- Devpost: https://devpost.com/software/vibes-n2qa73
- GitHub: https://github.com/luisricardodiaz/CalHacks-Project
- Video: https://www.youtube.com/embed/SIptYXoCE3A?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 3 GitHub contributor(s) — Ricky Diaz (2 commits), Emmanuel Cobian Duarte (1 commits), Cris-Moran (1 commits)

## Devpost submission (written by the team)

### Inspiration

While at the CalHacks venue, a team member, Jeffrey, took a picture of the city buildings and wondered, "How can we capture the vibe of this picture?". The team had previously wanted to work on an AI and music-related project, so we got to work in trying to successfully capture the VibeS.

### What it does

Provided a picture and a Spotify account, the application will provide a curated playlist that captures the VibeS of the picture based on the user's preferences.

### How we built it

We trained a visual transformer BEIT from Hugging Face using Intel's Cloud Computing services that categorizes pictures. Using some of these categories, we then fetch from a Convex database populated with the user's songs from his "niche" playlists and provide the top results.

### Challenges we ran into

Making a dataset for the visual transformer was challenging as well as learning new technologies like Convex database mutation and querying.

### Accomplishments we're proud of

We are proud of building accurate models for the time of the day and environment classifications. We are also proud of being able to build an appealing front end for our project.

### What we learned

We learned a lot of skills such as training and testing models using Hugging Face and Intel's Cloud Computing Services, using Convex for database mutations and queries, fetching user information using Spotify's API, and getting more comfortable building in TypeScript and React.

### What's next

More data, more classifications for images, and more features such as instantly adding a playlist to Spotify, linking results to Spotify song links, etc.

## README (from the GitHub repository)

# Link to Devpost post and Demo

<a href = "https://devpost.com/software/vibes-n2qa73?ref_content=my-projects-tab&ref_feature=my_projects"> Link </a>


## Detected evidence (automated analysis)

Indexed codebase: 34 recognized source files, 50 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- Hugging Face (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (45 of 45)

```
_gitignore
.eslintrc.cjs
.gitignore
components.json
convex/_generated/api.d.ts
convex/_generated/api.js
convex/_generated/dataModel.d.ts
convex/_generated/server.d.ts
convex/_generated/server.js
convex/_schema.ts
convex/checkIfExists.ts
convex/daySongs.ts
convex/files.js
convex/morningSongs.ts
convex/myFunctions.ts
convex/mySongs.ts
convex/nightSongs.ts
convex/README.md
convex/tasks.ts
convex/tsconfig.json
day-time-songs.jsonl
index.html
morning-time-songs.jsonl
night-time-songs.jsonl
package.json
postcss.config.js
README.md
src/App.jsx
src/components/playlist.css
src/components/SpotifyPlaylist.jsx
src/components/typography/code.tsx
src/components/typography/link.tsx
src/components/ui/button.tsx
src/components/ui/checkbox.tsx
src/components/ui/input.tsx
src/components/ui/label.tsx
src/ImageSlider.jsx
src/index.css
src/lib/utils.tsx
src/main.jsx
src/vite-env.d.ts
tailwind.config.js
tsconfig.json
tsconfig.node.json
vite.config.ts
```

### Dependencies

- package.json: @hookform/resolvers@^3.3.2, @radix-ui/react-checkbox@^1.0.4, @radix-ui/react-icons@^1.3.0, @radix-ui/react-label@^2.0.2, @radix-ui/react-slot@^1.0.2, @types/node@^7.0.5, @types/node@^20.7.0, @types/react@^18.2.21, @types/react-dom@^18.2.7, @typescript-eslint/eslint-plugin@^6.7.0, @typescript-eslint/parser@^6.7.0, @uiball/loaders@^1.3.0, @vitejs/plugin-react@^4.0.4, @xixixao/uploadstuff@^0.0.5, autoprefixer@^10.4.16, axios@^1.6.0, bootstrap@^5.3.2, class-variance-authority@^0.7.0, clsx@^2.0.0, cmdk@^0.2.0, convex@^1.4.1, date-fns@^2.30.0, eslint@^8.49.0, eslint-plugin-react-hooks@^4.6.0, eslint-plugin-react-refresh@^0.4.3, node-fetch@^3.3.2, npm-run-all@^4.1.5, postcss@^8.4.30, react@^18.2.0, react-bootstrap@^2.9.1, react-day-picker@^8.9.1, react-dom@^18.2.0, react-hook-form@^7.47.0, tailwind-merge@^1.14.0, tailwindcss@^3.3.3, tailwindcss-animate@^1.0.7, typescript@^5.2.2, vite@^4.4.9, zod@^3.22.4

### Recent commits (newest first)

- Update README.md
- Update README.md
- Ricky (#4)
- first-commit
- Merge pull request #2 from get-convex/bonus-exercises
- Reformat
- Reformat
- Add bonus challenge exercises
- Relax eslint settings, reformat
- Call out intentional breakage & minor fixes
- Fix footer
- upgrade convex
- eslint
- Remove unused ui components/deps
- Install Node directly (not NVM)
- Mention intentional breakage earlier to avoid confusion
- Upgrade shadcn
- fix git URLs
- link to Convex
- More readme resources

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "convex-hack-pack",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "npm-run-all dev:init --parallel dev:frontend dev:backend",
    "dev:frontend": "vite --open",
    "dev:backend": "convex dev",
    "dev:init": "convex dev --until-success && convex dashboard",
    "build": "tsc && vite build",
    "lint": "eslint . --ext ts,tsx ",
    "preview": "vite preview"
  },
  "dependencies": {
    "@hookform/resolvers": "^3.3.2",
    "@radix-ui/react-checkbox": "^1.0.4",
    "@radix-ui/react-icons": "^1.3.0",
    "@radix-ui/react-label": "^2.0.2",
    "@radix-ui/react-slot": "^1.0.2",
    "@uiball/loaders": "^1.3.0",
    "@xixixao/uploadstuff": "^0.0.5",
    "axios": "^1.6.0",
    "bootstrap": "^5.3.2",
    "class-variance-authority": "^0.7.0",
    "clsx": "^2.0.0",
    "cmdk": "^0.2.0",
    "convex": "^1.4.1",
    "date-fns": "^2.30.0",
    "react": "^18.2.0",
    "react-bootstrap": "^2.9.1",
    "react-day-picker": "^8.9.1",
    "react-dom": "^18.2.0",
    "react-hook-form": "^7.47.0",
    "tailwind-merge": "^1.14.0",
    "tailwindcss-animate": "^1.0.7",
    "zod": "^3.22.4", 
    "@types/node": "^7.0.5"
  },
  "devDependencies": {
    "@types/node": "^20.7.0",
    "@types/react": "^18.2.21",
    "@types/react-dom": "^18.2.7",
    "@typescript-eslint/eslint-plugin": "^6.7.0",
    "@typescript-eslint/parser": "^6.7.0",
    "@vitejs/plugin-react": "^4.0.4",
    "autoprefixer": "^10.4.16",
    "eslint": "^8.49.0",
    "eslint-plugin-react-hooks": "^4.6.0",
    "eslint-plugin-react-refresh": "^0.4.3",
    "npm-run-all": "^4.1.5",
    "postcss": "^8.4.30",
    "tailwindcss": "^3.3.3",
    "typescript": "^5.2.2",
    "vite": "^4.4.9",
    "node-fetch": "^3.3.2"
  }
}

```

### src/main.jsx

```javascript

import { ConvexProvider, ConvexReactClient } from "convex/react";
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App";
import "./index.css";
const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL);

ReactDOM.createRoot(document.getElementById("root")).render(

    <ConvexProvider client={convex}>
      <App />
    </ConvexProvider>
);

```

### src/App.jsx

```javascript
import axios from "axios";
import { useQuery, useMutation } from "convex/react";
import { api } from "../convex/_generated/api";
import { Button } from "react-bootstrap";
import React, { useEffect, useRef, useState } from "react";
import ImageSlider from "./ImageSlider";
import { Waveform } from "@uiball/loaders";
import SpotifyPlaylist from "./components/SpotifyPlaylist";
import { data } from "autoprefixer";

const images = [
  "src/assets/image1.jpeg",
  "src/assets/image2.jpeg",
  "src/assets/image3.jpeg",
  "src/assets/image4.jpeg",
  "src/assets/image5.jpeg",
  "src/assets/image6.jpeg",
  "src/assets/image7.jpeg",
  "src/assets/image8.jpeg",
  "src/assets/image9.jpeg",
  "src/assets/image10.jpeg",
  "src/assets/image11.jpeg",
];

function App() {
  const CLIENT_ID = "ff3b13358d7a4b2bbc5a0f4df6112b5d";
  const REDIRECT_URI = "http://localhost:5173/";
  const AUTH_ENDPOINT = "https://accounts.spotify.com/authorize";
  const RESPONSE_TYPE = "token";
  const [token, setToken] = useState("");
  const [uniqueId, setUniqueId] = useState("");
  const mutatePlaylist = useMutation(api.tasks.createTask);
  const playlistToVibe = {
    forest: {
      Sunrise: "Liminal Space Mix",
      Daytime: "Autumn Mix",
      Nighttime: "Running Mix",
    },
    buildings: {
      Sunrise: "Hype Morning Mix",
      Daytime: "Driving Mix",
      Nighttime: "Late Night Mix",
    },
    mountains: {
      Sunrise: "Feel Good Morning Mix",
      Daytime: "Happy Camping Mix",
      Nighttime: "Camping Night Time Mix",
    },
    sea: {
      Sunrise: "Surfing Mix",
      Daytime: "Ocean Mix",
      Nighttime: "Nature Mix",
    },
  };
  const [isLoading, setIsLoading] = useState(false);
  const [imageUploaded, setImageUploaded] = useState(false);
  const [imageUrl, setImageUrl] = useState("");
  const [imageName, setImageName] = useState("");

  const [playlistOptions, setPlaylistOptions] = useState([])
  const [timeOfDayLabel, setTimeOfDayLabel] = useState("")
  const [environmentLabel, setEnvironmentLabel] = useState("")
  const [populatedDatabase, setPopulatedDatabase] = useState(false)

  const dataBasePopulatedHook = useQuery(api.checkIfExists.get, {tableName: uniqueId})
  console.log("type of database before: " + (typeof dataBasePopulatedHook))

  // three labels: Sunrise, Daytime, Nighttime
  // TODO: these labels are undefined at first, I'm not sure why
  // const labelToSongs = {
  //   Sunrise: morningSongs,
  //   Daytime: daySongs,
  //   Nighttime: nightSongs,
  // };

  const handleGeneratePlaylistClick = async () => {
    if (imageUrl == "") {
      alert("No image found! Did you upload an image?")
    }
    setIsLoading(true);

    async function scenesQuery(filename) {
      const data = await (await fetch(filename)).arrayBuffer();
      const response = await fetch(
        "https://api-inference.huggingface.co/models/jmillan736/BEiT-Scenes",
        {
          headers: {
            Authorization: `Bearer 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`,
          },
          method: "POST",
          body: data,
        }
      );
      const result = await response.json();
      if ("error" in result) {
        throw new Error("Environment Model Call failed! " + result.error);
      }
      return result;
    }

    async function timeQuery(filename) {
      const data = await (await fetch(filename)).arrayBuffer();
      const response = await fetch(
        "https://api-inference.huggingface.co/models/jmillan736/BEiT-Time-Of-Day",
        {
          headers: {
            Authorization: `Bearer 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`,
          },
          method: "POST",
          body: data,
        }
      );
      const result = await response.json();
      if ("error" in result) {
        throw new Error("Time Of Day Model Call failed! " + result.error);
      }
      return result;
    }

    async function callQuery(maxRetries, query) {
      if (maxRetries > 5) {
        setIsLoading(false);
        alert("API Call failed, please try again")
        throw new Error("Maximum number of API Calls exceeded");
      }
      try {
        const answer = await query(imageUrl);
        return answer;
      } catch (error) {
        setTimeout(() => {
          console.log(error + " Retrying...");
        }, 2000);
        await callQuery(maxRetries + 1, query);
      }
    }

    const environment = await callQuery(0, scenesQuery);
    const timeOfDay = await callQuery(0, timeQuery);

    console.log("Environment Labels: " + JSON.stringify(environment))
    console.log("Time of day labels: " + JSON.stringify(timeOfDay))

    setTimeOfDayLabel(timeOfDay[0].label);
    setEnvironmentLabel(environment[0].label);
    setIsLoading(false);
  };

  function isBackgroundColorWhite(element) {
    if (!element) {
      return false; // Return false if the element is not provided
    }
    const computedStyle = window.getComputedStyle(element);
    const backgroundColor = computedStyle.backgroundColor;

    // Check if the background color is white
    return backgroundColor === "rgb(255, 255, 255)";
  }

  function shuffleArray(array) {
    for (let i = array.length - 1; i > 0; i--) {
      const j = Math.floor(Math.random() * (i + 1));
      [array[i], array[j]] = [array[j], array[i]];
    }
    return array;
  }

  const element = document.querySelector("body"); // Replace with your element selector
  const isLightMode = isBackgroundColorWhite(element);
  let shuffleimage = shuffleArray(images);

  useEffect(() => {
    function generateRandomString(secret, length) {
      const encoder = new TextEncoder();
      const data = encoder.encode(secret);
      return window.crypto.subtle.digest("SHA-256", data).then((hashBuffer) => {
        const hashArray = Array.from(new Uint8Array(hashBuffer));
        const hashHex = hashArray
          .map((byte) => byte.toString(16).padStart(2, "0"))
          .join("");
        return hashHex.substr(0, length);
      });
    }

    const hash = window.location.hash;
[truncated — 5956 more characters]
```

### convex/_generated/server.js

```javascript
/* eslint-disable */
/**
 * Generated utilities for implementing server-side Convex query and mutation functions.
 *
 * THIS CODE IS AUTOMATICALLY GENERATED.
 *
 * Generated by convex@1.4.1.
 * To regenerate, run `npx convex dev`.
 * @module
 */

import {
  actionGeneric,
  httpActionGeneric,
  queryGeneric,
  mutationGeneric,
  internalActionGeneric,
  internalMutationGeneric,
  internalQueryGeneric,
} from "convex/server";

/**
 * Define a query in this Convex app's public API.
 *
 * This function will be allowed to read your Convex database and will be accessible from the client.
 *
 * @param func - The query function. It receives a {@link QueryCtx} as its first argument.
 * @returns The wrapped query. Include this as an `export` to name it and make it accessible.
 */
export const query = queryGeneric;

/**
 * Define a query that is only accessible from other Convex functions (but not from the client).
 *
 * This function will be allowed to read from your Convex database. It will not be accessible from the client.
 *
 * @param func - The query function. It receives a {@link QueryCtx} as its first argument.
 * @returns The wrapped query. Include this as an `export` to name it and make it accessible.
 */
export const internalQuery = internalQueryGeneric;

/**
 * Define a mutation in this Convex app's public API.
 *
 * This function will be allowed to modify your Convex database and will be accessible from the client.
 *
 * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument.
 * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible.
 */
export const mutation = mutationGeneric;

/**
 * Define a mutation that is only accessible from other Convex functions (but not from the client).
 *
 * This function will be allowed to modify your Convex database. It will not be accessible from the client.
 *
 * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument.
 * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible.
 */
export const internalMutation = internalMutationGeneric;

/**
 * Define an action in this Convex app's public API.
 *
 * An action is a function which can execute any JavaScript code, including non-deterministic
 * code and code with side-effects, like calling third-party services.
 * They can be run in Convex's JavaScript environment or in Node.js using the "use node" directive.
 * They can interact with the database indirectly by calling queries and mutations using the {@link ActionCtx}.
 *
 * @param func - The action. It receives an {@link ActionCtx} as its first argument.
 * @returns The wrapped action. Include this as an `export` to name it and make it accessible.
 */
export const action = actionGeneric;

/**
 * Define an action that is only accessible from other Convex functions (but not from the client).
 *
 * @param func - The function. It receives an {@link ActionCtx} as its first argument.
 * @returns The wrapped function. Include this as an `export` to name it and make it accessible.
 */
export const internalAction = internalActionGeneric;

/**
 * Define a Convex HTTP action.
 *
 * @param func - The function. It receives an {@link ActionCtx} as its first argument, and a `Request` object
 * as its second.
 * @returns The wrapped endpoint function. Route a URL path to this function in `convex/http.js`.
 */
export const httpAction = httpActionGeneric;

```

### postcss.config.js

```javascript
export default {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
  },
}

```

### vite.config.ts

```typescript
import path from "path";
import react from "@vitejs/plugin-react";
import { defineConfig } from "vite";

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [react()],
  resolve: {
    alias: {
      "@": path.resolve(__dirname, "./src"),
    },
  },
});

```

### index.html

```html
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/svg+xml" href="/vite.svg" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Cal Hacks Project</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/src/main.jsx"></script>
  </body>
</html>

```

### tailwind.config.js

```javascript
/** @type {import('tailwindcss').Config} */
module.exports = {
  darkMode: ["class"],
  content: ["./index.html", "./src/**/*.{js,ts,jsx,tsx}"],
  theme: {
    container: {
      center: true,
      padding: "2rem",
      screens: {
        "2xl": "1400px",
      },
    },
    extend: {
      colors: {
        border: "hsl(var(--border))",
        input: "hsl(var(--input))",
        ring: "hsl(var(--ring))",
        background: "hsl(var(--background))",
        foreground: "hsl(var(--foreground))",
        primary: {
          DEFAULT: "hsl(var(--primary))",
          foreground: "hsl(var(--primary-foreground))",
        },
        secondary: {
          DEFAULT: "hsl(var(--secondary))",
          foreground: "hsl(var(--secondary-foreground))",
        },
        destructive: {
          DEFAULT: "hsl(var(--destructive))",
          foreground: "hsl(var(--destructive-foreground))",
        },
        muted: {
          DEFAULT: "hsl(var(--muted))",
          foreground: "hsl(var(--muted-foreground))",
        },
        accent: {
          DEFAULT: "hsl(var(--accent))",
          foreground: "hsl(var(--accent-foreground))",
        },
        popover: {
          DEFAULT: "hsl(var(--popover))",
          foreground: "hsl(var(--popover-foreground))",
        },
        card: {
          DEFAULT: "hsl(var(--card))",
          foreground: "hsl(var(--card-foreground))",
        },
      },
      borderRadius: {
        lg: "var(--radius)",
        md: "calc(var(--radius) - 2px)",
        sm: "calc(var(--radius) - 4px)",
      },
      keyframes: {
        "accordion-down": {
          from: { height: 0 },
          to: { height: "var(--radix-accordion-content-height)" },
        },
        "accordion-up": {
          from: { height: "var(--radix-accordion-content-height)" },
          to: { height: 0 },
        },
      },
      animation: {
        "accordion-down": "accordion-down 0.2s ease-out",
        "accordion-up": "accordion-up 0.2s ease-out",
      },
    },
  },
  plugins: [require("tailwindcss-animate")],
};

```

### src/vite-env.d.ts

```typescript
/// <reference types="vite/client" />

```

### convex/daySongs.ts

```typescript
import { query } from "./_generated/server";

export const get = query({
  args: {},
  handler: async (ctx) => {
    return await ctx.db.query("daySongs").collect();
  },
});
```

[23 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]