# Project export: Droneventory

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: 10% of food is wasted before consumption. Droneventory automates inventory with drones, aiming to reduce errors by 90%. Seeking co-founders and mentors to scale and revolutionize inventory management.
- Devpost: https://devpost.com/software/droneventory
- GitHub: https://github.com/jjestrada2/aideck-gap8-examples
- Demo: https://droneventory.com/
- Video: https://www.youtube.com/embed/3CsEf3BoWgM?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 0 GitHub contributor(s) — 

## Devpost submission (written by the team)

### Overview

💡

### Inspiration

Having worked in the restaurant industry as a bartender, I noticed firsthand how challenging inventory management can be. Restaurants experience high inventory turnover because food and drinks spoil under varying conditions, such as fluctuating demand and seasonal changes. This makes inventory prediction difficult, and manual counting is both time-consuming and often inaccurate. These challenges inspired me to create Droneventory, a solution to automate and streamline the inventory process, saving time and reducing errors. 🚀

### What it does

Droneventory uses a Crazyflie drone equipped with object recognition to automate inventory counts. By scanning shelves and recognizing objects, it provides real-time updates, significantly reducing inventory errors and saving labor costs. 🛠️

### How we built it

At the hackathon, I assembled the drone using a Crazyflie board integrated with an accelerometer, a pressure sensor, an ESP32 module, and a GAP8 microcontroller with 8 cores for running lightweight machine learning models efficiently. I connected it to Amazon Rekognition, creating an API gateway via AWS S3 to recognize objects through the camera integrated with the ESP32 over Wi-Fi. 🧩

### Challenges we ran into

The biggest challenge was that the drone’s bootloader was outdated and didn’t support over-the-air flashing with the Crazyradio antenna. I needed a JTAG cable to flash my code, but none were available at the event. Unfortunately, my Amazon order for the cable is set to arrive after the hackathon. 🎉

### Accomplishments we're proud of

I'm proud to have made my first contribution to the open-source Crazyflie drone project, which is used by labs and research institutions worldwide. While my code hasn’t been accepted yet, submitting a pull request to such a significant project was a great milestone for me. 📚

### What we learned

I learned a lot about the SPI communication protocol and how microcontrollers like the ESP32 and GAP8 work. I also gained experience in reading open-source projects with large codebases, identifying and modifying the right files for my contribution. This project deepened my understanding of how to interface with sensors and control hardware for drone navigation. 🔮

### What's next

Next steps include finding a co-founder, refining the minimum viable product, and testing the system with local restaurants. Once proven, I plan to seek clients and scale the solution further.

## README (from the GitHub repository)

# AI-deck examples repository

Check out the [documentation](https://www.bitcraze.io/documentation/repository/aideck-gap8-examples/master/)
for starting guides. 


## Detected evidence (automated analysis)

Indexed codebase: 66 recognized source files, 547 KB.
- C (language) — detected in the code
- Python (language) — detected in the code
- TensorFlow (technology) — detected in the code

## Codebase structure (from repository index)

### Files (99 of 99)

```
.github/workflows/CI.yml
.github/workflows/release.yml
.gitignore
docs/ai-examples/classification-demo.md
docs/ai-examples/index.md
docs/guides/camera-replace.md
docs/guides/index.md
docs/guides/writing_aideck_gap8_code.md
docs/img-proc-examples/face-detection.md
docs/img-proc-examples/index.md
docs/index.md
docs/infrastructure/cpx.md
docs/infrastructure/flashing.md
docs/infrastructure/gap8.md
docs/infrastructure/index.md
docs/infrastructure/jtag-programmer.md
docs/other-examples/index.md
docs/other-examples/stm_gap8_cpx.md
docs/other-examples/test-camera.md
docs/other-examples/uart-send-char.md
docs/simple-examples/hello-world.md
docs/simple-examples/index.md
docs/simple-examples/wifi-streamer.md
examples/ai/classification/.gitignore
examples/ai/classification/classification.c
examples/ai/classification/classification.h
examples/ai/classification/config.ini
examples/ai/classification/docker_train_classifier.sh
examples/ai/classification/Makefile
examples/ai/classification/model_decl.mk
examples/ai/classification/model_rules.mk
examples/ai/classification/model/.gitignore
examples/ai/classification/model/nntool_script
examples/ai/classification/model/nntool_script_q
examples/ai/classification/requirements.txt
examples/ai/classification/samples/.gitignore
examples/ai/classification/train_classifier.py
examples/ai/classification/training_data/.gitignore
examples/ai/classification/training_data/instructions.txt
examples/ai/testset.cfg
examples/image_processing/FaceDetection/cascade.h
examples/image_processing/FaceDetection/config.ini
examples/image_processing/FaceDetection/DataTypes.h
examples/image_processing/FaceDetection/face_cascade.h
examples/image_processing/FaceDetection/faceDet.c
examples/image_processing/FaceDetection/faceDet.h
examples/image_processing/FaceDetection/FaceDetBasicKernels.c
examples/image_processing/FaceDetection/FaceDetBasicKernels.h
examples/image_processing/FaceDetection/FaceDetGenerator.c
examples/image_processing/FaceDetection/FaceDetGenerator.h
examples/image_processing/FaceDetection/FaceDetModel.c
examples/image_processing/FaceDetection/ImageDraw.c
examples/image_processing/FaceDetection/ImageDraw.h
examples/image_processing/FaceDetection/imgTest0.pgm
examples/image_processing/FaceDetection/imgTest1.pgm
examples/image_processing/FaceDetection/imgTest3.pgm
examples/image_processing/FaceDetection/main.c
examples/image_processing/FaceDetection/Makefile
examples/image_processing/FaceDetection/opencv-viewer.py
examples/image_processing/FaceDetection/setup.h
examples/image_processing/FaceDetection/StdTypes.h
examples/image_processing/FaceDetection/testset.cfg
examples/image_processing/FaceDetection/wifi.h
examples/image_processing/image_manipulations/img.ppm
examples/image_processing/image_manipulations/ImgIO.c
examples/image_processing/image_manipulations/ImgIO.h
examples/image_processing/image_manipulations/Makefile
examples/image_processing/image_manipulations/test.c
examples/image_processing/simple_kernel_example/img_proc.c
examples/image_processing/simple_kernel_example/img_proc.h
examples/image_processing/simple_kernel_example/Makefile
examples/image_processing/simple_kernel_example/README.md
examples/image_processing/simple_kernel_example/test.c
examples/other/com-test/com-test.c
examples/other/com-test/Makefile
examples/other/hello_world_gap8/hello_world_gap8.c
examples/other/hello_world_gap8/Makefile
examples/other/stm_gap8_cpx/Makefile
examples/other/stm_gap8_cpx/stm_gap8_cpx.c
examples/other/test_functionalities/test_camera/img_proc.c
examples/other/test_functionalities/test_camera/img_proc.h
examples/other/test_functionalities/test_camera/Makefile
examples/other/test_functionalities/test_camera/test.c
examples/other/test_functionalities/uart_send_char/Makefile
examples/other/test_functionalities/uart_send_char/test.c
examples/other/test_functionalities/uart_send_char/test.h
examples/other/wifi-img-streamer/config.ini
examples/other/wifi-img-streamer/Makefile
examples/other/wifi-img-streamer/opencv-viewer.py
examples/other/wifi-img-streamer/stream_out/debayer/.gitignore
examples/other/wifi-img-streamer/stream_out/raw/.gitignore
examples/other/wifi-img-streamer/wifi-img-streamer.c
examples/other/wifi-img-streamer/wifi.h
lib/cpx/inc/com.h
lib/cpx/inc/cpx.h
lib/cpx/src/com.c
lib/cpx/src/cpx.c
module.json
README.md
```

### Dependencies

- examples/ai/classification/requirements.txt: absl-py@==1.0.0, astunparse@==1.6.3, cachetools@==4.2.4, certifi@==2021.10.8, charset-normalizer@==2.0.9, docopt@==0.6.2, flatbuffers@==2.0, gast@==0.4.0, google-auth@==2.3.3, google-auth-oauthlib@==0.4.6, google-pasta@==0.2.0, grpcio@==1.43.0, h5py@==3.6.0, idna@==3.3, importlib-metadata@==4.10.0, keras@==2.11.0, libclang@==14.0.6, Markdown@==3.3.6, numpy@==1.21.5, oauthlib@==3.1.1, opt-einsum@==3.3.0, Pillow@==8.4.0, pipreqs@==0.4.11, protobuf@==3.19.1, pyasn1@==0.4.8, pyasn1-modules@==0.2.8, requests@==2.26.0, requests-oauthlib@==1.3.0, rsa@==4.8, scipy@==1.7.3, six@==1.16.0, tensorboard@==2.11.0, tensorboard-data-server@==0.6.1, tensorboard-plugin-wit@==1.8.1, tensorflow@==2.11.0, tensorflow-estimator@==2.11.0, tensorflow-io-gcs-filesystem@==0.29.0, termcolor@==1.1.0, typing_extensions@==4.0.1, urllib3@==1.26.7, Werkzeug@==2.0.2, wrapt@==1.13.3, yarg@==0.1.9, zipp@==3.6.0

### Recent commits (newest first)

- improve clarity
- improve readability and fix minor punctuation issues
- Clean up duplicate info
- Document training classification example with TF docker
- Support TensorFlow 2.17.0, implement TensorBoard logging
- Fix CI for gap_sdk v4.22.0
- Update examples for gap_sdk v4.22.0
- Doc that partition table offsets must be removed when flashing over JTAG
- Merge pull request #141 from bitcraze/rik/doc
- Improve classification demo documentation
- Merge pull request #139 from williamleong/williamleong/fix-face-detection.md
- Fix typo in radio URI example
- Merge pull request #138 from valeriyvan/patch-1
- Fix punctuation and typos
- Update stm_gap8_cpx.md
- update release file
- Moved jtag-programmer to infrastructure
- Added jtag-programmer.md
- Merge pull request #134 from bitcraze/krichardsson/stm-gap8-cpx
- Dummy commit to trigger CI

## Key source files (fetched from GitHub, selected and truncated for size)

### docs/index.md

```markdown
---
title: Home
page_id: home
---

This repository contains examples that can be run on the GAP8 as well as some supporting
documentation for developing for the GAP8. The examples are a good starting point for
creating your own applications.

The examples in this repository rely on the fact that the
[Getting started with the AI deck tutorial](https://www.bitcraze.io/documentation/tutorials/getting-started-with-aideck/)
has been completed. This means that all the firmware is up to date,
the GAP8 bootloader has been flashed and the Crazyflie is used
to set up the WiFi on the ESP32 in access point mode.

```

### docs/guides/index.md

```markdown
---
title: Guides
page_id: guides-index
sort_order: 6
---
This section contains guides.

{% sub_page_menu %}

```

### examples/ai/classification/requirements.txt

```
absl-py==1.0.0
astunparse==1.6.3
cachetools==4.2.4
certifi==2021.10.8
charset-normalizer==2.0.9
docopt==0.6.2
flatbuffers==2.0
gast==0.4.0
google-auth==2.3.3
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.43.0
h5py==3.6.0
idna==3.3
importlib-metadata==4.10.0
keras==2.11.0
libclang==14.0.6
Markdown==3.3.6
numpy==1.21.5
oauthlib==3.1.1
opt-einsum==3.3.0
Pillow==8.4.0
pipreqs==0.4.11
protobuf==3.19.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
requests==2.26.0
requests-oauthlib==1.3.0
rsa==4.8
scipy==1.7.3
six==1.16.0
tensorboard==2.11.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.11.0
tensorflow-estimator==2.11.0
tensorflow-io-gcs-filesystem==0.29.0
termcolor==1.1.0
typing_extensions==4.0.1
urllib3==1.26.7
Werkzeug==2.0.2
wrapt==1.13.3
yarg==0.1.9
zipp==3.6.0
```

### .github/workflows/CI.yml

```yaml
# CI jobs

name: CI

# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]
  schedule:
    # Weekly build
    - cron: '15 15 * * 0'
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout Repo
      uses: actions/checkout@v3
      with:
        submodules: true

    - name: build wifi-img-streamer example using build script
      run: docker run --rm -v ${PWD}:/module bitcraze/aideck tools/build/build

    - name: build test_camera example
      run: docker run --rm -v ${PWD}:/module bitcraze/aideck tools/build/make-example examples/other/test_functionalities/test_camera build image

    - name: build uart_send_char example
      run: docker run --rm -v ${PWD}:/module bitcraze/aideck tools/build/make-example examples/other/test_functionalities/uart_send_char build image

    - name: build com-test example
      run: docker run --rm -v ${PWD}:/module bitcraze/aideck tools/build/make-example examples/other/com-test build image

    - name: rename bin file
      run: cp examples/other/com-test/BUILD/GAP8_V2/GCC_RISCV_FREERTOS/target.board.devices.flash.img aideck_gap8.bin

    - name: Upload Build Artifact
      uses: actions/upload-artifact@v3
      with:
        name: ${{ github.sha }}
        path: aideck_gap8.bin

```

### .github/workflows/release.yml

```yaml
# Release jobs

name: Release

on:
  workflow_dispatch:

jobs:
  release:
    name: Create Release on Github
    runs-on: ubuntu-latest
    steps:
    - name: Create Release
      id: create_release
      uses: actions/create-release@v1
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      with:
        tag_name: ${{ github.ref }}
        release_name: ${{ github.ref }}
        draft: true
        prerelease: true
    - name: Release URL on file
      run: echo "${{ steps.create_release.outputs.upload_url }}" > release_url.txt
    - name: Save Release URL File For Uploading Files
      uses: actions/upload-artifact@v3
      with:
        name: release_url
        path: release_url.txt

  upload:
    name: Upload Binaries to release
    needs: [release]
    runs-on: ubuntu-latest

    steps:
    - name: Checkout Repo
      uses: actions/checkout@v3
      with:
        submodules: true

    - name: Build
      run: docker run --rm -v ${PWD}:/module bitcraze/aideck bash -c "./tools/build/build"

    - name: Load Release URL File from release job
      uses: actions/download-artifact@v3
      with:
        name: release_url

    - name: Get Release File Name & Upload URL
      id: get_release_info
      run: |
        value=`cat release_url.txt`
        echo ::set-output name=upload_url::$value

    - name: Get the version
      id: get_release_version
      env:
        GITHUB_REF : ${{ github.ref }}
      run: echo ::set-output name=release_version::${GITHUB_REF/refs\/tags\//}

    - name: Upload binary
      uses: actions/upload-release-asset@v1
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      with:
        upload_url: ${{ steps.get_release_info.outputs.upload_url }}
        asset_path: examples/other/wifi-img-streamer/BUILD/GAP8_V2/GCC_RISCV_FREERTOS/target.board.devices.flash.img
        asset_name: aideck_gap8_wifi_img_streamer_with_ap.bin
        asset_content_type: application/octet-stream

```

### examples/ai/classification/docker_train_classifier.sh

```shell
#!/usr/bin/env bash
set -e

full_path=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )

cd ${full_path}

pip install pillow scipy
python train_classifier.py

```

### examples/image_processing/FaceDetection/wifi.h

```c
#pragma once

#include <stdint.h>

typedef enum {
  WIFI_CTRL_SET_SSID                = 0x10,
  WIFI_CTRL_SET_KEY                 = 0x11,

  WIFI_CTRL_WIFI_CONNECT            = 0x20,

  WIFI_CTRL_STATUS_WIFI_CONNECTED   = 0x31,
  WIFI_CTRL_STATUS_CLIENT_CONNECTED = 0x32,
} __attribute__((packed)) WiFiCTRLType;

typedef struct {
  WiFiCTRLType cmd;
  uint8_t data[50];
} __attribute__((packed)) WiFiCTRLPacket_t;
```

### examples/other/wifi-img-streamer/wifi.h

```c
#pragma once

#include <stdint.h>

typedef enum {
  WIFI_CTRL_SET_SSID                = 0x10,
  WIFI_CTRL_SET_KEY                 = 0x11,

  WIFI_CTRL_WIFI_CONNECT            = 0x20,

  WIFI_CTRL_STATUS_WIFI_CONNECTED   = 0x31,
  WIFI_CTRL_STATUS_CLIENT_CONNECTED = 0x32,
} __attribute__((packed)) WiFiCTRLType;

typedef struct {
  WiFiCTRLType cmd;
  uint8_t data[50];
} __attribute__((packed)) WiFiCTRLPacket_t;
```

### examples/image_processing/simple_kernel_example/img_proc.h

```c
#ifndef __IMG_PROC_H__
#define __IMG_PROC_H__

#include "pmsis.h"

typedef struct {
    char *srcBuffer;     // pointer to the input vector
    char *resBuffer;     // pointer to the output vector
    uint32_t width;      // image width
    uint32_t height;     // image height
    uint32_t nPE;        // number of cores
    uint32_t grayscale;        // grayscale if one
} plp_example_kernel_instance_i32;

void demosaicking(char *input, char* output, int width, int height, int grayscale);
void cluster_demosaicking(void* args);
void inverting(char *input, char* output, int width, int height);
void cluster_inverting(void* args);

#endif
```

### examples/image_processing/FaceDetection/StdTypes.h

```c
/*
 * Copyright 2019 GreenWaves Technologies, SAS
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

#ifndef _STD_TYPE_H_
#define _STD_TYPE_H_

typedef signed char Word8;
typedef unsigned char Wordu8;

typedef short int Word16;
typedef unsigned short int Wordu16;

typedef int Word32;
typedef unsigned int Wordu32;

#endif


```

[54 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]