Project Info

Plantcasso

Inspiration

The starting point for our project was artist David Bowen's Plant Machete, where plant signals were mapped into control signals for a robot arm holding a machete. We decided to take it one step further--what if the temporal changes of the artistic expression were retained? And better yet if this piece of technoart also had some practical uses.

What it does

This landed us with Plantcasso. By mapping a plant's electric biosignals into servo control signals, we are able to visualize through both the arm's movements and drawn brushstrokes a plant's hidden signs of life and signs of agitation from the environment. This in turn converts our piece into a multi-functional installation: it is as much a work of performance art as visual art; the plant's paintings can serve as true random signals for cryptographic hashes akin to CloudFlare's lava lamps; anomalies such as wildfires, air quality degredation, or plant sickness can be detected AND visually presented as chaos in the plant's paintings.

How we built it

The project can be broken down into four main segments. The first and most critical is reading the plant signals. By using conductive gel pads and an INA333 instrument amplifier we are able to detect and boost the voltage differences across a plant's leaves and stem, which are then read into an ESP32-S3 Supermini. The second part of the project is an unsupervised clustering model. With a dataset of collected plant readings, we sample sequences, extract a number of meaningful features, and project them into 3D space, after which we run a clustering algorithm. This is trained on a laptop; the resulting weights are deployed onto the ESP32-S3 for real-time inference. The third segment is the arm control: we 3D printed a robot arm motivated by 5 MG-90 servos, and map the plant signals to points in the legal range of motion such that as the plant lives, so does the arm--and so does the painting. Finally, we implement a deshboard where we can see rolling graphs of information such as the plant's agitation, detected voltage, and spikes in activity.

Challenges we ran into

The first challenge was just getting the signals at all. Plant biosignals are infamously weak and noisy. A lot of work went into filtering the signal to keep it as clear and clean as possible; not just through software filters but also in hardware with changing pad adhesion points periodically, braiding cables, and building physical isolation. The second challenge was finding features that made sense for anomaly detection. A little literature research yielded the hjorth complexity as a good indicator of externally-induced spikes, and thus was weighed more heavily in our final clustering implementation. Finally, the largest nightmare was also teh simplest: getting the servos to play nice. Using a 16-channel servo control board over I2C was fickle, to say the least. For this we relied on redundancies and checks, but at the end of the day we have little choice but to cross our fingers.

Accomplishments we're proud of

The project works--it reads signals, it reacts to agitation, and the arm does what it's supposed to. The fact that such an abstract idea was able to be realized in such a short time is amazing enough in and of itself. The clustering was clean, the arm looked alive, and aside from lacking a more reliable servo motor control interface it's just about everything we envisioned it to be.

What we learned

The devil's in the details, and it's the things you least expect that might trip you up. We thought processing the plant signals would be the hard part, but it was fighting with the servo motors that kept us up all night. ALWAYS verify your hardware works! We also learned that mint plant stems are more fragile than expected.

What's next

We're going to build a better hardware rig and try a broader range of plants than just our proof-of-concept mint. The idea is to gather a generalized dataset and map corresponding clusters for a variety of environmental stimuli such as fire, carbon dioxide concentrations, and other signals that we can't test at the venue but would make this infinitely more practical and useful in disaster-prone areas.

PlantCasso

Turning a plant's bioelectric signals into expressive robot-arm motion. Built at the Berkeley AI Hackathon 2026.

An electrode on a plant is sampled by an ESP32-S3 ADC. The signal is filtered, reduced to 9 cheap time-domain features per window, projected to 3D with PCA and clustered with K-means. The resulting embedding (plus two raw features) drives a 5-DOF servo arm so the plant's electrical "mood" becomes visible movement.

plant electrode → ESP32 ADC → lowpass filter → 9 features → StandardScaler
              → PCA (3D) → K-means cluster → servo angles (smoothed) → arm

Repository layout

Path	What it is
`pipeline/`	Offline Python ML pipeline (filter → features → PCA/K-means → C header)
`pipeline/out/`	Generated artifacts (figures, model, CSVs). Git-ignored — regenerate by running the pipeline
`plant_inference/`	ESP32-S3 firmware that runs real-time inference + servo control
`plant_inference/model_params.h`	Auto-generated C header (scaler/PCA/K-means/filter) — produced by `pipeline/02_train.py`
`POC_electrode_reader/`	Minimal Arduino sketch that streams raw ADC voltage over serial
`data/`	Recorded voltage datasets (`data_5hz.csv`, `data_100hz.csv`, `data_unhealthy.csv`)
`data.csv`	Raw capture from a collection session
`data_collection.py`	Logs serial voltage samples from the ESP32 to a CSV
`Base.3mf`	3D-printable arm base model
`ref_plantsignal.md`, `ref_plantsignal_converter.py`	Design notes / reference end-to-end script
`AGENTS.md`	Embedded C++ coding guidelines for the firmware

Quick start

1. Python environment

This project uses uv (see pyproject.toml / .python-version):

uv sync

Or with plain pip:

pip install numpy pandas scipy scikit-learn matplotlib pyserial

2. Run the pipeline (in order)

python pipeline/01_filter_extract.py --csv data/data_100hz.csv  # → pipeline/out/features.csv, filter_sos.json
python pipeline/02_train.py                                     # → model.pkl, plant_inference/model_params.h
python pipeline/03_visualize.py                                 # → pipeline/out/fig1..3.png

python main.py prints this sequence as a reminder.

3. Flash the firmware

Open plant_inference/plant_inference.ino in the Arduino IDE (Arduino-ESP32 core ≥ 2.0) with model_params.h alongside it.
Install the Adafruit PWM Servo Driver Library (pulls in Adafruit BusIO). The 5 servos are driven through a PCA9685 over I2C.
Wire the PCA9685: SDA/SCL → PIN_I2C_SDA/PIN_I2C_SCL, V+ → a dedicated 5–6 V servo supply (not the 3.3 V rail), GND common with the ESP32.
Set PIN_PLANT, the I2C pins, SERVO_CH, and the SERVO_US_MIN/MAX pulse range to match your wiring/servos, then upload.

To just capture data, flash POC_electrode_reader/ instead and run python data_collection.py --port <your-port>.

How it works

9 features per 1 s window (pipeline/01_filter_extract.py): mean, std, ptp, slope, zcr, spike_count, hjorth_mobility, hjorth_complexity, rms_first_diff. They are intentionally FFT-free so the exact same math runs in Python (training) and C++ on the ESP32 (inference) — keep extract_window() and extract_features() in sync.

Servo mapping (pipeline/02_train.py → firmware): joints 0–2 follow the 3 PCA axes (slow, smooth "posture"), while joints 3–4 follow spike_count and hjorth_complexity directly (snappy, expressive transients).

Note on windowing: the training window hop is set in pipeline/01_filter_extract.py (HOP = 50), while the firmware's HOP_SIZE is the on-device inference cadence and is configured independently in the generated header.

See ref_plantsignal.md for the design rationale behind the feature choices, window size, and PCA-vs-autoencoder decision.

Firmware conventions

C++ for the ESP32 follows the guidelines in AGENTS.md — C++23, float-only math (no hardware double), no heap allocation on hot paths, and fixed compile-time buffer sizes.

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

CIn code
CSSIn code
FlaskIn code
HTMLIn code
Next.jsIn code
PythonIn code
ReactIn code
Tailwind CSSIn code
TypeScriptIn code
C++Claimed

9 of 10 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

Claude CodeConfig
CodexConfig

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

268 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.