# Project export: Power Lever

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2026
- Tagline: Agentic inference optimization. Unlike static inference providers, we dynamically orchestrate GPUs and implement on-demand speculative decoding to deliver inference control, saving water and compute.
- Devpost: https://devpost.com/software/power-lever
- GitHub: not linked
- Demo: https://prod-lever.vercel.app/
- Video: https://www.youtube.com/embed/p0ruAH5erME?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner ([Greylock] Best multi-turn agent (Courtside tickets to Warriors game ($10k value) Office hours with Greylock Partners))
- Team: contributor stats unavailable

## Devpost submission (written by the team)

### Inspiration

What's the derivative of arccos(x)? I asked Claude Opus 4.6. Too many times, we've overkilled our prompts by feeding simple tasks into state-of-the-art models. The fact that inference providers only allow a small number of model options means that every task is either underserved or overserved. There’s no way to precisely match compute to problem difficulty; you either spin up a sledgehammer or settle for something too weak. We wanted inference to behave less like a fixed menu and more like a tunable dial.

### What it does

We introduce three core technical contributions: 1) On-demand speculative decoding (via vLLM) We implement speculative decoding as a first-class, runtime-configurable primitive using vLLM, dynamically adjusting draft model selection and tokens-ahead per request. Instead of statically enabling acceleration, speculative decoding becomes an adaptive service layer that optimizes acceptance rate, latency, and compute efficiency in real time. 2) Hardware flexibility (via Modal GPU orchestration) We decouple inference from fixed infrastructure by programmatically orchestrating Modal GPU profiles per request. The system can select between hardware configurations based on latency targets, utilization, and energy constraints, transforming GPU selection into a tunable parameter rather than a deployment-time decision. 3) Agentic routing (via Claude Agent SDK) We implement a Router Agent using the Claude Agent SDK that observes hardware state, user-defined performance constraints, and prior run metrics to plan and apply optimal inference configurations. The agent closes the loop between intent (“faster,” “cheaper,” “more efficient”) and execution by autonomously selecting hardware and speculative decoding hyperparameters before each inference call.

### How we built it

Modal, vLLM, FastAPI, Claude Agent SDK, Vercel, Next.js

### Accomplishments we're proud of

Serving models with less water waste.

### What's next

Use live telemetry (TTFT, throughput, early perplexity) to auto-escalate or downshift mid-generation. Improve speculative decoding instrumentation: report real per-token accept/reject rates and tune $k$ and $\tau$ automatically per request.

## README (from the GitHub repository)

No README available.

## Detected evidence (automated analysis)

No repository was indexed for this project. Claimed technologies below could not be checked against code.
- FastAPI (technology) — claimed on Devpost, not found in the code
- Node.js (technology) — claimed on Devpost, not found in the code
- OpenAI (technology) — claimed on Devpost, not found in the code
- Vercel (technology) — claimed on Devpost, not found in the code

## Codebase structure

No repository index available.

## Key source files

No repository index available; no source files included.