# Project export: Gravel This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: Provably private model inference; Protect your model queries behind fully homomorphic encryption. - Devpost: https://devpost.com/software/gravel-2ltjkd - GitHub: https://github.com/outercloudstudio/private-inference - Video: https://www.youtube.com/embed/z8BJq88PWsI?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 1 GitHub contributor(s) — Outer Cloud (25 commits) ## Devpost submission (written by the team) ### Inspiration 100% private computation is possible. When I discovered this 4 days ago, I knew I had to try and implement cyptographically private AI model inference. Everyone care's about data. People don't want to be spied on. Companies have proprietary information. The government has defense secrets. When everyone outsources their model inference to third parties, their data is in danger without extra precautions. Even if a company means well, this does not guarantee they aren't compromised. With Gravel's cryptographic protections, we can do model inference without leaking our data. ### What it does Gravel is a distributed system for parallel execution of homographic encryption secured programs. What the hell does that even mean? Image I have a super secret image. My life depends on the contents of this image and I need to be sure that this image does not get leaked. If I wanted to run a vision model on this image, I'd have to upload that image to a remote server. Consider my image compromised already. With Gravel, my image stays encrypted. From start to finish. The server running the AI model inference on my image can't even see my image. Yes, it's truly magical. Not only does Gravel support this private model inference, it accelerates it by providing the protocol for many machines to work together to parallelize the work, resulting in magnitudes of faster inference. How I built it Gravel is a conglomeration of MANY different technologies working together to make this possible. First, the example model is a bitwise neural network trained using pytorch and python. The parameters of this model are extracted and then executed in rust using inference I built on top of the "tfhe" homomorphic encryption library. These executions are coordinated via Deno scripts connected to Render web services as well Google's compute engine. Graphite is the combination of everything, the networking between servers, the secure inference, and the models trained for Gravel. Challenges I ran into Two major hurdles presented themselves over the course of this hackathon. Many libraries I tried to use for the homomorphic encryption were either non functional or too bare bones to support model inference. Only after about 18 hours did I finally manage to get the "tfhe" rust crate to work well enough that this project looked feasible. Not only was finding a library difficult, I've had to learn all about this technology on the fly as quickly as possible. The other major hurdle I faced was training a binary neural network for the first time. Many, many technical difficulties had to be overcome to figure out how to take this model trained in python and execute it in a secure rust runtime. Accomplishments that I'm proud of I am so proud that a real model is running privately on the Gravel platform. Truthfully I doubted and I would even get something close to working and many times I thought about how I would pivot if I couldn't keep pushing forward. Essentially I'm proud that I managed to pull of this ridiculous idea as a team of one. What I learned Of course I learned about homomorphic encryption this week, but I also learned how to orchestrate Google virtual machines. I learned how to deploy a web service on Render. I learned how bitwise nerual network work. It's truly remarkable what can be done in so short of a time. ### What's next One of places where Gravel can be explored is speed. Right now all the model inference is on CPUs! Compared to the speedups we could get on GPUs or even specialized hardware for homomorphic encryption, Gravel has a lot of room to improve. I specifically want to research developing cryptographic accelerators on FPGAs. Homomorphic encryption is an active area of research and it's going to grow larger in the future as running these secured programs becomes more practical. Gravel is on it's way to explore this new paradigm of computing. ## README (from the GitHub repository) # private-inference ## Detected evidence (automated analysis) Indexed codebase: 23 recognized source files, 65 KB. - JavaScript (language) — detected in the code - Python (language) — detected in the code - Rust (language) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (39 of 39) ``` .gitignore binary_model_small.pth binary_model_ultra_small.pth binary_model.json binary_model.pth Cargo.lock Cargo.toml data/MNIST/raw/t10k-images-idx3-ubyte data/MNIST/raw/t10k-labels-idx1-ubyte data/MNIST/raw/train-images-idx3-ubyte data/MNIST/raw/train-labels-idx1-ubyte deno.json index.js LICENSE model.pth package.json quantized_model.pth README.md src/bin/calculate.rs src/bin/decrypt_results.rs src/bin/encrypt_image.rs src/bin/generate_keys.rs src/distributed/demo.ts src/distributed/inference.ts src/distributed/node.ts src/distributed/share-keys.ts src/distributed/utils.ts src/main.rs src/model/binary.py src/model/codegen.py src/model/data.py src/model/extract_weights.py src/model/layers.py src/model/main.py src/model/mnist_binary.py src/model/model.py src/model/quantizer.py src/model/train.py src/model/utils.py ``` ### Dependencies - Cargo.toml: base64@0.22.1, bincode@1.3.3, futures-util@0.3.31, rayon@1.11.0, serde@1.0.228, serde_json@1.0.149, tfhe@*, tokio@1.49.0, tokio-tungstenite@0.28.0, url@2.5.8 - package.json: ws@^8.19.0 ### Recent commits (newest first) - remote - work - remote stuff - decrypting results - working - make model even smaller - Update node.ts - Update node.ts - Update node.ts - remote - some file streaming - remote - update server - more server - server work - work on server - fix spam logging - larger payload - reset node - remove ## Key source files (fetched from GitHub, selected and truncated for size) ### Cargo.toml ``` [package] name = "private-inference" version = "0.1.0" edition = "2024" [dependencies] base64 = "0.22.1" bincode = "1.3.3" futures-util = "0.3.31" rayon = "1.11.0" serde = "1.0.228" serde_json = "1.0.149" tfhe = { version = "*", features = ["boolean", "shortint", "integer"] } tokio = { version = "1.49.0", features = ["full"] } tokio-tungstenite = "0.28.0" url = "2.5.8" [[bin]] name = "private-inference" path = "main.rs" ``` ### package.json ``` { "name": "private-inference", "version": "1.0.0", "description": "", "main": "index.js", "scripts": {}, "repository": { "type": "git", "url": "git+https://github.com/outercloudstudio/private-inference.git" }, "author": "Outer Cloud", "license": "MIT", "bugs": { "url": "https://github.com/outercloudstudio/private-inference/issues" }, "homepage": "https://github.com/outercloudstudio/private-inference#readme", "dependencies": { "ws": "^8.19.0" } } ``` ### index.js ```javascript const WebSocket = require('ws'); const PORT = process.env.PORT || 8080; const wss = new WebSocket.Server({ port: PORT }); console.log('WebSocket server is running on ws://localhost:8080'); let layer = 0 let node = 0 let nodesLeft = 0 let inferenceSocket = undefined wss.on('connection', (ws) => { console.log('New client connected'); ws.on('message', async (message) => { const data = JSON.parse(message) console.log(`Received: ${data.id}`); if(data.id === 'inference') { inferenceSocket = ws layer = 0 node = 0 nodesLeft = 32 for(const client of wss.clients) { if(client === ws) continue if (client.readyState !== WebSocket.OPEN) continue client.send(JSON.stringify({ id: 'calculate', location: { layer, node } })); node++ } } else if(data.id === 'calculate-finished') { nodesLeft-- console.log(nodesLeft, layer, node) if(layer === 0 && node < 32) { ws.send(JSON.stringify({ id: 'calculate', location: { layer, node } })) node++ } else if(layer === 1 && node < 32) { ws.send(JSON.stringify({ id: 'calculate', location: { layer, node } })) node++ } else if(layer === 2 && node < 10) { ws.send(JSON.stringify({ id: 'calculate', location: { layer, node } })) node++ } if(nodesLeft === 0 && layer === 0) { node = 0 layer = 1 nodesLeft = 32 console.log('Beginning layer 1!') for(const client of wss.clients) { if(client === inferenceSocket) continue if (client.readyState !== WebSocket.OPEN) continue client.send(JSON.stringify({ id: 'calculate', location: { layer, node } })); node++ } } else if(nodesLeft === 0 && layer === 1) { node = 0 layer = 2 nodesLeft = 10 console.log('Beginning layer 2!') for(const client of wss.clients) { if(client === inferenceSocket) continue if (client.readyState !== WebSocket.OPEN) continue client.send(JSON.stringify({ id: 'calculate', location: { layer, node } })); node++ } } else if(nodesLeft === 0 && layer === 2) { for(const client of wss.clients) { if (client.readyState !== WebSocket.OPEN) continue client.send(JSON.stringify({ id: 'inference-complete', })); } } } else if(data.id === 'server-key') { for(const client of wss.clients) { if(client === ws) continue if (client.readyState !== WebSocket.OPEN) continue client.send(JSON.stringify(data)); } } else if(data.id === 'encrypted-zero') { for(const client of wss.clients) { if(client === ws) continue if (client.readyState !== WebSocket.OPEN) continue client.send(JSON.stringify(data)); } } else if(data.id === 'encrypted-inputs') { for(const client of wss.clients) { if(client === ws) continue if (client.readyState !== WebSocket.OPEN) continue client.send(JSON.stringify(data)); } } else if(data.id === 'calculate-result') { for(const client of wss.clients) { if(client === ws) continue if (client.readyState !== WebSocket.OPEN) continue client.send(JSON.stringify(data)); } } }); ws.on('close', () => { console.log('Client disconnected'); }); ws.on('error', (error) => { console.error('WebSocket error:', error); }); }); wss.on('error', (error) => { console.error('Server error:', error); }); ``` ### src/main.rs ```rust use tfhe::boolean::prelude::{BinaryBooleanGates, ServerKey}; use tfhe::prelude::*; use tfhe::{ConfigBuilder, FheBool, FheInt8, FheInt16, generate_keys, set_server_key}; use serde::{Deserialize, Serialize}; use serde_json::{self, from_str}; #[derive(Debug, Deserialize, Serialize)] struct BinaryLayer { weight: Vec>, } #[derive(Debug, Deserialize, Serialize)] struct LinearLayer { weight: Vec>, bias: Vec, } #[derive(Debug, Deserialize, Serialize)] struct Model { fc1: BinaryLayer, fc2: BinaryLayer, // fc3: BinaryLayer, // fc4: LinearLayer, fc3: LinearLayer, } fn binary_node(inputs: &Vec, weights: &Vec) -> FheInt16 { let mut sum = &inputs[0] * weights[0]; for i in 1..inputs.len() { sum = sum + &inputs[i] * weights[i]; } return sum; } fn binary_node_clear(inputs: &Vec, weights: &Vec) -> i16 { let mut sum = inputs[0] * weights[0]; for i in 1..inputs.len() { sum = sum + inputs[i] * weights[i]; } return sum; } fn relu(value: FheInt16, encrypted_zero: &FheInt16) -> FheInt16 { let comparison = value.ge(encrypted_zero); return comparison.select(&value, encrypted_zero); } const JSON_STR: &str = include_str!("binary_model.json"); fn run_clear_model() { let model: Model = from_str(JSON_STR).expect("Failed to parse JSON"); let clear_inputs: Vec = vec![ -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, ]; println!("{:?}", clear_inputs); let mut clear_layer_0: Vec = Vec::new(); for i in 0..64 { let mut weights: Vec = Vec::new(); for j in 0..49 { weights.push(model.fc1.weight[i][j] as i16); } clear_layer_0.push(i16::max(0, binary_node_clear(&clear_inputs, &weights))); } println!("{:?}", clear_layer_0); let mut clear_layer_1: Vec = Vec::new(); for i in 0..64 { let mut weights: Vec = Vec::new(); for j in 0..64 { weights.push(model.fc2.weight[i][j] as i16); } clear_layer_1.push(i16::max(0, binary_node_clear(&clear_layer_0, &weights))); } println!("{:?}", clear_layer_1); let mut clear_layer_2: Vec = Vec::new(); for i in 0..10 { let mut sum: i16 = 0i16; for j in 0..64 { sum += model.fc3.weight[i][j] as i16 * clear_layer_1[j] as i16; } clear_layer_2.push(sum + model.fc3.bias[i]); } println!("{:?}", clear_layer_2); let mut max_index = 0; for i in 1..10 { if clear_layer_2[i] > clear_layer_2[max_index] { max_index = i; } } println!("{}", max_index); } fn main() -> Result<(), Box> { run_clear_model(); let model: Model = from_str(JSON_STR).expect("Failed to parse JSON"); let config = ConfigBuilder::default().build(); let (client_key, server_keys) = generate_keys(config); let encrypted_zero = FheInt16::try_encrypt(0i8, &client_key)?; // On the server side: set_server_key(server_keys); let clear_inputs: Vec = vec![ -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, ]; let mut inputs: Vec = Vec::new(); for i in 0..49 { let encrypted_input = FheInt16::try_encrypt(clear_inputs[i], &client_key)?; inputs.push(encrypted_input); } let mut layer_0: Vec = Vec::new(); for i in 0..64 { let mut weights: Vec = Vec::new(); for j in 0..49 { weights.push(model.fc1.weight[i][j]); } let result = relu(binary_node(&inputs, &weights), &encrypted_zero); let clear_result: i16 = result.decrypt(&client_key); println!("{}", clear_result); layer_0.push(result); } // let clear_result: i8 = result.decrypt(&client_key); // println!("{}", clear_result); Ok(()) } ``` ### src/model/main.py ```python """ Binary Neural Network for MNIST MLP with binarized weights based on https://arxiv.org/abs/1602.02830 """ import torch import torch.nn as nn import torch.optim as optim from data import get_loaders from model import Model, train_epoch, validate def main(): device = torch.device("cuda" if torch.cuda.is_available() else "cpu") print(f"Using device: {device}") train_loader, val_loader = get_loaders(batch_size=256) model = Model().to(device) criterion = nn.CrossEntropyLoss() optimizer = optim.Adam(model.parameters(), lr=0.001) epochs = 5 print("\nStarting training...") for epoch in range(1, epochs + 1): train_loss = train_epoch(model, train_loader, criterion, optimizer, device, epoch) val_loss = validate(model, val_loader, criterion, device, epoch) print(f"Epoch {epoch}/{epochs} - " f"Train Loss: {train_loss:.4f} - " f"Val Loss: {val_loss:.4f}") print("Training complete!") torch.save(model.state_dict(), 'model.pth') if __name__ == "__main__": main() ``` ### src/bin/decrypt_results.rs ```rust use serde_json::{self, from_str}; use std::{env, fs}; use tfhe::prelude::*; use tfhe::safe_serialization::safe_deserialize; use tfhe::{ClientKey, FheInt16}; fn main() -> Result<(), Box> { let client_key: ClientKey = safe_deserialize( fs::read("./keys/client_key.bin").unwrap().as_slice(), 1 << 30, ) .unwrap(); for i in 0..9 { let input_buffer = fs::read(format!("./keys/layer_2_{}.bin", i)).unwrap(); let encrypted_input: FheInt16 = bincode::deserialize_from(&mut input_buffer.as_slice())?; let result: i16 = encrypted_input.decrypt(&client_key); println!("{} {}", i, result); } Ok(()) } ``` ### src/bin/encrypt_image.rs ```rust use serde_json::{self, from_str}; use std::{env, fs}; use tfhe::prelude::*; use tfhe::safe_serialization::safe_deserialize; use tfhe::{ClientKey, FheInt16}; fn main() -> Result<(), Box> { let args: Vec = env::args().collect(); let clear_inputs: Vec = from_str(&args[1]).expect("Failed to parse JSON"); let client_key: ClientKey = safe_deserialize( fs::read("./keys/client_key.bin").unwrap().as_slice(), 1 << 30, ) .unwrap(); println!("{:?}", clear_inputs); let mut serialized_encrypted_inputs = Vec::new(); bincode::serialize_into( &mut serialized_encrypted_inputs, &(clear_inputs.len() as i16), )?; for (_, &value) in clear_inputs.iter().enumerate() { let encrypted = FheInt16::try_encrypt(value, &client_key)?; bincode::serialize_into(&mut serialized_encrypted_inputs, &encrypted)?; } fs::write( format!("./keys/encrypted_inputs.bin"), &serialized_encrypted_inputs, )?; println!("Image encrypted!"); Ok(()) } ``` ### src/bin/generate_keys.rs ```rust use serde_json::{self, from_str}; use std::{env, fs}; use tfhe::prelude::*; use tfhe::safe_serialization::safe_serialize; use tfhe::shortint::prelude::PARAM_MESSAGE_2_CARRY_2_KS_PBS; use tfhe::{ConfigBuilder, FheInt16, generate_keys}; fn main() -> Result<(), Box> { let config = ConfigBuilder::default().build(); let (client_key, server_key) = generate_keys(config); let encrypted_zero = FheInt16::try_encrypt(0i8, &client_key)?; let mut serialized_encrypted_zero = Vec::new(); bincode::serialize_into(&mut serialized_encrypted_zero, &encrypted_zero)?; fs::write( format!("./keys/encrypted_zero.bin"), &serialized_encrypted_zero, )?; let mut server_key_buffer = vec![]; safe_serialize(&server_key, &mut server_key_buffer, 1 << 30).unwrap(); fs::write("./keys/server_key.bin", &server_key_buffer)?; let mut client_key_buffer = vec![]; safe_serialize(&client_key, &mut client_key_buffer, 1 << 30).unwrap(); fs::write("./keys/client_key.bin", &client_key_buffer)?; println!("Keys saved!"); Ok(()) } ``` ### src/model/extract_weights.py ```python import torch import json import numpy as np from layers import binarize def quantize(tensor): return torch.round(torch.mul(tensor, 100)) if __name__ == "__main__": state_dict = torch.load("binary_model_ultra_small.pth") data = {} data["fc1"] = { "weight": binarize(state_dict["fc1.weight"]).cpu().numpy().astype(np.int32).tolist(), } data["fc2"] = { "weight": binarize(state_dict["fc2.weight"]).cpu().numpy().astype(np.int32).tolist(), } # data["fc3"] = { # "weight": binarize(state_dict["fc3.weight"]).cpu().numpy().astype(np.int32).tolist(), # } # data["fc4"] = { # "weight": quantize(state_dict["fc4.weight"]).cpu().numpy().astype(np.int32).tolist(), # "bias": quantize(state_dict["fc4.bias"]).cpu().numpy().astype(np.int32).tolist(), # } data["fc3"] = { "weight": quantize(state_dict["fc3.weight"]).cpu().numpy().astype(np.int32).tolist(), "bias": quantize(state_dict["fc3.bias"]).cpu().numpy().astype(np.int32).tolist(), } with open('binary_model.json', 'w') as f: json.dump(data, f) ``` ### src/distributed/utils.ts ```typescript const CHUNK_SIZE = 64 * 1024; export async function sendChunks(data: Uint8Array, id: string, ws: WebSocket, extraData?: any) { const totalChunks = Math.ceil(data.length / CHUNK_SIZE); for (let i = 0; i < totalChunks; i++) { const start = i * CHUNK_SIZE; const end = Math.min(start + CHUNK_SIZE, data.length); const chunk = data.slice(start, end); // Convert chunk to base64 const base64Chunk = btoa(String.fromCharCode(...new Uint8Array(chunk))); if(extraData) { const message = JSON.stringify({ id, index: i, total: totalChunks, data: base64Chunk, ...extraData }); ws.send(message); } else { const message = JSON.stringify({ id, index: i, total: totalChunks, data: base64Chunk, }); ws.send(message); } await new Promise(resolve => setTimeout(resolve, 5)); } } ``` [14 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]