# Project export: Lattice: Orchestrating end-to-end web workflows at scale

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Massively parallelized agentic research workflows using spreadsheets as a primitive
- Devpost: https://devpost.com/software/lattice-data-at-your-fingertips
- GitHub: https://github.com/WeltonWang02/spreadsheet_hackathon
- Video: https://www.youtube.com/embed/spfLzOUNp44?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (Pear VC: Best Customer Insights ($25k Unncapped SAFE + Office Hours w/ Partners [1st] & Office Hours w/ Partners [2nd]); Paradigm: Best Spreadsheet-Adjacent Hack (Air Pods Max per team member))
- Team: 1 GitHub contributor(s) — weltonwang02 (54 commits)

## Devpost submission (written by the team)

### Overview

Summary Over the past few months, there've been a lot of really cool agentic research products that've been released (Exa, OpenAI/Perplexity Deep Research, etc). In general, these are point solutions and relatively sequential in manner (ie, the next action is conditioned on the previous actions). But AI much more exciting if we can: Build products around end-to-end workflows Do this at scale that humans never could (ie, browse hundreds of websites at once) We want to build something that takes advantage of the parallelized nature of research agents to solve end-to-end workflows en masse (we can run hundreds of agents at once!). What is this? Lattice allows you to orchestrate complex end-to-end research workflows at scale, with complex/structured dependencies. Consider a sales representative sending cold outbound: This involves: Finding potential companies to sell to and qualifying them Finding potential points of contact at these companies Finding the right form of outreach for the best point of contact Draft + sending the email Today, Lattice solves the vast majority of this workflow – and we do this at scale, across hundreds of companies in parallel. We first find potential companies, and then for each company, find a set of defined attributes. https://imgur.com/a/bsq1S3i Then, for each company, we can find the key people we want to reach and augment this information. https://imgur.com/a/7b0FhLY Then, for each companies' list of people, we can qualify them based on attributes we defined + have researched. https://imgur.com/a/dl6gfzL Finally, we can pipe all the research information to a language model to produce a final work product. https://imgur.com/a/fAdaElA We do this at scale and save a considerable amount of time over manual research – but also existing tools that only do this for a single part of the workflow (ie, Clay)! As a page one end-to-end workflow. We let you build any workflow like this and chain spreadsheets together. More Use Cases Anything that involves scaled web research! But concretely: Insurance companies Start with a provided list of companies and find company-level information. Then for each company, find all their products, and for each product, create a grid of if they flag certain policies. Then aggregate all the violations across the products, run some risk/modeling calculation, and compile into a per-company report. Finance Find a list of recently funded startups (Series A), then for each startup catalogue all of their rounds, how much they raised, and who they raised from at each round. Across the rounds of each company, count up the number of T1 VCs. Draft emails to send outbound to companies with more than 3 T1 VCs on their cap table. Fun Example Find a list of film festivals in the past year. Then for each festival, find which films won the top prizes, the names of the movies, and the authors. Then, choose the film across with the funniest title from each of the festivals. For each of those films, then find other films created by those artists. For each of those films, then collect the Rotten Tomato and IMDB scores. Then, choose the film for each artist with the highest aggregate scores and write a congratulations emails! We know that all three (maybe two) and millions of other similar use cases are very important and significant workflows conducted by humans today. Tech We treat spreadsheets (or really tables) as a core primitive. This is how data is displayed, edited, and passed between different steps in a workflow. Most of the heavy lifting is on the frontend (Next/Typescript/Tailwind) – we build a primitive spreadsheet interface from scratch, added multiple dimension to it, then created a chain of dependencies between the different spreadsheets, and then added a bunch of agent orchestration on top of it! The cool thing is this all actually works, and works quite well! Why Spreadsheets Frankly, we thought Paradigm's challenge was really cool, and were inspired in part by what they do. We think spreadsheets are a fun primitive to build around: It's a surprisingly challenging task. It's a surprisingly challenging task. Spreadsheets are one of the best ways to visualize bulk agentic outputs and enable a lot of cool things. Spreadsheets are one of the best ways to visualize bulk agentic outputs and enable a lot of cool things. We started with what we thought were an interesting limitation of 2D spreadsheets – we can't do a nested series of "Find a list of XYZ, then for each find a list of ABC"-style inputs. There's a lot of powerful workflows though, that depend on being able to serve multiple layers of these kinds of queries – so we designed a "3D spreadsheet" where each sheet is mapped from a row in a 2D spreadsheet. Our spreadsheet workflows, in some ways, are essentially a directed graph with dependencies. This is roughly how we treated the orchestration / execution of individual agents within our system. At any point, we can have a one-to-many relationship between incoming and outgoing edges, or many-to-one relationship. This allows us to manage state/context passing between different steps within a workflow. For example, when going from a single row into a spreadsheet -> a sheet in a 3D spreadsheet -> a single row in a spreadsheet again, we pass parts of the initial row through the entire chain so our agents have the right context. Another cool thing that comes out of this is that we provided verifiable + auditable traces into how exactly the end to end workflow was executed. As any step, you can look up the full table of data that the end considers when making decisions, all the raw attributes collected, and how they flowed in the workflow. What Next? A lot! Here's a few: Making systems and workflows more robust (prototypes are easy; building production grade agentic systems are extremely hard + painful) Make orchestration simpler (we wanted to do this but ran out of time) – describe what you want to complete, and we'll handle the orchestration itself Improve latency - our end to end workflows are slow. We think this is okay, because most of our use cases aren't latency bound. But there is definitely a lot of value in optimizing latency further

## README (from the GitHub repository)

This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app).

## Getting Started

First, run the development server:

```bash
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
```

Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.

You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file.

This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel.

## Learn More

To learn more about Next.js, take a look at the following resources:

- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.

You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome!

## Deploy on Vercel

The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.

Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.


## Detected evidence (automated analysis)

Indexed codebase: 24 recognized source files, 117 KB.
- CSS (language) — detected in the code
- Next.js (technology) — detected in the code
- OpenAI (technology) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code

## Codebase structure (from repository index)

### Files (29 of 29)

```
.gitignore
eslint.config.mjs
next.config.ts
package.json
postcss.config.mjs
README.md
src/app/api/aggregate/route.ts
src/app/api/findall/route.ts
src/app/api/llm/route.ts
src/app/api/orchestrate/route.ts
src/app/api/runCells/route.ts
src/app/globals.css
src/app/layout.tsx
src/app/page.tsx
src/app/workflow/page.tsx
src/components/Chat.tsx
src/components/LLMPipeSpreadsheet.tsx
src/components/SingleSpreadsheet.tsx
src/components/Spreadsheet.tsx
src/components/ThreeDSpreadsheet.tsx
src/components/WorkflowAssistant.tsx
src/components/WorkflowBuilder.tsx
src/components/WorkflowMapper.tsx
src/lib/agent.ts
src/lib/llm.ts
src/lib/looper.ts
src/lib/storage.ts
tailwind.config.ts
tsconfig.json
```

### Dependencies

- package.json: @eslint/eslintrc@^3, @types/node@^20, @types/react@^19, @types/react-dom@^19, eslint@^9, eslint-config-next@15.1.7, next@15.1.7, openai@^4.85.1, postcss@^8, react@^19.0.0, react-dom@^19.0.0, tailwindcss@^3.4.1, typescript@^5

### Recent commits (newest first)

- clean up
- routes
- fix exe state
- change model
- fix indexing
- tweaks
- tweaks to run
- parallel
- paralle tweak
- parallel api
- find all for subcells
- find all v1
- loading
- aggregate workign
- fix
- working
- not working
- tweaks / aggregate
- working p1p2
- llm piping works

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "spreadsheet_hackathon",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev --turbopack",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "next": "15.1.7",
    "openai": "^4.85.1",
    "react": "^19.0.0",
    "react-dom": "^19.0.0"
  },
  "devDependencies": {
    "@eslint/eslintrc": "^3",
    "@types/node": "^20",
    "@types/react": "^19",
    "@types/react-dom": "^19",
    "eslint": "^9",
    "eslint-config-next": "15.1.7",
    "postcss": "^8",
    "tailwindcss": "^3.4.1",
    "typescript": "^5"
  }
}

```

### src/app/page.tsx

```typescript
import dynamic from 'next/dynamic';
import ThreeDSpreadsheet from '@/components/ThreeDSpreadsheet';

export default function Home() {
  return (
    <div className="min-h-screen bg-gray-50">
      <div className="h-[calc(100vh-5rem)]">
        <ThreeDSpreadsheet initialSheets={3} initialRows={15} initialCols={8} />
      </div>
    </div>
  );
}

```

### src/app/layout.tsx

```typescript
import type { Metadata } from "next";
import { Geist, Geist_Mono } from "next/font/google";
import "./globals.css";

const geistSans = Geist({
  variable: "--font-geist-sans",
  subsets: ["latin"],
});

const geistMono = Geist_Mono({
  variable: "--font-geist-mono",
  subsets: ["latin"],
});

export const metadata: Metadata = {
  title: "Create Next App",
  description: "Generated by create next app",
};

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  return (
    <html lang="en">
      <body
        className={`${geistSans.variable} ${geistMono.variable} antialiased`}
      >
        {children}
      </body>
    </html>
  );
}

```

### src/app/workflow/page.tsx

```typescript
'use client';

import React from 'react';
import { WorkflowBuilder } from '@/components/WorkflowBuilder';

export default function WorkflowPage() {
  return (
    <div className="min-h-screen bg-gray-50">
      <WorkflowBuilder />
    </div>
  );
} 
```

### src/app/api/llm/route.ts

```typescript
import { NextResponse } from 'next/server';
import { llm } from '@/lib/llm';
interface LLMRequest {
  inputs: string[];
  prompt: string;
}   

export async function POST(req: Request) {
  try {
    const { inputs, prompt } = await req.json() as LLMRequest;

    // For demonstration, we'll just echo back a modified version of each input
    // In a real implementation, you would call your LLM service here
    const outputs = await Promise.all(inputs.map(async (input) => {
      // Replace any {input} placeholders in the prompt with the actual input
      const processedPrompt = `
        ${prompt}

        Inputs: 
        ${JSON.stringify(input)}

        Follow the instructions in the prompt and return the response using the context as described. You should substitue the variables when you can / with the provided context.
      `
      const llmResponse = await llm(processedPrompt, "gpt-4o");  

      return llmResponse;
    }));

    return NextResponse.json({
      success: true,
      outputs,
      prompt
    });

  } catch (error) {
    console.error('Error in LLM API:', error);
    return NextResponse.json(
      { 
        success: false, 
        error: 'Internal server error',
        outputs: []
      },
      { status: 500 }
    );
  }
} 
```

### src/app/api/findall/route.ts

```typescript
import { NextResponse } from 'next/server';
import { createAndExecuteTask } from '@/lib/agent';

export async function POST(req: Request) {
  try {
    const { query, sheet_level } = await req.json();

    const findAllTask = {
      name: "find_all",
      description: "Find certain entities as requested",
      model: "rhodium",
      prompt: `Find a list of entities that match the query: ${query}`,
      input_schema: {
        type: "object",
        properties: {
          query: {
            type: "string",
            description: "The search query"
          }
        },
        required: ["query"],
        additionalProperties: false
      },
      output_schema: {
        type: "object",
        properties: {
          results: {
            type: "array",
            description: "Array of matching entities",
            items: {
              type: "string"
            }
          }
        },
        required: ["results"],
        additionalProperties: false
      }
    };

    const response = await createAndExecuteTask(findAllTask, { query });
    console.log(response);
    if (!response.output?.results) {
      throw new Error('No results returned from task');
    }

    return NextResponse.json({
      success: true,
      results: response.output.results,
      query: query,
      total: response.output.results.length
    });

  } catch (error) {
    console.error('Error in findAll API:', error);
    return NextResponse.json(
      {
        success: false,
        error: 'Internal server error',
        results: []
      },
      { status: 500 }
    );
  }
}
```

### src/app/api/runCells/route.ts

```typescript
import { NextResponse } from 'next/server';
import { createAndExecuteTask } from '@/lib/agent';

export async function POST(req: Request) {
  try {
    const body = await req.json();
    const input = body.input;
    const columns = body.columns;

    // Execute the task
    const colString = Object.keys(columns).join(', ');
    const taskResponse = await createAndExecuteTask({
      name: "Process Row Data", 
      description: "Process data",
      model: "neon",
      prompt: "For the entity {input}, find the " + colString,
      input_schema: {
        type: "object",
        properties: {
          input: {
            type: "string",
            description: "The input entity"
          }
        },
        required: ["input"],
        additionalProperties: false
      },
      output_schema: {
        type: "object",
        properties: Object.keys(columns).reduce((acc, colName) => {
          acc[colName] = {
            type: "string", 
            description: `${colName}`
          };
          return acc;
        }, {} as Record<string, { type: string; description: string }>),
        required: Object.keys(columns),
        additionalProperties: false
      }
    }, { input });

    if (!taskResponse.output) {
      throw new Error('No output returned from task');
    }
    
    return NextResponse.json({
      success: true,
      results: taskResponse.output,
      input
    });

  } catch (error) {
    console.error('Error in runCells API:', error);
    return NextResponse.json(
      { 
        success: false,
        error: 'Internal server error',
        results: {}
      },
      { status: 500 }
    );
  }
}
```

### src/app/api/orchestrate/route.ts

```typescript
import { NextResponse } from 'next/server';

interface ToolCall {
  type: 'findall_sheets' | 'update_headers';
  params: {
    query?: string;
    headers?: string[];  // For update_headers
  };
}

export async function POST(req: Request) {
  try {
    const { messages } = await req.json();
    const lastMessage = messages[messages.length - 1];
    const toolCalls: ToolCall[] = [];
    
    // Check for header update request
    if (lastMessage.content.toLowerCase().includes('update header') || 
        lastMessage.content.toLowerCase().includes('change header') ||
        lastMessage.content.toLowerCase().includes('update column')) {
      // This is a simple example - in reality, you'd want more sophisticated parsing
      // For demo, we'll just create some sample headers
      toolCalls.push({
        type: 'update_headers',
        params: {
          headers: ['Name', 'Age', 'Email', 'Phone', 'Address']
        }
      });
      
      return NextResponse.json({
        response: "I'll update the column headers for you.",
        toolCalls
      });
    }

    // Existing findall_sheets logic
    if (lastMessage.content.toLowerCase().includes('find sheets') || 
        lastMessage.content.toLowerCase().includes('search sheets')) {
      toolCalls.push({
        type: 'findall_sheets',
        params: {
          query: lastMessage.content
        }
      });
      
      return NextResponse.json({
        response: "I'll help you find those sheets. Let me search for them...",
        toolCalls
      });
    }

    // Default response without tool calls
    return NextResponse.json({
      response: "This is a placeholder response. Implement your orchestration logic here.",
      toolCalls: []
    });

  } catch (error) {
    console.error('Error in orchestrate API:', error);
    return NextResponse.json(
      { error: 'Internal server error' },
      { status: 500 }
    );
  }
} 
```

### src/app/api/aggregate/route.ts

```typescript
import { NextResponse } from 'next/server';
import { llm } from '@/lib/llm';

interface AggregationRequest {
  data: {
    cells: Array<{ value: string; row: number; col: number }>;
    prevRow: Array<{ value: string; row: number; col: number }>;
  };
  columns: string[];
  prevTableHeaders: string[];
  aggregationPrompt: string;
  sheetName: string;
  sheetIndex: number;
}

export async function POST(req: Request) {
  try {
    const body = await req.json() as AggregationRequest;
    
    // Get the previous sheet name from the first cell of prevRow
    const prevSheetName = body.data.prevRow[0]?.value || body.sheetName;
    
    // Construct prompt for LLM
    const prompt = `
      Analyze this data follow the provided instructions to aggregate the data:
      User Prompt: ${body.aggregationPrompt}
      
      The below table's headers are: ${body.prevTableHeaders.join(', ')}

      Data:
      ${JSON.stringify(body.data.cells, null, 2)}
      
      Here are the columns you should return values for: ${body.columns.join(', ')}
      You should return ${body.columns.length} values, one for each column.

      For the first column (${body.columns[0]}), use this exact value: ${prevSheetName}

      For each other column, analyze the data and return an appropriate aggregated value.
      Return the values in a JSON object where each key is the exact column name and the value is the aggregated value.
      Make sure to include all columns from the input list.

      Example format:
      {
        "${body.columns[0]}": "${prevSheetName}",
        "Column2": "aggregated value",
        ...
      }

      Return only the JSON object.
    `;

    const llmResponse = await llm(prompt);
    let parsedData = {};
    
    try {
      parsedData = JSON.parse(llmResponse.text);
      // Ensure all values are strings and all columns are present
      parsedData = body.columns.reduce((acc, column) => {
        acc[column] = (parsedData[column] || '').toString();
        return acc;
      }, {} as Record<string, string>);
    } catch (error) {
      console.error('Error parsing LLM response:', error);
      // If parsing fails, create an object with empty values for all columns
      parsedData = body.columns.reduce((acc, column) => {
        acc[column] = column === body.columns[0] ? prevSheetName : '';
        return acc;
      }, {} as Record<string, string>);
    }

    // Return aggregated results
    return NextResponse.json({
      success: true,
      sheetName: prevSheetName,
      aggregatedInsights: parsedData
    });

  } catch (error) {
    console.error('Error in aggregate API:', error);
    return NextResponse.json(
      {
        success: false,
        error: 'Internal server error',
        sheetName: '',
        aggregatedInsights: {}
      },
      { status: 500 }
    );
  }
}
```

### next.config.ts

```typescript
import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  /* config options here */
};

export default nextConfig;

```

[13 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]