# Project export: Kanga: Streamlining Research Papers into Presentations

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2024
- Tagline: We are making research more accessible by automatically generating understandable, concise, and aesthetically clean presentation slides from a research paper with just one click.
- Devpost: https://devpost.com/software/kanga-slides-made-easy
- GitHub: https://github.com/theAnshulGupta/kanga
- Video: https://www.youtube.com/embed/7mnYWuqBTDo?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (Canva: Best New Canva App Submission (4x Wireless Headphones + Canva Sweatshirts))
- Team: 2 GitHub contributor(s) — Anshul Gupta (7 commits), wesley (2 commits)

## Devpost submission (written by the team)

### Inspiration

There’s two fundamental issues we want to address: a) Research is not generally accessible to the public. Most people don’t want to spend hours trying to decipher jargon and figure out why a paper is important. b) Presenting research and making it accessible is a very tedious task. Authors have to compile their research into a presentation, figure out what key points to highlight, and what to present to make it understandable to those without industry knowledge. Plus – no one wants an ugly presentation, so significant time is spent on design too. Put together, this means that researchers spend a lot of time building presentations (our empirical survey found researchers spent ~5.8 hours on average building presentations) if they want to be able to present them to the general public. Yet, research is incredibly valuable in driving forward innovation, ensuring people understand what is happening in society, and helping inspire and educate students currently in school who will become further scientists and leaders. It's not enough for research to simply live in the bubble of academia -- the wider public (which are all impacted by research findings) need to be engaged, and there needs to be a easier way of doing that.

### What it does

The current ways that people create slides: Slidesgo / SlidesCarnival: Tools like Slidesgo and SlidesCarnival only provide templates, rather than content creation. It is very time consuming to add content and design a presentation. Slidesgo / SlidesCarnival: Tools like Slidesgo and SlidesCarnival only provide templates, rather than content creation. It is very time consuming to add content and design a presentation. Tome.ai / Gamma: These existing AI-powered slide generation tools do not have dynamic content creation and placement. These tools produced slides that look fragmented and do not flow as a full presentation. Tome.ai / Gamma: These existing AI-powered slide generation tools do not have dynamic content creation and placement. These tools produced slides that look fragmented and do not flow as a full presentation. And neither are suitable for research presentations. Our system takes an URL to the PDF of the research paper, and retreives the research paper in order to summarize and compiled a structured presentation of the key background, contributions, and results of a given study. From there, we organize the data and assign it into logically-ordered slides broken down by purpose and dynamically render visual elements and text onto a Canva presentation. All of this is completed in under 60 seconds, where as most people don’t even have a title slide created during that time!

### How we built it

We built this application onto of the Canva SDK in Typescript, using Bun as our runtime to speed up the backend. To process the PDF, we built an API endpoint using Together AI’s hosted Mistral 8x7B model and Python to provide cleaned text. From there, we stitched together multiple layers of LLM agents in order to analyze the research paper, extract the relevant information, convert the relevant information into slides, and organize the slides and content into Canva objects that are properly spaced and positioned. We also utilized Canva’s images library in order to add and implement dynamic styling of elements based on a user selected theme.

### Challenges we ran into

Learning to use the Canva SDK was quite challenging, especially since it was recently released so the documentation describing various functionality and nuances were not as extensive. It was also very challenging and time consuming finding the most optimal LLM pipeline setup that ensured that any content we had was: Relevant, and concise Accurate and minimzed hallucinations. Finally, figuring out how to best position the content on a slide in a way that can be generalized to all research papers was a major challenge. We attempted to build out a system to generate more advanced / complex elements and layouts on the fly using LLMs, but were not able to complete this in time.

### Accomplishments we're proud of

We’re very proud of the pipeline we built — it's quite robust, and we’re able to parse essentially any research paper from any journal across any domain. This is no small feat, considering that there are libraries built specifically to parse certain journal articles from specific publications. We’re also quite proud of how coherently we were able to get content – it’s output in a very understandable manner that is simple and accurate. Finally, we’re very proud of the interface we built. A big part of our focus was around simplicity – unlike traditional design tools like Figma or Adobe XD, we aren’t catering our tool towards designers, but rather than researchers who frequently have minimal if any design experience. Therefore, we focused on minimize the complexity and failure cases that researcher would potentially face in the process, to make our tool as easy to use as possible. We're excited to be working on a big problem that has to potential to change the way research is being communicated if our solution works -- the "no-code" design market is ~60B USD (Canva's TAM is ~40B). Universities alone spent ~20B USD on software tools. We estimate that the demand for good design tools at research institutions is at least ~3B USD -- given that there's no other tool on the market doing this, we'd be positioned for significant market capture, and that's before we expand beyond just research.

### What we learned

We learned a lot about working with Canva’s SDK, which we found quite impressive. We believe there’s a lot of potential for AI to be integrated more tightly into the Canva ecosystem through it's SDK to supercharge how people design presentations and event posters. Furthermore, given that accuracy is very important in research contexts, we learned about techniques to minimize LLM hallucination — reducing the temperature parameter of models, and breaking tasks into smaller subtasks that can be processed iteratively. Finally, through the process, we learned how important it was to effectively communicate and delegate tasks – at first, we had multiple team members working on similar sub-projects because while we were all clear on the components we needed to build to make this project work, there was a bit of miscommunication as to the best way to divide up the necessary components to develop, given a lot of components have significant interdependency.

### What's next

for Kanga We plan on applying for the Canva Innovation Fund to further develop this project. We’d also like to refine the various stages of this process. While in 36 hours we managed to build out an arguably-impressive prototype, there’s a lot of room for improvement, especially in terms of designing more presentation-ready papers. Some features we’d lile to implement include: Extracting relevant diagrams and images from papers and positioning them within slides Generating icons and diagrams that present various contributions and concepts discussed within the paper using Dalle / SDXL Building more variety in the positioning of elements and layouts within a slide show Providing for fine grained control over the final result of the presentation Expand beyond just presentations into posters and other graphics Given ample time, we’d also like to fine-tune our own LLM models that are catered towards distilling research information, to ensure that we are capturing the most optimal contributions and key points with maximum accuracy. We want to become the default app for Canva users and become part of their daily workflow, and ideally be integrated fully into Canva as a “design-assist” agent and get acquired by Canva in 18 months. We’d use a variety of per-user pricing plans to generate revenue, primarily catering towards researchers and educational institutions, where we’d be able to close high-value contracts and push our product to a large amount of relevant users (stuents, professors, researchers) very quickiy.

## README (from the GitHub repository)

# Kanga: Research to Presentation
Developed by Welton Wang, Anshul Gupta, Wesley Dela Cruz, and Arun Moorthy
<br>
Stanford TreeHacks Sponsor Winner 2024 [\[devpost\]](https://devpost.com/software/kanga-slides-made-easy)


## Detected evidence (automated analysis)

Indexed codebase: 44 recognized source files, 150 KB.
- CSS (language) — detected in the code
- Express (technology) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- React (technology) — detected in the code
- TypeScript (language) — detected in the code
- Node.js (technology) — claimed on Devpost, not found in the code
- Python (language) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (49 of 49)

```
.gitignore
components/audio_player/audio_context.tsx
components/audio_player/audio_cover.css
components/audio_player/audio_cover.tsx
components/audio_player/index.ts
components/draggable_audio/draggable_audio.css
components/draggable_audio/draggable_audio.tsx
components/draggable_audio/index.ts
components/draggable_embed/draggable_embed.css
components/draggable_embed/draggable_embed.tsx
components/draggable_embed/index.ts
components/draggable_image/draggable_image.css
components/draggable_image/draggable_image.tsx
components/draggable_image/index.ts
components/draggable_text/draggable_text.css
components/draggable_text/draggable_text.tsx
components/draggable_text/index.ts
components/draggable_video/draggable_video.css
components/draggable_video/draggable_video.tsx
components/draggable_video/index.ts
declarations/declarations.d.ts
jest.config.js
package.json
public/text.txt
README.md
scripts/ssl.js
scripts/start.js
sdk/preview/asset/index.d.ts
sdk/preview/asset/index.js
sdk/preview/data/index.d.ts
sdk/preview/data/index.js
sdk/preview/design/index.d.ts
sdk/preview/design/index.js
sdk/preview/LICENSE.md
sdk/preview/package.json
src/app.tsx
src/backend/server.ts
src/designer.tsx
src/extractText.tsx
src/index.tsx
src/llm.tsx
styles/components.css
tsconfig.json
utils/backend/base_backend/create.ts
utils/backend/jwt_middleware/index.ts
utils/backend/jwt_middleware/jwt_middleware.ts
utils/backend/jwt_middleware/tests/jwt_middleware.tests.ts
utils/use_selection_hook.ts
webpack.config.js
```

### Dependencies

- package.json: @canva/app-ui-kit@^3.1.0, @canva/asset@^1.2.0, @canva/design@^1.4.0, @canva/error@^1.1.0, @canva/platform@^1.0.1, @canva/preview@./sdk/preview, @canva/user@^1.0.0, @svgr/webpack@^8.0.1, @types/cookie-parser@^1.4.4, @types/cors@^2.8.13, @types/debug@^4.1.7, @types/express@^4.17.21, @types/jest@^29.4.0, @types/jsonwebtoken@^9.0.1, @types/node@^20.11.19, @types/node-fetch@^2.6.2, @types/nodemon@^1.19.2, @types/react-dom@^18.1.0, @types/webpack-env@^1.18.0, axios@^1.6.7, bun@^1.0.27, chalk@^4.1.2, cli-table3@^0.6.2, clsx@^1.2.1, cookie-parser@^1.4.6, cors@^2.8.5, css-loader@^6.7.1, css-modules-typescript-loader@^4.0.1, cssnano@^6.0.1, debug@^4.3.4, dotenv@^16.4.4, exponential-backoff@^3.1.0, express@^4.18.2, express-basic-auth@^1.2.1, jest@^29.4.2, jsonwebtoken@^9.0.0, jwks-rsa@^3.0.1, mini-css-extract-plugin@^2.6.1, ngrok@^4.3.3, node-fetch@^2.6.7, node-forge@^1.3.1, nodemon@^3.0.1, openai@^4.28.0, postcss-loader@^7.3.3, prettier@^2.7.1, react@^18.1.0, react-dom@^18.1.0, style-loader@^3.3.1, terser-webpack-plugin@^5.3.5, ts-jest@^29.0.5, ts-loader@^9.3.1, ts-node@^10.9.1, typescript@^5.2.2, url-loader@^4.1.1, webpack@^5.74.0, webpack-cli@^4.10.0, webpack-dev-server@^4.10.0, yargs@^17.5.1
- sdk/preview/package.json: @canva/error@^1.0.0

### Recent commits (newest first)

- Update README.md
- Update README.md
- Update README.md
- production
- page changes
- Merge branch 'main' of https://github.com/theAnshulGupta/canva
- app data
- add hot reload
- Add .gitignore with node_modules
- init
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### sdk/preview/LICENSE.md

```markdown
# A message from Canva

By making these software components available to you, our goal is to give you
the right to use the components to build your apps for Canva's ecosystem. See
below for the specific licence terms that apply to your use.

# Licence

Canva Pty Ltd (ACN 158 929 938) ("Canva") owns the copyright in the software and
associated documentation files to which this Licence relates (each a "Software
Component"). Canva reserves all of its rights.

Permission is hereby granted, free of charge, to any person obtaining a copy of
the Software Components ("You"), to use the Software Components, but strictly
subject to the following restrictions and conditions:

1. You must only use the Software Components and any Derivative on the Canva
   Platform.

2. You must only use the Software Components and any Derivative for the purpose
   of building or updating Permitted Apps, and for no other purpose.

3. You must include a copy of this Licence in all copies of any Software
   Component and Derivative.

4. To the fullest extent permitted by law, the Software Components are provided
   "as is", without warranty of any kind, express or implied, including but not
   limited to the warranties of merchantability, fitness for a particular
   purpose and non-infringement. To the fullest extent permitted by law, in no
   event shall the authors or copyright holders be liable for any claim,
   damages or other liability, whether in an action of contract, tort or
   otherwise, arising from, out of or in connection with the Software Components
   or any Derivative, or the use of or other dealings in the Software Components
   or any Derivative.

5. In this Licence:

   (a) "Canva Platform" means the visual communications platform owned and
   operated by Canva that is made available on Canva.com (as well as any
   sub-domains and international versions) and via mobile applications, desktop
   applications, and in other forms provided or made available by Canva;

   (b) "Derivative" means any software, document or other material that: (i)
   contains any Software Component; and/or (ii) comprises or contains a
   derivative, adaptation or substantial part of any Software Component; and

   (c) "Permitted App" means a software application that is solely offered to
   end-users on the Canva Platform.

```

### package.json

```
{
  "name": "canva-apps-sdk-starter-kit",
  "description": "A boilerplate for creating Canva Apps.",
  "engines": {
    "node": "^18 || ^20.10.0",
    "npm": "^9 || ^10"
  },
  "scripts": {
    "start": "node ./scripts/start.js",
    "build": "webpack --config webpack.config.js --mode production && tsc",
    "lint:types": "tsc",
    "format": "prettier '{components,examples,src,utils}/**/*.{css,ts,tsx}' --no-config --write",
    "format:check": "prettier '{components,examples,src,utils}/**/*.{css,ts,tsx}' --no-config --check --ignore-path",
    "format:file": "prettier $1 --no-config --write",
    "test": "jest --no-cache",
    "test:watch": "jest --no-cache --watch",
    "tsc-build": "tsc",
    "node-start": "node dist/server.js"
  },
  "keywords": [],
  "author": "Canva Pty Ltd.",
  "license": "SEE LICENSE IN LICENSE.md",
  "private": true,
  "dependencies": {
    "@canva/app-ui-kit": "^3.1.0",
    "@canva/asset": "^1.2.0",
    "@canva/design": "^1.4.0",
    "@canva/error": "^1.1.0",
    "@canva/platform": "^1.0.1",
    "@canva/preview": "./sdk/preview",
    "@canva/user": "^1.0.0",
    "axios": "^1.6.7",
    "bun": "^1.0.27",
    "clsx": "^1.2.1",
    "cookie-parser": "^1.4.6",
    "openai": "^4.28.0",
    "react": "^18.1.0",
    "react-dom": "^18.1.0"
  },
  "devDependencies": {
    "@svgr/webpack": "^8.0.1",
    "@types/cookie-parser": "^1.4.4",
    "@types/cors": "^2.8.13",
    "@types/debug": "^4.1.7",
    "@types/express": "^4.17.21",
    "@types/jest": "^29.4.0",
    "@types/jsonwebtoken": "^9.0.1",
    "@types/node": "^20.11.19",
    "@types/node-fetch": "^2.6.2",
    "@types/nodemon": "^1.19.2",
    "@types/react-dom": "^18.1.0",
    "@types/webpack-env": "^1.18.0",
    "chalk": "^4.1.2",
    "cli-table3": "^0.6.2",
    "cors": "^2.8.5",
    "css-loader": "^6.7.1",
    "css-modules-typescript-loader": "^4.0.1",
    "cssnano": "^6.0.1",
    "debug": "^4.3.4",
    "dotenv": "^16.4.4",
    "exponential-backoff": "^3.1.0",
    "express": "^4.18.2",
    "express-basic-auth": "^1.2.1",
    "jest": "^29.4.2",
    "jsonwebtoken": "^9.0.0",
    "jwks-rsa": "^3.0.1",
    "mini-css-extract-plugin": "^2.6.1",
    "ngrok": "^4.3.3",
    "node-fetch": "^2.6.7",
    "node-forge": "^1.3.1",
    "nodemon": "^3.0.1",
    "postcss-loader": "^7.3.3",
    "prettier": "^2.7.1",
    "style-loader": "^3.3.1",
    "terser-webpack-plugin": "^5.3.5",
    "ts-jest": "^29.0.5",
    "ts-loader": "^9.3.1",
    "ts-node": "^10.9.1",
    "typescript": "^5.2.2",
    "url-loader": "^4.1.1",
    "webpack": "^5.74.0",
    "webpack-cli": "^4.10.0",
    "webpack-dev-server": "^4.10.0",
    "yargs": "^17.5.1"
  }
}

```

### sdk/preview/package.json

```
{
  "name": "@canva/preview",
  "version": "0.9.0",
  "description": "The Canva Apps SDK preview libraries",
  "author": "Canva Pty Ltd.",
  "license": "SEE LICENSE IN LICENSE.md",
  "private": true,
  "peerDependencies": {
    "@canva/error": "^1.0.0"
  }
}

```

### src/index.tsx

```typescript
import { AppUiProvider } from "@canva/app-ui-kit";
import * as React from "react";
import { createRoot } from "react-dom/client";
import { App } from "./app";
import "@canva/app-ui-kit/styles.css";

const root = createRoot(document.getElementById("root")!);
function render() {
  root.render(
    <AppUiProvider>
      <App />
    </AppUiProvider>
  );
}

render();

if (module.hot) {
  module.hot.accept("./app", render);
}

```

### src/app.tsx

```typescript
import {
  Button,
  Rows,
  Text,
  ImageCard,
  Columns,
  Link,
  Column,
  Title,
  Swatch,
  TextInput,
  LoadingIndicator,
} from "@canva/app-ui-kit";
import { requestOpenExternalUrl } from "@canva/platform";
import type { NativeEmbedElement } from "@canva/design";
import { auth } from "@canva/user";
import React, { useState, useEffect } from "react";
import styles from "styles/components.css";
import {
  addPage,
  getDefaultPageDimensions,
  addNativeElement,
  Dimensions,
  NativeGroupElement,
} from "@canva/design";
import { extractText } from "./extractText";
import { upload } from "@canva/asset";

const BACKEND_URL = `${BACKEND_HOST}/custom-route`;
// const BACKEND_HOST_OPENAI = `${BACKEND_HOST}/openai`; // Change 3000 to your server's port
// http://localhost:3001/openai
const BACKEND_HOST_OPENAI = `http://localhost:3001/openai`;

type State = "idle" | "loading" | "success" | "error";

const links = [
  {
    url: "https://anshulgupta.com/imgs/1.jpg",
  },
  {
    url: "https://anshulgupta.com/imgs/4.jpg",
  },
  {
    url: "https://anshulgupta.com/imgs/8.jpg",
  },
  {
    url: "https://anshulgupta.com/imgs/9.jpg",
  },
  {
    url: "https://anshulgupta.com/imgs/10.jpg",
  },
];

const colors = [
  { name: "green", hex: "#00bf63", urlIndex: 0 },
  { name: "blue", hex: "#5ce1e6", urlIndex: 1 },
  { name: "red", hex: "#ff3131", urlIndex: 2 },
  { name: "pink", hex: "#cb6ce6", urlIndex: 3 },
  { name: "black", hex: "#a6a6a6", urlIndex: 4 },
];

const defaultPageDimensions = await getDefaultPageDimensions();
if (!defaultPageDimensions) {
  throw Error("Failed to get default page dimensions");
}

let canvasWidth = defaultPageDimensions.width;
let canvasHeight = defaultPageDimensions.height;

let edgeCounter = 0;

async function githubLink() {
  const response = await requestOpenExternalUrl({
    url: "https://github.com",
  });
}

async function handleClick(colorName: string) {
  const color = colors.find((color) => color.name === colorName);
  if (!color) {
    console.error(`No color found for name "${colorName}"`);
    return;
  }

  const url = links[color.urlIndex].url;

  const result = await upload({
    type: "IMAGE",
    id: "decoration",
    mimeType: "image/jpeg",
    url: url,
    thumbnailUrl: url,
  });

  let scaledImg = Math.random() * (200 - 100) + 50;

  let rotation = Math.random() * (180 - -180) + -180;

  let edge = edgeCounter % 4;
  let left, top;

  switch (edge) {
    case 0:
      left = Math.random() * (canvasWidth - scaledImg);
      top = -scaledImg / 2;
      break;
    case 1:
      left = Math.random() * (canvasWidth - scaledImg);
      top = canvasHeight - scaledImg / 2;
      break;
    case 2:
      left = -scaledImg / 2;
      top = Math.random() * (canvasHeight - scaledImg);
      break;
    case 3:
      left = canvasWidth - scaledImg / 2;
      top = Math.random() * (canvasHeight - scaledImg);
      break;
  }

  await addNativeElement({
    type: "IMAGE",
    ref: result.ref,
    left: left,
    top: top,
    width: scaledImg,
    height: scaledImg,
    rotation: rotation,
  });

  edgeCounter++;
}
export function parseJsonFromString(text) {
  const startIndex = text.indexOf("{");
  const endIndex = text.lastIndexOf("}");

  if (startIndex === -1 || endIndex === -1 || endIndex < startIndex) {
    throw new Error("Invalid string format for JSON parsing.");
  }
  const jsonString = text
    .substring(startIndex, endIndex + 1)
    .replace(/(\r\n|\n|\r)/gm, "");
  console.log(jsonString);

  try {
    return JSON.parse(jsonString);
  } catch (error) {
    console.error("Failed to parse JSON:", error);
    throw new Error("Failed to parse JSON.");
  }

  // const jsonString = text.substring(startIndex, endIndex + 1).replace(/(\r\n|\n|\r)/gm,"");
  // console.log(jsonString);
  // try {
  //   return JSON.parse(jsonString);
  // } catch (error) {
  //   console.error('Failed to parse JSON:', error);
  //   throw new Error('Failed to parse JSON.');
  // }
}

export const App = () => {
  const [activeColor, setActiveColor] = React.useState<string>(colors[0].name);
  const [link, setLink] = useState("");
  const [responseContent, setResponseContent] = useState("");

  const handleSubmit = async (event) => {
    event.target.parentNode.style = "background: gray;";
    event.target.style = "background: gray;";
    event.target.disabled = true;
    event.preventDefault();
    try {
      const response = await extractText(link);

      if (!response.ok) {
        throw new Error(`HTTP error! status: ${response.status}`);
      }

      const text = await response.text();

      try {
        const token = await auth.getCanvaUserToken();
        const res = await fetch(BACKEND_HOST_OPENAI, {
          method: "POST",
          body: JSON.stringify({
            prompt:
              `Based on a the below research paper, generate a 10 slide presentation in the following structure:\n\nIntroduction and background on topic\nKey contributions / novelties\nResults / datapoints\nImpact / contributions / caveats\n\nUse 2-3 slides per part, and present things as slideshows aimed for a presentation for the general public\n\nUse 2-3 comprehensive, clear, but insightful and value-adding full sentence bullet points per slide max, and generate a sub heading for each slide. Do not make up data and do not make up things that you do not know. Emphasize specific results and data points\n\nRefer to the document, results, and actions. Return it in json format in the following json format: {"title":"<title>", "authors":"<authors", "slides": [{"heading":"<heading>", "body":"<body>"}...]}. Here is the research paper: ` +
              text,
          }),
          headers: new Headers({
            "content-type": "application/json",
            Authorization: `Bearer ${token}`,
          }),
        });

        // const body = await res.json();
        const body = await res.json();
        // const processedResponse = processResponseBody(body);

        // The content i
[truncated — 7741 more characters]
```

### components/draggable_text/index.ts

```typescript
export { DraggableText } from "./draggable_text";

```

### components/draggable_audio/index.ts

```typescript
export { DraggableAudio } from "./draggable_audio";

```

### components/draggable_embed/index.ts

```typescript
export { DraggableEmbed } from "./draggable_embed";

```

### components/draggable_image/index.ts

```typescript
export { DraggableImage } from "./draggable_image";

```

### components/draggable_video/index.ts

```typescript
export { DraggableVideo } from "./draggable_video";

```

[35 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]