# Project export: ScratchML
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: TreeHacks 2024
- Tagline: The no-code platform for ML designed for students in the age of AI.
- Devpost: https://devpost.com/software/scratchml
- GitHub: https://github.com/theDubW/ScratchML
- Result: winner (Intel: Best Use of Intel Developer Cloud ($10k Credits + 4x Lenovo AI PC [1st] & $4k Credits [2nd] & $2k Credits [3rd]); Warp: Best Developer Tool by Warp (4x Keytron Keyboards + Warp Tumblers))
- Team: 4 GitHub contributor(s) — theDubW (54 commits), Yatin Kukreja (50 commits), Dylan (27 commits), Matthew Lee (15 commits)
## Devpost submission (written by the team)
### Overview
TL:DR; It’s hard and important to learn machine learning and data science; we’re making it easier than ever.
### Inspiration
We’re students ranging in various levels of machine learning experience, but we all started the same way; working through infamous online courses and filling out starter code with lines we don’t understand. Although we eventually were able to mature our understanding of machine learning, through ScratchML, we are changing the ML education paradigm from our shared experiences and struggles to creating a platform where kids can develop intuition on data analysis and machine learning that abstracts away the parts that make it difficult. Instead of “learning” by filling out lines of pytorch code that aren’t self evident, we want kids to start thinking about the richness of data around us, what it can be used for, and how to leverage models to infer things about the world around us. We are from the generation that was inspired by Scratch, where drag-and-drop code helped us to make toy games and develop our intuition to code even before we learned our first programming language. ScratchML aims to take a similar approach, creating a learning platform that's easy-to-use and develops our way of thinking and intuition behind the scenes.
### What it does
Our platform is designed to teach machine learning and data analysis principles. The challenge is two-fold: making the learning fun and engaging while also providing a high quality curriculum developed around experiential learning. ScratchML delivers on both aspects by providing a reliable drag-and-drop interface used just like Scratch for model development and data experimentation, and through engineered datasets where the student are guided through exploring the data and reporting findings, students will be much more engaged through trying out a bunch of different approaches to accomplish a mission. Each lesson comes with a workspace, where students can drag-and-drop models and other blocks to create a no-code, data analysis pipeline. A personalized tutor system is designed to guide the learning process, offering explanations and tips to guide the learning process. This system leverages the Prediction Guard LLM API to provide real-time insights into user decisions and outcomes within lessons. It employs state management to ensure continuous progress while fostering a sandbox-style learning environment.
### How we built it
The tech stack for the project consisted of: React for the frontend, utilizing the Chakra UI component library and Tailwind CSS for styling. A Flask server that runs and trains models based on the layout specified by the user. We also used Intels extension for Scikit-learn/PyTorch to deliver faster training and inference time critical for making the user experience on the site seamless. Firebase database for storing user data and storage of models that can be evaluated on the fly. APIs: Prediction Guard LLM API to provide personalized real-time feedback using Neural-Chat-7B. Models: Scikit-learn and PyTorch Dev tools: Intel's Developer Cloud for constructing and testing the sandbox model, leveraging the PyTorch optimizations from Intel.
### Challenges we ran into
The largest challenge was to constantly rework the design to build a more intuitive, highly-functional interface. We are software developers by training; it was difficult to settle on a UI/UX design that achieves all of our priorities. Prior approaches for no-code machine learning are primarily designed for use in industry. Although they work well for older users who need to model data, at each stage we were super focused on whether or not each of the design elements were good for students. Furthermore, with this being our first or second hackathon experience, we worked hard to coordinate distributing work, ideating what could be feasibly accomplished in 36 hours, and ensuring that we accomplish our tasks in a timely manner. A final challenge we faced was simple endurance - the last few hours of development were extremely difficult given the sleep deprivation we all suffered from - overcoming this challenge was simply a matter of willpower and perseverance.
### Accomplishments we're proud of
As amateur hackers, we are proud of what we accomplished this weekend - from building a helpful and innovative product from scratch to just the sheer amount of hard work we exhibited - this weekend proved to us that we are each capable of much more than we originally thought. Our team has limited hackathon experience, and we went in with the approach of not compromising on even our most ambitious ideas. We hacked together the base form of ScratchML, which supports all of the critical features that we set out to do at the beginning of the hackathon, and we are super excited to continue to work at the idea and think creatively and collaboratively on ways we can improve the learning experience for students in the future.
### What we learned
One of the largest takeaways from this weekend was simply that we are capable of much more than we originally believed. Getting together a team of passionate and driven individuals with aligned goals is a powerful tool to create and build. Another lesson we learned in hindsight is the importance of sleep. Sometimes sacrifice can be beneficial, but it’s likely that our excitement in the earlier stages of the weekend came back to bite us during the final stretch. Finally, this challenge asked all of us to wear a wide variety of hats, working with technologies and frameworks that we have limited experiences with. We learned a variety of different tools and also learned how to quickly rise to the occasion and accomplish the needs of the team.
### What's next
We plan on making the UI more intuitive, adding more lessons, increasing the number of blocks available in the sandbox, and increasing the number of datasets users can play with. It is also imperative that we continue to think of creative ways to encourage learning. Our vision for the future of education, shared with many leaders in the space, is turning the classroom into a laboratory, where students can experiment and grow through trial-and-error. This requires coordinated collaboration and a lot of learning on our end as well, and we are eager to innovate and learn from innovators in the educational space to grow ScratchML into the go-to platform for young students trying to learn machine learning and data science principles.
## README (from the GitHub repository)
TL:DR; It’s hard and important to learn machine learning and data science; we’re making it easier than ever.
## Inspiration
We’re students ranging in various levels of machine learning experience, but we all started the same way; working through infamous online courses and filling out starter code with lines we don’t understand. Although we eventually were able to mature our understanding of machine learning, through ScratchML, we are changing the ML education paradigm from our shared experiences and struggles to creating a platform where kids can develop intuition on data analysis and machine learning that abstracts away the parts that make it difficult. Instead of “learning” by filling out lines of pytorch code that aren’t self evident, we want kids to start thinking about the richness of data around us, what it can be used for, and how to leverage models to infer things about the world around us. We are from the generation that was inspired by Scratch, where drag-and-drop code helped us to make toy games and develop our intuition to code even before we learned our first programming language. ScratchML aims to take a similar approach, creating a learning platform that's easy-to-use and develops our way of thinking and intuition behind the scenes.
## What it does
Our platform is designed to teach machine learning and data analysis principles. The challenge is two-fold: making the learning fun and engaging while also providing a high quality curriculum developed around experiential learning. ScratchML delivers on both aspects by providing a reliable drag-and-drop interface used just like Scratch for model development and data experimentation, and through engineered datasets where the student are guided through exploring the data and reporting findings, students will be much more engaged through trying out a bunch of different approaches to accomplish a mission.
Each lesson comes with a workspace, where students can drag-and-drop models and other blocks to create a no-code, data analysis pipeline. A personalized tutor system is designed to guide the learning process, offering explanations and tips to guide the learning process. This system leverages the Prediction Guard LLM API to provide real-time insights into user decisions and outcomes within lessons. It employs state management to ensure continuous progress while fostering a sandbox-style learning environment.
## How we built it
The tech stack for the project consisted of:
React for the frontend, utilizing the Chakra UI component library and Tailwind CSS for styling.
A Flask server that runs and trains models based on the layout specified by the user.
We also used Intels extension for Scikit-learn/PyTorch to deliver faster training and inference time critical for making the user experience on the site seamless.
Firebase database for storing user data and storage of models that can be evaluated on the fly.
APIs: Prediction Guard LLM API to provide personalized real-time feedback using Neural-Chat-7B.
Models: Scikit-learn and PyTorch
Dev tools: Intel's Developer Cloud for constructing and testing the sandbox model, leveraging the PyTorch optimizations from Intel.
## Challenges we ran into
The largest challenge was to constantly rework the design to build a more intuitive, highly-functional interface. We are software developers by training; it was difficult to settle on a UI/UX design that achieves all of our priorities. Prior approaches for no-code machine learning are primarily designed for use in industry. Although they work well for older users who need to model data, at each stage we were super focused on whether or not each of the design elements were good for students. Furthermore, with this being our first or second hackathon experience, we worked hard to coordinate distributing work, ideating what could be feasibly accomplished in 36 hours, and ensuring that we accomplish our tasks in a timely manner. A final challenge we faced was simple endurance - the last few hours of development were extremely difficult given the sleep deprivation we all suffered from - overcoming this challenge was simply a matter of willpower and perseverance.
## Accomplishments that we're proud of
As amateur hackers, we are proud of what we accomplished this weekend - from building a helpful and innovative product from scratch to just the sheer amount of hard work we exhibited - this weekend proved to us that we are each capable of much more than we originally thought. Our team has limited hackathon experience, and we went in with the approach of not compromising on even our most ambitious ideas. We hacked together the base form of ScratchML, which supports all of the critical features that we set out to do at the beginning of the hackathon, and we are super excited to continue to work at the idea and think creatively and collaboratively on ways we can improve the learning experience for students in the future.
## What we learned
One of the largest takeaways from this weekend was simply that we are capable of much more than we originally believed. Getting together a team of passionate and driven individuals with aligned goals is a powerful tool to create and build.
Another lesson we learned in hindsight is the importance of sleep. Sometimes sacrifice can be beneficial, but it’s likely that our excitement in the earlier stages of the weekend came back to bite us during the final stretch.
Finally, this challenge asked all of us to wear a wide variety of hats, working with technologies and frameworks that we have limited experiences with. We learned a variety of different tools and also learned how to quickly rise to the occasion and accomplish the needs of the team.
## What's next for ScratchML
We plan on making the UI more intuitive, adding more lessons, increasing the number of blocks available in the sandbox, and increasing the number of datasets users can play with.
It is also imperative that we continue to think of creative ways to encourage learning. Our vision for the future of education, shared with many leaders in the space, is turning the classroom into a laboratory, where students can experiment and grow through trial-and-error. This requires coordinated collaboration and a lot of learning on our end as well, and we are eager to innovate and learn from innovators in the educational space to grow ScratchML into the go-to platform for young students trying to learn machine learning and data science principles.
## Detected evidence (automated analysis)
Indexed codebase: 30 recognized source files, 98 KB.
- CSS (language) — detected in the code
- Firebase (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- Flask (technology) — claimed on Devpost, not found in the code
- PyTorch (technology) — claimed on Devpost, not found in the code
## Codebase structure (from repository index)
### Files (56 of 56)
```
.DS_Store
.gitignore
backend/.firebaserc
backend/.gitignore
backend/database_utils.py
backend/endpoint_utils.py
backend/firebase.json
backend/firestore-debug.log
backend/server.py
backend/storage.rules
backend/ui-debug.log
data/MNIST/raw/t10k-images-idx3-ubyte
data/MNIST/raw/t10k-labels-idx1-ubyte
data/MNIST/raw/train-images-idx3-ubyte
data/MNIST/raw/train-labels-idx1-ubyte
frontend/.gitignore
frontend/package.json
frontend/public/index.html
frontend/public/manifest.json
frontend/public/robots.txt
frontend/README.md
frontend/setupTests.js
frontend/src/App.css
frontend/src/App.js
frontend/src/App.test.js
frontend/src/components/dnd/Constants.jsx
frontend/src/components/dnd/DndBins.jsx
frontend/src/components/dnd/DndBins.scss
frontend/src/components/dnd/DndElements.jsx
frontend/src/components/dnd/DndElements.scss
frontend/src/components/dnd/Draggable.jsx
frontend/src/components/dnd/Droppable.jsx
frontend/src/components/level.jsx
frontend/src/components/map.jsx
frontend/src/components/map.scss
frontend/src/components/sandbox.jsx
frontend/src/components/TaskList.jsx
frontend/src/components/TaskList.scss
frontend/src/firebase/init.js
frontend/src/helpers/callEndpoint.js
frontend/src/index.css
frontend/src/index.js
frontend/src/output.css
frontend/src/reportWebVitals.js
frontend/src/setupTests.js
frontend/src/views/landing_page.jsx
frontend/src/views/landing_page.scss
frontend/src/views/lesson_one.jsx
frontend/src/views/LevelPage.jsx
frontend/src/views/LevelPage.scss
frontend/tailwind.config.js
lesson_notebook/notebook.ipynb
lesson_notebook/test.csv
lesson_notebook/train.csv
README.md
yatin.txt
```
### Dependencies
- frontend/package.json: @chakra-ui/react@^2.8.2, @dnd-kit/core@^6.1.0, @emotion/react@^11.11.3, @emotion/styled@^11.11.0, @fontsource/lilita-one@^5.0.18, @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, firebase@^10.8.0, framer-motion@^11.0.5, leader-line@^1.0.7, localforage@^1.10.0, match-sorter@^6.3.4, node-sass@^7.0.3, react@^18.2.0, react-beautiful-dnd@^13.1.1, react-dnd@^16.0.1, react-dnd-html5-backend@^16.0.1, react-dom@^18.2.0, react-icons@^5.0.1, react-router-dom@^6.22.1, react-scripts@^5.0.1, sass@^1.71.0, tailwindcss@^3.4.1, web-vitals@^2.1.4
### Recent commits (newest first)
- Merge pull request #15 from theDubW/dylanjpaulson-patch-2
- Update README.md
- Merge pull request #14 from theDubW/dylanjpaulson-patch-1
- Update README.md
- Merge pull request #13 from theDubW/cleanup_structure
- Update README, organize files
- Merge pull request #12 from theDubW/update_readme
- Merge branch 'sandbox_flask'
- Merge pull request #11 from theDubW/readme
- Create README.md
- pirate ship logo
- Merge pull request #10 from theDubW/sandbox_flask
- map layout fixes
- display sandbox results
- working sandbox train
- semi working sandbox train
- Merge branch 'yatCity+Dyldo' of https://github.com/theDubW/ScratchML into yatCity+Dyldo
- added sandbox train and eval
- Merge pull request #9 from theDubW/yatCity+Dyldo
- tooltips for models
## Key source files (fetched from GitHub, selected and truncated for size)
### frontend/package.json
```
{
"name": "frontend",
"version": "0.1.0",
"private": true,
"dependencies": {
"@chakra-ui/react": "^2.8.2",
"@dnd-kit/core": "^6.1.0",
"@emotion/react": "^11.11.3",
"@emotion/styled": "^11.11.0",
"@fontsource/lilita-one": "^5.0.18",
"@testing-library/jest-dom": "^5.17.0",
"@testing-library/react": "^13.4.0",
"@testing-library/user-event": "^13.5.0",
"firebase": "^10.8.0",
"framer-motion": "^11.0.5",
"leader-line": "^1.0.7",
"localforage": "^1.10.0",
"match-sorter": "^6.3.4",
"node-sass": "^7.0.3",
"react": "^18.2.0",
"react-beautiful-dnd": "^13.1.1",
"react-dnd": "^16.0.1",
"react-dnd-html5-backend": "^16.0.1",
"react-dom": "^18.2.0",
"react-icons": "^5.0.1",
"react-router-dom": "^6.22.1",
"react-scripts": "^5.0.1",
"sass": "^1.71.0",
"web-vitals": "^2.1.4"
},
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject"
},
"eslintConfig": {
"extends": [
"react-app",
"react-app/jest"
]
},
"browserslist": {
"production": [
">0.2%",
"not dead",
"not op_mini all"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
},
"devDependencies": {
"tailwindcss": "^3.4.1"
}
}
```
### backend/server.py
```python
from flask import Flask, jsonify, request
import firebase_admin
from firebase_admin import credentials, firestore
from flask_cors import CORS
import os
from database_utils import get_data
import json
# from paramiko import SSHClient
# from fabric import Connection
# Assuming evaluate_model function is defined in endpoint_utils or a similar module
from endpoint_utils import (
gen_data,
train_and_upload_model,
evaluate_model,
generate_ml_experiment_feedback,
train_and_upload_sandbox_model,
evaluate_sandbox_model,
load_mnist_data,
)
app = Flask(__name__)
CORS(app)
cred = credentials.Certificate("./firebase-admin.json")
firebase_admin.initialize_app(
cred,
{
"projectId": "scratchml-treehacks",
# "databaseURL": "http://localhost:8080",
"storageBucket": "scratchml-treehacks.appspot.com",
# "auth": {"emulatorHost": "localhost:9099"},
},
)
db = firestore.client()
#
# client = SSHClient()
# client.load_system_host_keys("/Users/maxweinreb/.ssh/known_hosts")
print("connecting")
# c = Connection("100.82.12.111", user="ubuntu")
# c.open()
# print("connected")
# check if connected
# print("connected")
# result = c.run("echo YAH")
# print(result.stdout)
# Invoke-WebRequest -Uri http://localhost:5000/gen_data -Method Post -ContentType "application/json" -Body '{"uid": "user_10", "problem_name": "FoolsGold", "n": 10}'
# curl -X POST -H "Content-Type: application/json" -d '{"uid": "user_10", "problem_name": "FoolsGold", "n": 10}' http://localhost:5000/gen_data
@app.route("/gen_data", methods=["POST"])
def gen_user_data():
data = request.get_json()
print(data)
uid = data["uid"]
problem_name = data["problem_name"]
n = int(data["n"])
gen_data(db, uid, problem_name, n, True)
gen_data(db, uid, problem_name, n, False)
return jsonify({"status": "success"})
# Invoke-WebRequest -Uri http://localhost:5000/train -Method Post -ContentType "application/json" -Body '{"uid": "user_10", "problem_name": "FoolsGold", "model_name": "Decision Tree", "features": ["Hardness", "Density", "Conductivity", "Shininess", "Shape", "Texture"]}'
# curl -X POST -H "Content-Type: application/json" -d '{"uid": "user_10", "problem_name": "FoolsGold", "model_name": "Decision Tree", "features": ["Hardness", "Density", "Conductivity", "Shininess", "Shape", "Texture"]}' http://localhost:5000/train
@app.route("/train", methods=["POST"])
def train_model():
data = request.get_json()
print(data)
uid = data["uid"]
problem_name = data["problem_name"]
model_name = data["model_name"]
features = data["features"]
train_and_upload_model(db, uid, problem_name, model_name, features)
return jsonify({"status": "success"})
# Invoke-WebRequest -Uri http://localhost:5000/evaluate -Method Post -ContentType "application/json" -Body '{"uid": "user_10", "problem_name": "FoolsGold", "model_name": "Decision Tree", "features": ["Hardness", "Density", "Conductivity", "Shininess", "Shape", "Texture"]}'
# curl -X POST -H "Content-Type: application/json" -d '{"uid": "user_10", "problem_name": "FoolsGold", "model_name": "Decision Tree", "features": ["Hardness", "Density", "Conductivity", "Shininess", "Shape", "Texture"]}' http://localhost:5000/evaluate
@app.route("/evaluate", methods=["POST"])
def evaluate_user_model():
data = request.get_json()
uid = data["uid"]
problem_name = data["problem_name"]
model_name = data["model_name"]
features = data["features"]
evaluation_results = evaluate_model(db, uid, problem_name, model_name, features)
print(evaluation_results)
# Generate feedback using the evaluation results
data = get_data(db, uid, problem_name, True)
print("training data length", len(data))
feedback = generate_ml_experiment_feedback(
len(data), features, model_name, evaluation_results["accuracy"]
)
print(feedback)
return jsonify(
{"status": "success", "result": evaluation_results, "feedback": feedback}
)
# Invoke-RestMethod -Uri 'http://localhost:5000/train_sandbox' -Method Post -ContentType "application/json" -Body '{"uid":"user123","problem_name":"MNIST_Classification","model_name":"Custom_CNN","layer_list":[["conv",6],["conv",16],["linear",120],["linear",84]],"learning_rate":0.0001,"epochs":5,"optimizer_name":"Adam","criterion_name":"Cross Entropy","dataset":"MNIST"}'
@app.route("/train_sandbox", methods=["POST"])
def train_sandbox():
data = request.get_json()
uid = data["uid"]
problem_name = data["problem_name"]
model_name = data["model_name"]
layer_list = data["layer_list"]
learning_rate = data["learning_rate"]
epochs = data["epochs"]
optimizer_name = data["optimizer_name"]
criterion_name = data["criterion_name"]
dataset = data["dataset"]
train_loader, _ = load_mnist_data(32, 0.133, 0.31012)
layer_list_cleaned = []
print("LAYER LIST" + str(layer_list))
for layer in layer_list:
# print(layer)
if layer[0] == "Input" or layer[0] == "Output":
continue
layer_list_cleaned.append(layer)
print("CLEANED" + str(layer_list_cleaned))
train_and_upload_sandbox_model(
uid,
problem_name,
model_name,
layer_list_cleaned,
learning_rate,
epochs,
optimizer_name,
criterion_name,
train_loader,
)
return jsonify({"status": "success"})
# Invoke-RestMethod -Uri 'http://localhost:5000/evaluate_sandbox' -Method Post -ContentType "application/json" -Body '{"uid":"user123","problem_name":"MNIST_Classification","model_name":"Custom_CNN","criterion_name":"Cross Entropy"}'
@app.route("/evaluate_sandbox", methods=["POST"])
def evaluate_sandbox():
data = request.get_json()
uid = data["uid"]
problem_name = data["problem_name"]
model_name = data["model_name"]
criterion_name = data["criterion_name"]
_, test_loader = load_mnist_data(32, 0.133, 0.31012)
evaluatio
[truncated — 264 more characters]
```
### frontend/src/index.js
```javascript
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';
const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
);
// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();
```
### frontend/src/App.js
```javascript
import './App.css';
import { ChakraProvider, } from '@chakra-ui/react'
import { BrowserRouter as Router, Route, Routes, Link} from 'react-router-dom';
import LandingPage from './views/landing_page';
import lessonOne from './views/lesson_one';
import { Button, Stack, ButtonGroup} from '@chakra-ui/react'
import { Heading } from '@chakra-ui/react'
import { extendTheme, Text } from '@chakra-ui/react'
import './firebase/init'
import '@fontsource/lilita-one';
import {Card} from './components/dnd/DndElements'
import { Level } from './components/level';
import sandbox from './components/sandbox';
// import Card from
function App() {
const theme = extendTheme({
fonts: {
heading: `'Lilita One', sans-serif`,
// body: `'Lilita One', sans-serif`,
},
});
return (
ScratchML
);
}
export default App;
```
### frontend/setupTests.js
```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';
```
### frontend/tailwind.config.js
```javascript
/** @type {import('tailwindcss').Config} */
module.exports = {
content: ["./src/**/*.{html,js,jsx}"],
theme: {
extend: {},
},
theme: {
extend: {
fontFamily: {
// indieFlower: ["Indie Flower", "sans-serif"],
lilitaOne: ["Lilita One"],
signika: ["Signika"]
},
},
},
plugins: [],
}
```
### backend/database_utils.py
```python
from io import BytesIO
from joblib import dump, load
import pandas as pd
from firebase_admin import storage
from firebase_admin.firestore import Client
from sklearn.base import BaseEstimator
from typing import Any
def one_hot_encoding(df: pd.DataFrame) -> pd.DataFrame:
# get categorical columns
cat_columns = df.select_dtypes(include=["object"]).columns
for col in cat_columns:
dummies = pd.get_dummies(df[col], prefix=col)
df = df.drop(col, axis=1)
df = pd.concat([df, dummies], axis=1)
# print(cat_columns)
return df
# return pandas df with one hot encoding from firestore db
def get_data(db: Client, uid: str, problem_name: str, train: bool) -> pd.DataFrame:
# data_stream = db.collection("Users").document(uid).collection(problem_name).stream()
docName = "train" if train else "test"
docRef = (
db.collection("Users").document(uid).collection(problem_name).document(docName)
)
doc = docRef.get()
print(f"Getting data for {uid} {problem_name} {docName}")
if doc.exists:
item_dict = doc.to_dict()
df = pd.DataFrame.from_dict(item_dict)
return df
else:
print("doc doesn't exist")
return pd.DataFrame()
# print(df)
def upload_model_to_storage(
uid: str, problem_name: str, model_type: str, model: BaseEstimator
) -> None:
bucket = storage.bucket()
blob = bucket.blob(f"models/{uid}/{problem_name}/{model_type}.joblib")
buffer = BytesIO()
dump(model, buffer)
buffer.seek(0)
blob.upload_from_string(buffer.getvalue(), content_type="application/octet-stream")
buffer.close()
def download_model_from_storage(
uid: str, problem_name: str, model_type: str
) -> BaseEstimator:
bucket = storage.bucket()
blob = bucket.blob(f"models/{uid}/{problem_name}/{model_type}.joblib")
buffer = BytesIO()
blob.download_to_file(buffer)
buffer.seek(0)
model = load(buffer)
buffer.close()
return model
```
### backend/endpoint_utils.py
```python
import numpy as np
import pandas as pd
from typing import Union
from firebase_admin.firestore import Client
from sklearn.base import BaseEstimator
from database_utils import (
get_data,
one_hot_encoding,
upload_model_to_storage,
download_model_from_storage,
)
from sklearn.tree import DecisionTreeClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, confusion_matrix
from typing import List, Tuple, Any
import os
import predictionguard as pg
import torch
import torch.utils
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
from torch.utils.data import DataLoader
# import intel_extension_for_pytorch as ipex
from torchvision import datasets, transforms
from torch.utils.data import DataLoader, Subset
def gen_data(
db: Client, uid: str, problem_name: str, n: int, train: bool = True
) -> None:
# relevant features
hardness = np.random.randint(10, 40, n)
density = np.random.randint(1, 11, n)
conductivity = np.random.randint(1, 11, n)
textures = np.random.choice(["smooth", "rough"], size=n)
# irrelevant features
shininess = np.random.randint(1, 11, n)
shapes = np.random.choice(["square", "circle", "rectangle", "triangle"], size=n)
# Assign numerical values to colors based on an arbitrary property (e.g., "warmth")
texture_values = {"smooth": 0, "rough": 1}
texture_numerical = np.array([texture_values[t] for t in textures])
# Calculate condition numbers using the non-linear transformation, now including color
conditions = [0] * n
for i in range(n):
if hardness[i] < 25:
conditions[i] = (
hardness[i]
+ (10 + 5 * density[i]) * texture_numerical[i]
+ np.log(conductivity[i] + 1)
)
else:
conditions[i] = (
hardness[i]
- (10 + 5 * density[i]) * texture_numerical[i]
- np.log(conductivity[i] + 1)
)
labels = [1 if v > 25 else 0 for v in conditions]
# Create DataFrame
new_data = pd.DataFrame(
{
"Label": labels,
"Hardness": hardness,
"Density": density,
"Conductivity": conductivity,
"Shininess": shininess,
"Shape": shapes,
"Texture": textures,
}
)
print("Getting past")
cur_data = get_data(db, uid, problem_name, train)
print("Got past")
new_data = pd.concat([new_data, cur_data])
db_firestore = {}
# Push data to Firebase
for record in new_data.columns:
db_firestore[record] = new_data[record].tolist()
print("pushing to server")
docName = "train" if train else "test"
db.collection("Users").document(uid).collection(problem_name).document(docName).set(
db_firestore, merge=True
)
def train_and_upload_model(
db: Client, uid: str, problem_name: str, model_type: str, features: list
) -> None:
# Read raw data
raw_df = get_data(db, uid, problem_name, train=True)
# Filter the DataFrame to only include the specified features before one-hot encoding
# Assuming 'label' is not included in the features list and is added separately
filtered_df = raw_df[features + ["Label"]]
# Apply one-hot encoding to the filtered DataFrame
df = one_hot_encoding(filtered_df)
X = df.drop("Label", axis=1)
y = df["Label"]
# Determine model type
if model_type == "Decision Tree":
model = DecisionTreeClassifier()
elif model_type == "Logistic Regression":
model = LogisticRegression(max_iter=1000)
elif model_type == "K-Nearest Neighbors":
model = KNeighborsClassifier()
else:
print(model_type)
raise ValueError("Bad model type")
model.fit(X, y)
upload_model_to_storage(uid, problem_name, model_type, model)
def evaluate_model(
db: Client, uid: str, problem_name: str, model_type: str, features: list
) -> dict:
# Read raw data
raw_df = get_data(db, uid, problem_name, train=False)
# Filter the DataFrame to only include the specified features before one-hot encoding
filtered_df = raw_df[features + ["Label"]]
# Apply one-hot encoding to the filtered DataFrame
test_df = one_hot_encoding(filtered_df)
X_test = test_df.drop("Label", axis=1)
y_test = test_df["Label"]
# Load the model
model = download_model_from_storage(uid, problem_name, model_type)
# Make predictions and evaluate
predictions = model.predict(X_test)
accuracy = accuracy_score(y_test, predictions)
# Compute confusion matrix
cm = confusion_matrix(y_test, predictions)
# Flatten the confusion matrix if it's for binary classification
cm_flattened = cm.flatten() if cm.size == 4 else cm
# Return evaluation metrics including the confusion matrix
return {
"accuracy": np.round(accuracy, 3),
"confusion_matrix": cm_flattened.tolist(), # Convert numpy array to list for JSON serialization
}
def translateLayerToCode(layer_name):
if layer_name == "Convolutional":
return "conv"
elif layer_name == "Linear":
return "linear"
else:
return ValueError("Bad layer type")
def setup_predictionguard_token(token: str) -> None:
os.environ["PREDICTIONGUARD_TOKEN"] = token
def generate_ml_experiment_feedback(
n: int, features: List[str], model_type: str, accuracy: float
) -> str:
system_message_1 = {
"role": "system",
"content": """
Guide students when their dataset size is small. Encourage exploring the impact of dataset size on model accuracy.
Feedback template:
"Nice job on starting your experiment! If your accuracy isn't quite where you want it to be, consider how more data might help your model learn better. What happens if you increase your dataset?"
""",
}
[truncated — 11395 more characters]
```
### frontend/src/setupTests.js
```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';
```
### frontend/src/App.test.js
```javascript
import { render, screen } from '@testing-library/react';
import App from './App';
test('renders learn react link', () => {
render();
const linkElement = screen.getByText(/learn react/i);
expect(linkElement).toBeInTheDocument();
});
```
[18 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]