# Project export: GhostWriter This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: your silent partner in progress - Devpost: https://devpost.com/software/ghostwriter-491kuy - GitHub: https://github.com/rqchao/calhacks24 - Video: https://www.youtube.com/embed/aWd5eajo6KI?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (Cal Hacks: 2nd Overall) - Team: 2 GitHub contributor(s) — Derek Xu (15 commits), Richard Chao (9 commits) ## Devpost submission (written by the team) ### Inspiration Introducing Ghostwriter: Your silent partner in progress. Ever been in a class where resources are so hard to come by, you find yourself practically living at office hours? As teaching assistants on increasingly short-handed course staffs, it can be difficult to keep up with student demands while making long-lasting improvements to your favorite courses. Imagine effortlessly improving your course materials as you interact with students during office hours. Ghostwriter listens intelligently to these conversations, capturing valuable insights and automatically updating your notes and class documentation. No more tedious post-session revisions or forgotten improvement ideas. Instead, you can really focus on helping your students in the moment. Ghostwriter is your silent partner in educational excellence, turning every interaction into an opportunity for long-term improvement. It's the invisible presence that delivers visible results, making continuous refinement effortless and impactful. With Ghostwriter, you're not just tutoring or bug-bashing - you're evolving your content with every conversation. ### What it does Ghostwriter hosts your class resources, and supports searching across them in many ways (by metadata, semantically by content). It allows adding, deleting, and rendering markdown notes. However, Ghostwriter's core feature is in its recording capabilities. The record button starts a writing session. As you speak, Ghostwriter will transcribe and digest your speech, decide whether it's worth adding to your notes, and if so, navigate to the appropriate document and insert them at a line-by-line granularity in your notes, integrating seamlessly with your current formatting. ### How we built it We used Reflex to build the app full-stack in Python, and support the various note-management features including addition, deleting, selecting, and rendering. As notes are added to the application database, they are also summarized and then embedded by Gemini 1.5 Flash-8B before being added to ChromaDB with a shared key. Our semantic search is also powered by Gemini-embedding and ChromaDB. The recording feature is powered by Deepgram's threaded live-audio transcription API. The text is processed live by Gemini, and chunks are sent to ChromaDB for queries. Distance metrics are used as thresholds to not create notes, add to an existing note, or create a new note. In the latter two cases, llama3-70b-8192 is run through Groq to write on our (existing) documents. It does this through a RAG on our docs, as well as some prompt-engineering. To make insertion granular we add unique tokens to identify candidate insertion-points throughout our original text. We then structurally generate the desired markdown, as well as the desired point of insertion, and render the changes live to the user. ### Challenges we ran into Using Deepgram and live-generation required a lot of tasks to run concurrently, without blocking UI interactivity. We had some trouble reconciling the requirements posed by Deepgram and Reflex on how these were handled, and required us redesign the backend a few times. Generation was also rather difficult, as text would come out with irrelevant vestiges and explanations. It took a lot of trial and error through prompting and other tweaks to the generation calls and structure to get our required outputs. ### Accomplishments we're proud of Our whole live note-generation pipeline! From audio transcription process to the granular retrieval-augmented structured generation process. Spinning up a full-stack application using Reflex (especially the frontend, as two backend engineers) We were also able to set up a few tools to push dummy data into various points of our process, which made debugging much, much easier. ### What's next Ghostwriter can work on the student-side as well, allowing a voice-interface to improving your own class notes, perhaps as a companion during lecture. We find Ghostwriter's note identification and improvement process very useful ourselves. On the teaching end, we hope GhostWriter will continue to grow into a well-rounded platform for educators on all ends. We envision that office hour questions and engagement going through our platform can be aggregated to improve course planning to better fit students' needs. Ghostwriter's potential doesn't stop at education. In the software world, where companies like AWS and Databricks struggle with complex documentation and enormous solutions teams, Ghostwriter shines. It transforms customer support calls into documentation gold, organizing and structuring information seamlessly. This means fewer repetitive calls and more self-sufficient users! ## README (from the GitHub repository) ![title](https://github.com/user-attachments/assets/c4c0f9e8-4919-4b2f-9348-7325e92bdfa6) Introducing Ghostwriter: Your silent partner in educational excellence. Ever been in a class where resources are so hard to come by, you find yourself living in office hours? As teaching assistants on increasingly short-handed course staffs, it can be difficult to keep up with student demands, while making long-lasting improvements to your favorite courses. Imagine effortlessly improving your course materials as you interact with students during office hours. Ghostwriter listens intelligently to these conversations, capturing valuable insights and automatically updating your notes and class documentation. No more tedious post-session revisions or forgotten improvement ideas. Ghostwriter turns every interaction into an opportunity for improvement. It's the invisible presence that delivers visible results, making continuous refinement effortless and impactful. With Ghostwriter, you're not just tutoring or bug-bashing - you're evolving your content with every conversation. ## Detected evidence (automated analysis) Indexed codebase: 25 recognized source files, 62 KB. - Python (language) — detected in the code - Google Gemini (technology) — claimed on Devpost, not found in the code - React (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (31 of 31) ``` .DS_Store .gitignore alembic.ini alembic/env.py alembic/README alembic/script.py.mako alembic/versions/277cad49d2b0_.py alembic/versions/333b61328fdd_.py alembic/versions/7aaec6b87d88_.py alembic/versions/8fdaf04c547e_.py alembic/versions/b0a981c8e408_.py alembic/versions/d107f950e0e1_.py alembic/versions/e565fdc23e6c_something_changed.py contract.md customer_data_app/__init__.py customer_data_app/backend/__init__.py customer_data_app/backend/backend.py customer_data_app/backend/client.py customer_data_app/backend/const.py customer_data_app/backend/summarization.py customer_data_app/backend/vectordb.py customer_data_app/components/__init__.py customer_data_app/components/form_field.py customer_data_app/components/status_badges.py customer_data_app/customer_data_app.py customer_data_app/views/__init__.py customer_data_app/views/main.py customer_data_app/views/navbar.py README.md requirements.txt rxconfig.py ``` ### Dependencies - requirements.txt: chromadb, deepgram-sdk, groq, janus@==1.0.0, psycopg2-binary, PyAudio@==0.2.14, python-dotenv, reflex@>=0.5.2, websockets@==12.0 ### Recent commits (newest first) - readme - added different enough, make new page - start text streaming - prompt engineering and using better model - prompt tuning - fix frontend, note selction - llm output wrangling - transcription and adding to note working now - Merge branch 'main' of https://github.com/rqchao/calhacks24 - deepgram setup with record button - persistent db - semantic search working - fixing chromadb and backend logic - Added debug components - Another frontend rehaul - Frontend rehaul - adding sample notes - adding course id - vectordb done and some addtl backend - remove submodule & change client to listen only ## Key source files (fetched from GitHub, selected and truncated for size) ### contract.md ```markdown POST - `record_start` Response: bool POST - `record_end` Response: bool ``` ### requirements.txt ``` reflex>=0.5.2 psycopg2-binary PyAudio==0.2.14 websockets==12.0 janus==1.0.0 python-dotenv deepgram-sdk groq chromadb ``` ### customer_data_app/views/main.py ```python import reflex as rx from ..backend.backend import State, Note from ..components.form_field import form_field, form_field_textarea def show_notes(note: Note): """Show a note in a table row with selectable styling.""" is_selected = State.selected_note.uuid == note.uuid return rx.table.row( rx.table.cell(note.uuid[:8]), rx.table.cell(note.name), rx.table.cell(note.course_id), rx.table.cell(note.date), rx.table.cell( rx.hstack( rx.icon_button( rx.icon("trash-2", size=22), on_click=lambda: State.delete_note(getattr(note, "name")), size="2", variant="solid", color_scheme="red", ), ) ), style={ "_hover": {"bg": rx.color("gray", 3)}, "bg": rx.cond(is_selected, rx.color("gray", 2), "transparent"), }, on_click=lambda: State.select_note(note.uuid), cursor="pointer", align="center", ) def push_test_data(): return rx.el.button( "Push Data", background_color="#4F46E5", transition_duration="300ms", font_weight="500", _hover={"background-color": "#4338CA"}, padding_left="1rem", padding_right="1rem", padding_top="0.5rem", padding_bottom="0.5rem", border_radius="9999px", font_size="0.875rem", line_height="1.25rem", color="#ffffff", on_click=State.create_sample_notes, transition_property="background-color, border-color, color, fill, stroke, opacity, box-shadow, transform", transition_timing_function="cubic-bezier(0.4, 0, 0.2, 1)", ) def add_document_button() -> rx.Component: return rx.dialog.root( rx.dialog.trigger( rx.button( rx.icon("plus", size=26), rx.text("Add Note", size="4", display=[ "none", "none", "block"]), size="3", ), ), rx.dialog.content( rx.hstack( rx.badge( rx.icon(tag="users", size=34), color_scheme="grass", radius="full", padding="0.65rem", ), rx.vstack( rx.dialog.title( "Add New Note", weight="bold", margin="0", ), rx.dialog.description( "Fill out the form with the note details and content", ), spacing="1", height="100%", align_items="start", ), height="100%", spacing="4", margin_bottom="1.5em", align_items="center", width="100%", ), rx.flex( rx.form.root( rx.flex( # Name form_field( "Title", "Note Title", "text", "name", "a-large-small", ), # Course ID form_field( "Course ID", "CS 61A", "text", "course_id", "scan-barcode", ), # Note Content form_field_textarea( label="Note Content", placeholder="Content", name="content", icon="notepad-text", ), # Status # rx.vstack( # rx.hstack( # rx.icon("truck", size=16, stroke_width=1.5), # rx.text("Status"), # align="center", # spacing="2", # ), # rx.radio( # ["Delivered", "Pending", "Cancelled"], # name="status", # direction="row", # as_child=True, # required=True, # ), # ), direction="column", spacing="3", ), rx.flex( rx.dialog.close( rx.button( "Cancel", variant="soft", color_scheme="gray", ), ), rx.form.submit( rx.dialog.close( rx.button("Add notes"), ), as_child=True, ), padding_top="2em", spacing="3", mt="4", justify="end", ), on_submit=State.add_note_to_db, reset_on_submit=False, ), width="100%", direction="column", spacing="4", ), style={"max_width": 450}, box_shadow="lg", padding="1.5em", border=f"2px solid {rx.color('accent', 7)}", border_radius="25px", ), ) def document_dis [truncated — 5886 more characters] ``` ### rxconfig.py ```python import reflex as rx config = rx.Config(app_name="customer_data_app") ``` ### customer_data_app/customer_data_app.py ```python """Welcome to Reflex! This file outlines the steps to create a basic app.""" import reflex as rx from .views.main import document_display_box from .views.navbar import navbar from .views.main import main_table def index() -> rx.Component: return rx.vstack( navbar(), rx.flex( rx.box(main_table(), width=["100%", "100%", "100%", "60%"]), document_display_box(), spacing="6", width="100%", flex_direction=["column", "column", "column", "row"], ), height="100vh", bg=rx.color("accent", 1), width="100%", spacing="6", padding_x=["1.5em", "1.5em", "3em"], padding_y=["1em", "1em", "2em"], ) # Create app instance and add index page. app = rx.App( theme=rx.theme( appearance="dark", has_background=True, radius="large", accent_color="indigo" ), ) app.add_page( index, title="Ghostwriter", description="Your virtual teaching assistant.", ) ``` ### alembic/env.py ```python from logging.config import fileConfig from alembic import context from sqlalchemy import engine_from_config, pool # this is the Alembic Config object, which provides # access to the values within the .ini file in use. config = context.config # Interpret the config file for Python logging. # This line sets up loggers basically. if config.config_file_name is not None: fileConfig(config.config_file_name) # add your model's MetaData object here # for 'autogenerate' support # from myapp import mymodel # target_metadata = mymodel.Base.metadata target_metadata = None # other values from the config, defined by the needs of env.py, # can be acquired: # my_important_option = config.get_main_option("my_important_option") # ... etc. def run_migrations_offline() -> None: """Run migrations in 'offline' mode. This configures the context with just a URL and not an Engine, though an Engine is acceptable here as well. By skipping the Engine creation we don't even need a DBAPI to be available. Calls to context.execute() here emit the given string to the script output. """ url = config.get_main_option("sqlalchemy.url") context.configure( url=url, target_metadata=target_metadata, literal_binds=True, dialect_opts={"paramstyle": "named"}, ) with context.begin_transaction(): context.run_migrations() def run_migrations_online() -> None: """Run migrations in 'online' mode. In this scenario we need to create an Engine and associate a connection with the context. """ connectable = engine_from_config( config.get_section(config.config_ini_section, {}), prefix="sqlalchemy.", poolclass=pool.NullPool, ) with connectable.connect() as connection: context.configure(connection=connection, target_metadata=target_metadata) with context.begin_transaction(): context.run_migrations() if context.is_offline_mode(): run_migrations_offline() else: run_migrations_online() ``` ### customer_data_app/components/status_badges.py ```python import reflex as rx def _badge(icon: str, text: str, color_scheme: str): return rx.badge(rx.icon(icon, size=16), text, color_scheme=color_scheme, radius="full", variant="soft", size="3") def status_badge(status: str): badge_mapping = { "Delivered": ("check", "Delivered", "green"), "Pending": ("loader", "Pending", "yellow"), "Cancelled": ("ban", "Cancelled", "red") } return _badge(*badge_mapping.get(status, ("loader", "Pending", "yellow"))) ``` ### alembic/versions/277cad49d2b0_.py ```python """empty message Revision ID: 277cad49d2b0 Revises: 7aaec6b87d88 Create Date: 2024-05-30 10:58:18.235598 """ from typing import Sequence, Union from alembic import op import sqlalchemy as sa import sqlmodel # revision identifiers, used by Alembic. revision: str = '277cad49d2b0' down_revision: Union[str, None] = '7aaec6b87d88' branch_labels: Union[str, Sequence[str], None] = None depends_on: Union[str, Sequence[str], None] = None def upgrade() -> None: # ### commands auto generated by Alembic - please adjust! ### with op.batch_alter_table('customer', schema=None) as batch_op: batch_op.add_column(sa.Column('date', sqlmodel.sql.sqltypes.AutoString(), nullable=False)) # ### end Alembic commands ### def downgrade() -> None: # ### commands auto generated by Alembic - please adjust! ### with op.batch_alter_table('customer', schema=None) as batch_op: batch_op.drop_column('date') # ### end Alembic commands ### ``` ### alembic/versions/e565fdc23e6c_something_changed.py ```python """something changed. Revision ID: e565fdc23e6c Revises: 333b61328fdd Create Date: 2024-01-23 13:49:28.720768 """ from typing import Sequence, Union import sqlalchemy as sa from alembic import op # revision identifiers, used by Alembic. revision: str = "e565fdc23e6c" down_revision: Union[str, None] = "333b61328fdd" branch_labels: Union[str, Sequence[str], None] = None depends_on: Union[str, Sequence[str], None] = None def upgrade() -> None: # ### commands auto generated by Alembic - please adjust! ### op.drop_column("customer", "edit") op.drop_column("customer", "icon") # ### end Alembic commands ### def downgrade() -> None: # ### commands auto generated by Alembic - please adjust! ### op.add_column( "customer", sa.Column("icon", sa.VARCHAR(), autoincrement=False, nullable=False) ) op.add_column( "customer", sa.Column("edit", sa.VARCHAR(), autoincrement=False, nullable=False) ) # ### end Alembic commands ### ``` ### alembic/versions/b0a981c8e408_.py ```python """empty message Revision ID: b0a981c8e408 Revises: 8fdaf04c547e Create Date: 2024-10-19 18:13:23.027532 """ from typing import Sequence, Union from alembic import op import sqlalchemy as sa import sqlmodel # revision identifiers, used by Alembic. revision: str = 'b0a981c8e408' down_revision: Union[str, None] = '8fdaf04c547e' branch_labels: Union[str, Sequence[str], None] = None depends_on: Union[str, Sequence[str], None] = None def upgrade() -> None: # ### commands auto generated by Alembic - please adjust! ### with op.batch_alter_table('note', schema=None) as batch_op: batch_op.add_column(sa.Column('uuid', sqlmodel.sql.sqltypes.AutoString(), nullable=False)) batch_op.drop_column('id') # ### end Alembic commands ### def downgrade() -> None: # ### commands auto generated by Alembic - please adjust! ### with op.batch_alter_table('note', schema=None) as batch_op: batch_op.add_column(sa.Column('id', sa.INTEGER(), nullable=False)) batch_op.drop_column('uuid') # ### end Alembic commands ### ``` [11 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]