# Project export: Posthuman This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: Posthuman ensures your digital legacy lives on with AI-powered estate management, secure asset delegation, and blockchain-backed will execution—seamlessly handling your affairs when you no longer can. - Devpost: https://devpost.com/software/posthuman - GitHub: https://github.com/r-agni/posthuman - Demo: https://docs.google.com/presentation/d/1Pkberz8pt2w35VmKCpIlouXlYrs8ZRXCfQo6l2RSj3Q/edit?usp=sharing - Video: https://www.youtube.com/embed/fiaekXgazbY?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — Jathin Pranav Singaraju (12 commits), jeffreygong1207 (9 commits), ATG (5 commits), Shivansh Soni (4 commits) ## Devpost submission (written by the team) No Devpost description available. ## README (from the GitHub repository) # Posthuman: Personal Family Interaction and Asset Management AI Posthuman is an advanced AI-driven system designed to assist with personal family interactions, future event planning, digital asset management, and posthumous account execution. It leverages cutting-edge technologies and operates entirely on local storage to ensure data privacy and security. ## Features ### Personal Interaction and Memory Preservation - **Private and Public Memory Pools**: Integrates data from private and public memory sources for personalized interaction (e.g., friends/family memory pools). - **Family Interaction Chatbot**: Uses Mistral AI to enable meaningful conversations based on user-provided data. - **Future Event Messages**: Auto-generates and sends messages for important events using: - **Deep Fake Video**: Personalized video creation. - **AI Audio Recording**: Authentic voice replication using ElevenLabs. - **Speech and Lip Synchronization**: Realistic animation using Luma for positioning and Muse Talk for lip-sync. - **Digital Memorialization**: Creates personalized video messages to memorialize an online presence for loved ones. ### Data Sources All data and analysis are stored locally for enhanced privacy. Sources include: - Text messages, photos, and emails. - Cloud accounts and storage (e.g., Drive, personal cloud profiles). - Financial statements and password managers. - Cryptocurrency wallets and blockchain data. --- ## Asset Discovery and Management - **Digital Footprint Discovery**: - Identifies digital assets, bank accounts, and subscriptions. - Analyzes fiscal statements/returns using KNN with Mistral local models and OCR. - **Asset Delegation**: - Stores key assets and accounts for delegation upon an inflection point (e.g., post-mortem event). - Links with Ethereum wallets and manages digital portfolios validated via EigenLayer. --- ## Executor Assistance - **Automated Operations**: - Cancels subscriptions and payments posthumously using AI Operator Agent Scrappybara with Mistral context inputs. - Deletes accounts and profiles efficiently. - **Task Scheduling**: - Cron jobs for post-mortem tasks. - Live context tasks managed via Perplexity Sonar API. --- ## Blockchain Integration - **Tamper-Proof Execution**: - Blockchain ledger maintained with EigenLayer AVS for transparent will execution. - Human-agent conversation logs validate transactions. - **Encrypted Hashing**: - AI agent ensures encrypted on-chain verification of critical tasks. --- ## Technology Stack - **AI Models**: - Mistral AI for interaction and context-based analysis. - ElevenLabs for voice replication. - Luma for video/image positioning. - Langgraph for Agentic Workflows - Scrappybara for agentic web browser based task automation. - **Databases**: - Elastic Vector Database for storing local data. - **Blockchain**: - Ethereum with EigenLayer for transaction validation and portfolio management. - **Full Stack**: - Vercel v0: Frontend --- ## Privacy and Security - **Local Data Storage**: All data and analysis are stored locally, ensuring no external access to sensitive information. - **Encrypted Blockchain**: Transactions and data are secured with encrypted hashes on-chain. --- ## Getting Started 1. Clone the repository: ```bash git clone https://github.com/your-repo-name/posthuman.git ## Detected evidence (automated analysis) Indexed codebase: 90 recognized source files, 400 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - Flask (technology) — detected in the code - Hugging Face (technology) — detected in the code - Next.js (technology) — detected in the code - Ollama (technology) — detected in the code - Python (language) — detected in the code - PyTorch (technology) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TensorFlow (technology) — detected in the code - TypeScript (language) — detected in the code - Docker (technology) — claimed on Devpost, not found in the code - Mistral AI (technology) — claimed on Devpost, not found in the code - Vercel (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (110 of 110) ``` .gitignore ai-models/finance-assets/backend.py ai-models/knowledge-rag/download_elastic.py ai-models/requirements.txt ai-models/scrapybara/index.tsx ai-models/text-agent/talk.py ai-models/text-agent/temp.py components.json eslint.config.mjs next.config.ts package.json postcss.config.mjs public/events.json README.md server/MuseTalk/app.py server/MuseTalk/configs/inference/realtime.yaml server/MuseTalk/configs/inference/test.yaml server/MuseTalk/elevenlabs.py server/MuseTalk/entrypoint.sh server/MuseTalk/luma.py server/MuseTalk/main.py server/MuseTalk/musetalk/models/unet.py server/MuseTalk/musetalk/models/vae.py server/MuseTalk/musetalk/utils/__init__.py server/MuseTalk/musetalk/utils/blending.py server/MuseTalk/musetalk/utils/dwpose/default_runtime.py server/MuseTalk/musetalk/utils/dwpose/rtmpose-l_8xb32-270e_coco-ubody-wholebody-384x288.py server/MuseTalk/musetalk/utils/face_detection/__init__.py server/MuseTalk/musetalk/utils/face_detection/api.py server/MuseTalk/musetalk/utils/face_detection/detection/__init__.py server/MuseTalk/musetalk/utils/face_detection/detection/core.py server/MuseTalk/musetalk/utils/face_detection/detection/sfd/__init__.py server/MuseTalk/musetalk/utils/face_detection/detection/sfd/bbox.py server/MuseTalk/musetalk/utils/face_detection/detection/sfd/detect.py server/MuseTalk/musetalk/utils/face_detection/detection/sfd/net_s3fd.py server/MuseTalk/musetalk/utils/face_detection/detection/sfd/sfd_detector.py server/MuseTalk/musetalk/utils/face_detection/models.py server/MuseTalk/musetalk/utils/face_detection/README.md server/MuseTalk/musetalk/utils/face_detection/utils.py server/MuseTalk/musetalk/utils/face_parsing/__init__.py server/MuseTalk/musetalk/utils/face_parsing/model.py server/MuseTalk/musetalk/utils/face_parsing/resnet.py server/MuseTalk/musetalk/utils/preprocessing.py server/MuseTalk/musetalk/utils/utils.py server/MuseTalk/musetalk/whisper/audio2feature.py server/MuseTalk/musetalk/whisper/whisper/__init__.py server/MuseTalk/musetalk/whisper/whisper/__main__.py server/MuseTalk/musetalk/whisper/whisper/assets/gpt2/merges.txt server/MuseTalk/musetalk/whisper/whisper/assets/gpt2/special_tokens_map.json server/MuseTalk/musetalk/whisper/whisper/assets/gpt2/tokenizer_config.json server/MuseTalk/musetalk/whisper/whisper/assets/gpt2/vocab.json server/MuseTalk/musetalk/whisper/whisper/assets/mel_filters.npz server/MuseTalk/musetalk/whisper/whisper/assets/multilingual/added_tokens.json server/MuseTalk/musetalk/whisper/whisper/assets/multilingual/merges.txt server/MuseTalk/musetalk/whisper/whisper/assets/multilingual/special_tokens_map.json server/MuseTalk/musetalk/whisper/whisper/assets/multilingual/tokenizer_config.json server/MuseTalk/musetalk/whisper/whisper/assets/multilingual/vocab.json server/MuseTalk/musetalk/whisper/whisper/audio.py server/MuseTalk/musetalk/whisper/whisper/decoding.py server/MuseTalk/musetalk/whisper/whisper/model.py server/MuseTalk/musetalk/whisper/whisper/normalizers/__init__.py server/MuseTalk/musetalk/whisper/whisper/normalizers/basic.py server/MuseTalk/musetalk/whisper/whisper/normalizers/english.json server/MuseTalk/musetalk/whisper/whisper/normalizers/english.py server/MuseTalk/musetalk/whisper/whisper/tokenizer.py server/MuseTalk/musetalk/whisper/whisper/transcribe.py server/MuseTalk/musetalk/whisper/whisper/utils.py server/MuseTalk/requirements.txt server/MuseTalk/router.py server/MuseTalk/scripts/__init__.py server/MuseTalk/scripts/inference.py server/MuseTalk/scripts/realtime_inference.py server/MuseTalk/temp_config.yaml server/MuseTalk/testing.py server/MuseTalk/utils.py src/app/(home)/assets/page.tsx src/app/(home)/chat/page.tsx src/app/(home)/events/page.tsx src/app/(home)/finances/page.tsx src/app/(home)/layout.tsx src/app/(home)/profile/page.tsx src/app/(home)/subscriptions/page.tsx src/app/(home)/will/page.tsx src/app/api/chat/route.ts src/app/api/events/route.ts src/app/api/scrapybara/route.ts src/app/api/send-video/route.ts src/app/api/upload-pic/route.ts src/app/globals.css src/app/layout.tsx src/app/page.tsx src/components/sidebar.tsx src/components/ui/alert.tsx src/components/ui/badge.tsx src/components/ui/button.tsx src/components/ui/calendar.tsx src/components/ui/card.tsx src/components/ui/checkbox.tsx src/components/ui/dialog.tsx src/components/ui/input.tsx src/components/ui/label.tsx src/components/ui/progress.tsx src/components/ui/select.tsx src/components/ui/tabs.tsx src/components/ui/textarea.tsx src/hooks/cronjob.ts src/lib/utils.ts src/types/scrapybara.d.ts tailwind.config.ts tsconfig.json ``` ### Dependencies - ai-models/requirements.txt: annotated-types@==0.7.0, anthropic@==0.39.0, anyio@==4.8.0, blinker@==1.9.0, certifi@==2025.1.31, charset-normalizer@==3.4.1, click@==8.1.8, distro@==1.9.0, elastic-transport@==8.17.0, elasticsearch@==8.17.1, filelock@==3.17.0, Flask@==3.1.0, fsspec@==2025.2.0, greenlet@==3.1.1, h11@==0.14.0, httpcore@==1.0.7, httpx@==0.28.1, huggingface-hub@==0.28.1, idna@==3.10, itsdangerous@==2.2.0, Jinja2@==3.1.5, jiter@==0.8.2, joblib@==1.4.2, MarkupSafe@==3.0.2, mpmath@==1.3.0, networkx@==3.4.2, numpy@==2.2.3, ollama@==0.4.7, packaging@==24.2, pillow@==11.1.0, playwright@==1.50.0, pydantic@==2.10.6, pydantic_core@==2.27.2, pyee@==12.1.1, PyYAML@==6.0.2, regex@==2024.11.6, requests@==2.32.3, safetensors@==0.5.2, scikit-learn@==1.6.1, scipy@==1.15.1, scrapybara@==2.2.7, sentence-transformers@==3.4.1, sniffio@==1.3.1, sympy@==1.13.1, threadpoolctl@==3.5.0, tokenizers@==0.21.0, torch@==2.6.0, tqdm@==4.67.1, transformers@==4.48.3, typing_extensions@==4.12.2, urllib3@==2.3.0, Werkzeug@==3.1.3 - package.json: @emotion/react@^11.14.0, @emotion/styled@^11.14.0, @eslint/eslintrc@^3, @mui/material@^6.4.4, @radix-ui/react-checkbox@^1.1.4, @radix-ui/react-dialog@^1.1.6, @radix-ui/react-label@^2.1.2, @radix-ui/react-progress@^1.1.2, @radix-ui/react-select@^2.1.6, @radix-ui/react-separator@^1.1.2, @radix-ui/react-slot@^1.1.2, @radix-ui/react-tabs@^1.1.3, @radix-ui/react-toast@^1.2.6, @radix-ui/react-tooltip@^1.1.8, @react-three/drei@^9.121.5, @react-three/fiber@^8.17.14, @types/node@^20, @types/node-cron@^3.0.11, @types/pdfjs-dist@^2.10.378, @types/react@^19, @types/react-dom@^19, class-variance-authority@^0.7.1, clsx@^2.1.1, date-fns@^3.6.0, eslint@^9, eslint-config-next@15.1.7, framer-motion@^12.4.3, lucide-react@^0.475.0, next@^15.1.7, node-cron@^3.0.3, pdfjs-dist@^4.10.38, postcss@^8, react@^18.3.1, react-day-picker@^8.10.1, react-dom@^18.3.1, react-toastify@^11.0.3, scrapybara@^2.2.2, shadcn-ui@^0.9.4, tailwind-merge@^3.0.1, tailwindcss@^3.4.1, tailwindcss-animate@^1.0.7, tesseract.js@^6.0.0, typescript@^5, zustand@^5.0.3 - server/MuseTalk/requirements.txt: absl-py@==2.1.0, accelerate@==0.28.0, addict@==2.4.0, aiofiles@==23.2.1, aliyun-python-sdk-core@==2.16.0, aliyun-python-sdk-kms@==2.16.5, annotated-types@==0.7.0, antlr4-python3-runtime@==4.9.3, anyio@==4.8.0, astunparse@==1.6.3, beautifulsoup4@==4.13.3, cachetools@==5.5.1, certifi@==2025.1.31, cffi@==1.17.1, charset-normalizer@==3.4.1, chumpy@==0.70, click@==8.1.8, colorama@==0.4.6, contourpy@==1.3.1, crcmod@==1.7, cryptography@==44.0.1, cycler@==0.12.1, Cython@==3.0.12, decorator@==5.1.1, diffusers@==0.32.2, distro@==1.9.0, fastapi@==0.115.8, ffmpeg-python@==0.2.0, ffmpy@==0.5.0, filelock@==3.14.0, flatbuffers@==25.2.10, fonttools@==4.56.0, fsspec@==2025.2.0, future@==1.0.0, gast@==0.6.0, gdown@==5.2.0, google-api-core@==2.24.1, google-api-python-client@==2.161.0, google-auth@==2.38.0, google-auth-httplib2@==0.2.0, google-auth-oauthlib@==1.2.1, google-pasta@==0.2.0, googleapis-common-protos@==1.67.0, gradio@==5.16.0, gradio_client@==1.7.0, grpcio@==1.70.0, h11@==0.14.0, h5py@==3.12.1, httpcore@==1.0.7, httplib2@==0.22.0, httpx@==0.28.1, huggingface-hub@==0.28.1, idna@==3.10, imageio@==2.37.0, imageio-ffmpeg@==0.6.0, importlib_metadata@==8.6.1, Jinja2@==3.1.5, jmespath@==0.10.0, json-tricks@==3.17.3, keras@==3.8.0, kiwisolver@==1.4.8, libclang@==18.1.1, lumaai@==1.4.0, Markdown@==3.7, markdown-it-py@==3.0.0, MarkupSafe@==2.1.5, matplotlib@==3.10.0, mdurl@==0.1.2, ml-dtypes@==0.3.2, mmcv@==2.1.0, mmdet@==3.2.0, mmengine@==0.10.6, mmpose@==1.3.2, model-index@==0.1.11, moviepy@==2.1.2, mpmath@==1.3.0, munkres@==1.1.4, namex@==0.0.8, networkx@==3.4.2, numpy@==1.26.4, oauthlib@==3.2.2, omegaconf@==2.3.0, opencv-python@==4.9.0.80, opendatalab@==0.0.10, openmim@==0.3.9, openxlab@==0.1.2, opt_einsum@==3.4.0, optree@==0.14.0, ordered-set@==4.1.0, orjson@==3.10.15, oss2@==2.17.0, packaging@==24.2, pandas@==2.2.3, pillow@==10.4.0, platformdirs@==4.3.6, proglog@==0.1.10, progress@==1.6, proto-plus@==1.26.0, protobuf@==4.25.6, psutil@==5.9.8, pyasn1@==0.6.1, pyasn1_modules@==0.4.1, pycocotools@==2.0.8, pycparser@==2.22, pycryptodome@==3.21.0, pydantic@==2.10.6, pydantic_core@==2.27.2, pydub@==0.25.1, Pygments@==2.19.1, pyparsing@==3.2.1, PySocks@==1.7.1, python-dateutil@==2.9.0.post0, python-dotenv@==1.0.1, python-multipart@==0.0.20, pytz@==2023.4, PyYAML@==6.0.2, regex@==2024.11.6, requests@==2.28.2, requests-oauthlib@==2.0.0, rich@==13.4.2, rsa@==4.9, ruff@==0.9.6, safehttpx@==0.1.6, safetensors@==0.5.2, scipy@==1.15.1, semantic-version@==2.10.0, setuptools@==60.2.0, shapely@==2.0.7, shellingham@==1.5.4, six@==1.17.0, sniffio@==1.3.1, soundfile@==0.12.1, soupsieve@==2.6, spaces@==0.32.0, starlette@==0.45.3, static_ffmpeg@==2.8, sympy@==1.13.3, tabulate@==0.9.0, tensorboard@==2.16.0, tensorboard-data-server@==0.7.2, tensorflow@==2.16.1, termcolor@==2.5.0, terminaltables@==3.1.10, tf_keras-nightly@==2.16.0.dev2024021410, tokenizers@==0.21.0, tomlkit@==0.13.2, torch@==2.2.0, torchaudio@==2.2.0, torchvision@==0.17.0, tqdm@==4.65.2, transformers@==4.48.3, typer@==0.15.1, typing_extensions@==4.12.2, tzdata@==2025.1, uritemplate@==4.1.1, urllib3@==1.26.20, uvicorn@==0.34.0, websockets@==14.2, Werkzeug@==3.1.3, wheel@==0.45.1, wrapt@==1.17.2, xtcocotools@==1.14.3, yapf@==0.43.0, zipp@==3.21.0 ### Recent commits (newest first) - Merge branch 'main' of https://github.com/r-agni/posthuman - finished readme - crypto - Update README.md - Update README.md - Update README.md - Merge pull request #3 from r-agni/scrapybara - Merge pull request #2 from r-agni/va - finished image inclusion - final commit maybe - added open ai crypto tool - integrated scrapybara - router complete - fixed chatting - first va commit - removed requirements.txt - minor bug fixes - Merge pull request #1 from r-agni/assets - landing page update - updating code ## Key source files (fetched from GitHub, selected and truncated for size) ### package.json ``` { "name": "gravestone", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "@emotion/react": "^11.14.0", "@emotion/styled": "^11.14.0", "@mui/material": "^6.4.4", "@radix-ui/react-checkbox": "^1.1.4", "@radix-ui/react-dialog": "^1.1.6", "@radix-ui/react-label": "^2.1.2", "@radix-ui/react-progress": "^1.1.2", "@radix-ui/react-select": "^2.1.6", "@radix-ui/react-separator": "^1.1.2", "@radix-ui/react-slot": "^1.1.2", "@radix-ui/react-tabs": "^1.1.3", "@radix-ui/react-toast": "^1.2.6", "@radix-ui/react-tooltip": "^1.1.8", "@react-three/drei": "^9.121.5", "@react-three/fiber": "^8.17.14", "@types/pdfjs-dist": "^2.10.378", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "date-fns": "^3.6.0", "framer-motion": "^12.4.3", "lucide-react": "^0.475.0", "next": "^15.1.7", "node-cron": "^3.0.3", "pdfjs-dist": "^4.10.38", "react": "^18.3.1", "react-day-picker": "^8.10.1", "react-dom": "^18.3.1", "react-toastify": "^11.0.3", "scrapybara": "^2.2.2", "shadcn-ui": "^0.9.4", "tailwind-merge": "^3.0.1", "tailwindcss-animate": "^1.0.7", "tesseract.js": "^6.0.0", "zustand": "^5.0.3" }, "devDependencies": { "@eslint/eslintrc": "^3", "@types/node": "^20", "@types/node-cron": "^3.0.11", "@types/react": "^19", "@types/react-dom": "^19", "eslint": "^9", "eslint-config-next": "15.1.7", "postcss": "^8", "tailwindcss": "^3.4.1", "typescript": "^5" } } ``` ### ai-models/requirements.txt ``` annotated-types==0.7.0 anthropic==0.39.0 anyio==4.8.0 blinker==1.9.0 certifi==2025.1.31 charset-normalizer==3.4.1 click==8.1.8 distro==1.9.0 elastic-transport==8.17.0 elasticsearch==8.17.1 filelock==3.17.0 Flask==3.1.0 fsspec==2025.2.0 greenlet==3.1.1 h11==0.14.0 httpcore==1.0.7 httpx==0.28.1 huggingface-hub==0.28.1 idna==3.10 itsdangerous==2.2.0 Jinja2==3.1.5 jiter==0.8.2 joblib==1.4.2 MarkupSafe==3.0.2 mpmath==1.3.0 networkx==3.4.2 numpy==2.2.3 ollama==0.4.7 packaging==24.2 pillow==11.1.0 playwright==1.50.0 pydantic==2.10.6 pydantic_core==2.27.2 pyee==12.1.1 PyYAML==6.0.2 regex==2024.11.6 requests==2.32.3 safetensors==0.5.2 scikit-learn==1.6.1 scipy==1.15.1 scrapybara==2.2.7 sentence-transformers==3.4.1 sniffio==1.3.1 sympy==1.13.1 threadpoolctl==3.5.0 tokenizers==0.21.0 torch==2.6.0 tqdm==4.67.1 transformers==4.48.3 typing_extensions==4.12.2 urllib3==2.3.0 Werkzeug==3.1.3 ``` ### server/MuseTalk/requirements.txt ``` absl-py==2.1.0 accelerate==0.28.0 addict==2.4.0 aiofiles==23.2.1 aliyun-python-sdk-core==2.16.0 aliyun-python-sdk-kms==2.16.5 annotated-types==0.7.0 antlr4-python3-runtime==4.9.3 anyio==4.8.0 astunparse==1.6.3 beautifulsoup4==4.13.3 cachetools==5.5.1 certifi==2025.1.31 cffi==1.17.1 charset-normalizer==3.4.1 chumpy==0.70 click==8.1.8 colorama==0.4.6 contourpy==1.3.1 crcmod==1.7 cryptography==44.0.1 cycler==0.12.1 Cython==3.0.12 decorator==5.1.1 diffusers==0.32.2 distro==1.9.0 fastapi==0.115.8 ffmpeg-python==0.2.0 ffmpy==0.5.0 filelock==3.14.0 flatbuffers==25.2.10 fonttools==4.56.0 fsspec==2025.2.0 future==1.0.0 gast==0.6.0 gdown==5.2.0 google-api-core==2.24.1 google-api-python-client==2.161.0 google-auth==2.38.0 google-auth-httplib2==0.2.0 google-auth-oauthlib==1.2.1 google-pasta==0.2.0 googleapis-common-protos==1.67.0 gradio==5.16.0 gradio_client==1.7.0 grpcio==1.70.0 h11==0.14.0 h5py==3.12.1 httpcore==1.0.7 httplib2==0.22.0 httpx==0.28.1 huggingface-hub==0.28.1 idna==3.10 imageio==2.37.0 imageio-ffmpeg==0.6.0 importlib_metadata==8.6.1 Jinja2==3.1.5 jmespath==0.10.0 json-tricks==3.17.3 keras==3.8.0 kiwisolver==1.4.8 libclang==18.1.1 lumaai==1.4.0 Markdown==3.7 markdown-it-py==3.0.0 MarkupSafe==2.1.5 matplotlib==3.10.0 mdurl==0.1.2 ml-dtypes==0.3.2 mmcv==2.1.0 mmdet==3.2.0 mmengine==0.10.6 mmpose==1.3.2 model-index==0.1.11 moviepy==2.1.2 mpmath==1.3.0 munkres==1.1.4 namex==0.0.8 networkx==3.4.2 numpy==1.26.4 oauthlib==3.2.2 omegaconf==2.3.0 opencv-python==4.9.0.80 opendatalab==0.0.10 openmim==0.3.9 openxlab==0.1.2 opt_einsum==3.4.0 optree==0.14.0 ordered-set==4.1.0 orjson==3.10.15 oss2==2.17.0 packaging==24.2 pandas==2.2.3 pillow==10.4.0 platformdirs==4.3.6 proglog==0.1.10 progress==1.6 proto-plus==1.26.0 protobuf==4.25.6 psutil==5.9.8 pyasn1==0.6.1 pyasn1_modules==0.4.1 pycocotools==2.0.8 pycparser==2.22 pycryptodome==3.21.0 pydantic==2.10.6 pydantic_core==2.27.2 pydub==0.25.1 Pygments==2.19.1 pyparsing==3.2.1 PySocks==1.7.1 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 python-multipart==0.0.20 pytz==2023.4 PyYAML==6.0.2 regex==2024.11.6 requests==2.28.2 requests-oauthlib==2.0.0 rich==13.4.2 rsa==4.9 ruff==0.9.6 safehttpx==0.1.6 safetensors==0.5.2 scipy==1.15.1 semantic-version==2.10.0 setuptools==60.2.0 shapely==2.0.7 shellingham==1.5.4 six==1.17.0 sniffio==1.3.1 soundfile==0.12.1 soupsieve==2.6 spaces==0.32.0 starlette==0.45.3 static_ffmpeg==2.8 sympy==1.13.3 tabulate==0.9.0 tensorboard==2.16.0 tensorboard-data-server==0.7.2 tensorflow==2.16.1 termcolor==2.5.0 terminaltables==3.1.10 tf_keras-nightly==2.16.0.dev2024021410 tokenizers==0.21.0 tomlkit==0.13.2 torch==2.2.0 torchaudio==2.2.0 torchvision==0.17.0 tqdm==4.65.2 transformers==4.48.3 typer==0.15.1 typing_extensions==4.12.2 tzdata==2025.1 uritemplate==4.1.1 urllib3==1.26.20 uvicorn==0.34.0 websockets==14.2 Werkzeug==3.1.3 wheel==0.45.1 wrapt==1.17.2 xtcocotools==1.14.3 yapf==0.43.0 zipp==3.21.0 ``` ### src/app/layout.tsx ```typescript import type { Metadata } from "next"; import "./globals.css"; export const metadata: Metadata = { title: "Posthuman", description: "App for posthumanism", icons: { icon: "/logo.png", }, }; export default function RootLayout({ children, }: Readonly<{ children: React.ReactNode; }>) { return ( {children} ); } ``` ### ai-models/scrapybara/index.tsx ```typescript import { ScrapybaraClient } from "scrapybara"; import { anthropic } from "scrapybara/anthropic"; import { UBUNTU_SYSTEM_PROMPT } from "scrapybara/prompts"; import { bashTool, computerTool, editTool } from "scrapybara/tools"; import { z } from "zod"; interface Step { text: string; } export async function main() { // Initialize client with API key from environment variable const client = new ScrapybaraClient({ apiKey: process.env.NEXT_PUBLIC_SCRAPYBARA_API_KEY || "scrapy-ca6aafb8-49aa-4ecf-977d-539ea7e76a58", }); const instance = await client.startUbuntu(); try { const streamUrl = await instance.getStreamUrl(); console.log("Stream URL:", streamUrl); const { output } = await client.act({ model: anthropic(), tools: [bashTool(instance), computerTool(instance), editTool(instance)], system: UBUNTU_SYSTEM_PROMPT, prompt: "Visit the CNN main page at cnn.com . Agree to any terms of service and click the WHITE sign in button to the right of the RED subscribe button on the main page. Then login using the following credentials. myposthuman@gmail.commyposthuman@1234 Then go to https://www.cnn.com/account/settings . Scroll down to the bottom of the page then click delete account, all the way at the bottom, and then click delete account on the popup. The goal is to delete the CNN account, follow the necessary steps until you finally delete your account. if this doesnt work, try going directly to the login url and then deleting the account from there. You dont need to ask me any permissions you can act on my behalf for all tasks. Remember to scroll through the entirety of each page, to get in its full context. Do not click out of CNN, do not go to HBO, stay on CNN", schema: z.object({ posts: z.array( z.object({ title: z.string(), url: z.string(), points: z.number(), }) ), }), onStep: (step: Step) => console.log(step.text), }); return output?.posts; } catch (error) { console.error("Error during Scrapybara operation:", error); throw error; // Rethrow or handle as needed } finally { await instance.stop(); } } ``` ### src/app/page.tsx ```typescript "use client"; import { Button } from "@/components/ui/button"; import Link from "next/link"; import Image from "next/image"; import { useRouter } from "next/navigation"; import { motion } from "framer-motion"; export default function Page() { const router = useRouter(); return (

Your Legacy Management Agent

Posthumous asset management, will generation, and social memory preservation.

); } ``` ### server/MuseTalk/main.py ```python import os import sys from static_ffmpeg import run import argparse from omegaconf import OmegaConf import torch from dotenv import load_dotenv from utils import send_email_with_attachment, get_duration_wave ffmpeg_path, _ = run.get_or_fetch_platform_executables_else_raise() os.environ['FFMPEG_PATH'] = ffmpeg_path current_dir = os.path.dirname(os.path.abspath(__file__)) musetalk_dir = os.path.join(current_dir, 'MuseTalk') sys.path.extend([current_dir, musetalk_dir]) from musetalk.utils.utils import load_all_model from scripts.inference import main as musetalk_inference from luma import generate_video, generate_double_video, generate_triple_video from elevenlabs import create_instant_voice_clone, generate_audio # Load model weights audio_processor, vae, unet, pe = load_all_model() device = torch.device("cuda" if torch.cuda.is_available() else "cpu") timesteps = torch.tensor([0], device=device) def run_musetalk(video_path, audio_path, result_dir='./results', bbox_shift=0, fps=25, batch_size=8): # temporary config file config = { 'task_0': { 'video_path': video_path, 'audio_path': audio_path, 'bbox_shift': bbox_shift } } config_path = 'temp_config.yaml' with open(config_path, 'w') as f: OmegaConf.save(config=config, f=f) # Set up arguments for MuseTalk inference args = argparse.Namespace( inference_config=config_path, bbox_shift=bbox_shift, result_dir=result_dir, fps=fps, batch_size=batch_size, output_vid_name=None, use_saved_coord=False, use_float16=False ) # Run MuseTalk inference musetalk_inference(args) # Clean up temporary config file os.remove(config_path) # Return the path of the generated video output_basename = f"{os.path.basename(video_path).split('.')[0]}_{os.path.basename(audio_path).split('.')[0]}" return os.path.join(result_dir, f"{output_basename}.mp4") def main(image_path, voice_sample_path, text, image_prompt, luma_api_key, elevenlabs_api_key, recipient_email, result_dir='./results'): voice_id = "CkVOwuK94BHPOuKUR76m" if voice_id: audio_path = generate_audio(voice_id, text, elevenlabs_api_key) audio_length = get_duration_wave(audio_path) else: raise Exception("Voice cloning failed") # video_path = generate_video(image_path, image_prompt, luma_api_key) if (audio_length / 5) <= 1.0: video_path = generate_video(image_path, image_prompt, luma_api_key) elif (audio_length / 5) <= 2.0: video_path = generate_double_video(image_path, image_prompt, luma_api_key) else: video_path = generate_triple_video(image_path, image_prompt, luma_api_key) result_path = run_musetalk(video_path, audio_path, result_dir, bbox_shift=-10) if recipient_email: subject = "Posthuman - A Message from Someone Close to You" body = "Please find attached the video." send_email_with_attachment(recipient_email, subject, body, result_path) return result_path if __name__ == "__main__": load_dotenv(dotenv_path="/Users/shivanshsoni/Desktop/posthuman/server/MuseTalk/.env") luma_api_key = os.getenv("LUMAAI_API_KEY") elabs_api_key = os.getenv("ELEVENLABS_API_KEY") image_path = "/Users/shivanshsoni/Documents/GitHub/posthuman/server/assets/jeffreyTwo.JPG" voice_sample_path = "/Users/shivanshsoni/Desktop/posthuman/server/jeffrey_audio/sample.mp3" text = "Happy Birthday Son. I love you so much. I hope you have a great day." image_prompt = ("Still shot of person's face showing an expression of love and care. Their body is not moving. Face very slightly.") result_dir = "./results" recipient_email = "shivanshmsoni@gmail.com" result = main(image_path, voice_sample_path, text, image_prompt, luma_api_key, elabs_api_key, recipient_email, result_dir) print(f"MuseTalk inference completed. Final output video saved at: {result}") ``` ### server/MuseTalk/app.py ```python import os import time import pdb import re import gradio as gr import spaces import numpy as np import sys import subprocess from huggingface_hub import snapshot_download import requests import argparse import os from omegaconf import OmegaConf import numpy as np import cv2 import torch import glob import pickle from tqdm import tqdm import copy from argparse import Namespace import shutil import gdown import imageio import ffmpeg from moviepy.editor import * ProjectDir = os.path.abspath(os.path.dirname(__file__)) CheckpointsDir = os.path.join(ProjectDir, "models") def print_directory_contents(path): for child in os.listdir(path): child_path = os.path.join(path, child) if os.path.isdir(child_path): print(child_path) def download_model(): if not os.path.exists(CheckpointsDir): os.makedirs(CheckpointsDir) print("Checkpoint Not Downloaded, start downloading...") tic = time.time() snapshot_download( repo_id="TMElyralab/MuseTalk", local_dir=CheckpointsDir, max_workers=8, local_dir_use_symlinks=True, force_download=True, resume_download=False ) # weight os.makedirs(f"{CheckpointsDir}/sd-vae-ft-mse/") snapshot_download( repo_id="stabilityai/sd-vae-ft-mse", local_dir=CheckpointsDir+'/sd-vae-ft-mse', max_workers=8, local_dir_use_symlinks=True, force_download=True, resume_download=False ) #dwpose os.makedirs(f"{CheckpointsDir}/dwpose/") snapshot_download( repo_id="yzd-v/DWPose", local_dir=CheckpointsDir+'/dwpose', max_workers=8, local_dir_use_symlinks=True, force_download=True, resume_download=False ) #vae url = "https://openaipublic.azureedge.net/main/whisper/models/65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9/tiny.pt" response = requests.get(url) # 确保请求成功 if response.status_code == 200: # 指定文件保存的位置 file_path = f"{CheckpointsDir}/whisper/tiny.pt" os.makedirs(f"{CheckpointsDir}/whisper/") # 将文件内容写入指定位置 with open(file_path, "wb") as f: f.write(response.content) else: print(f"请求失败，状态码：{response.status_code}") #gdown face parse url = "https://drive.google.com/uc?id=154JgKpzCPW82qINcVieuPH3fZ2e0P812" os.makedirs(f"{CheckpointsDir}/face-parse-bisent/") file_path = f"{CheckpointsDir}/face-parse-bisent/79999_iter.pth" gdown.download(url, file_path, quiet=False) #resnet url = "https://download.pytorch.org/models/resnet18-5c106cde.pth" response = requests.get(url) # 确保请求成功 if response.status_code == 200: # 指定文件保存的位置 file_path = f"{CheckpointsDir}/face-parse-bisent/resnet18-5c106cde.pth" # 将文件内容写入指定位置 with open(file_path, "wb") as f: f.write(response.content) else: print(f"请求失败，状态码：{response.status_code}") toc = time.time() print(f"download cost {toc-tic} seconds") print_directory_contents(CheckpointsDir) else: print("Already download the model.") download_model() # for huggingface deployment. from musetalk.utils.utils import get_file_type,get_video_fps,datagen from musetalk.utils.preprocessing import get_landmark_and_bbox,read_imgs,coord_placeholder,get_bbox_range from musetalk.utils.blending import get_image from musetalk.utils.utils import load_all_model @spaces.GPU(duration=600) @torch.no_grad() def inference(audio_path,video_path,bbox_shift,progress=gr.Progress(track_tqdm=True)): args_dict={"result_dir":'./results/output', "fps":25, "batch_size":8, "output_vid_name":'', "use_saved_coord":False}#same with inferenece script args = Namespace(**args_dict) input_basename = os.path.basename(video_path).split('.')[0] audio_basename = os.path.basename(audio_path).split('.')[0] output_basename = f"{input_basename}_{audio_basename}" result_img_save_path = os.path.join(args.result_dir, output_basename) # related to video & audio inputs crop_coord_save_path = os.path.join(result_img_save_path, input_basename+".pkl") # only related to video input os.makedirs(result_img_save_path,exist_ok =True) if args.output_vid_name=="": output_vid_name = os.path.join(args.result_dir, output_basename+".mp4") else: output_vid_name = os.path.join(args.result_dir, args.output_vid_name) ############################################## extract frames from source video ############################################## if get_file_type(video_path)=="video": save_dir_full = os.path.join(args.result_dir, input_basename) os.makedirs(save_dir_full,exist_ok = True) # cmd = f"ffmpeg -v fatal -i {video_path} -start_number 0 {save_dir_full}/%08d.png" # os.system(cmd) # 读取视频 reader = imageio.get_reader(video_path) # 保存图片 for i, im in enumerate(reader): imageio.imwrite(f"{save_dir_full}/{i:08d}.png", im) input_img_list = sorted(glob.glob(os.path.join(save_dir_full, '*.[jpJP][pnPN]*[gG]'))) fps = get_video_fps(video_path) else: # input img folder input_img_list = glob.glob(os.path.join(video_path, '*.[jpJP][pnPN]*[gG]')) input_img_list = sorted(input_img_list, key=lambda x: int(os.path.splitext(os.path.basename(x))[0])) fps = args.fps #print(input_img_list) ############################################## extract audio feature ############################################## whisper_feature = audio_processor.audio2feat(audio_path) whisper_chunks = audio_processor.feature2chunks(feature_array=whisper_feature,fps=fps) ############################################## pr [truncated — 9971 more characters] ``` ### src/app/(home)/layout.tsx ```typescript "use client"; import { Sidebar } from "@/components/sidebar"; import { useKeepCronAlive } from "@/hooks/cronjob"; export default function SecondaryLayout({ children, }: Readonly<{ children: React.ReactNode; }>) { useKeepCronAlive(); return (
{children}
); } ``` ### src/app/api/scrapybara/route.ts ```typescript import { NextResponse } from "next/server"; import { main as runScrapyInstance } from "ai-models/scrapybara/index"; // Remove the @ prefix export async function GET() { try { const data = await runScrapyInstance(); return NextResponse.json({ data }); } catch (error: any) { console.error("Scrapybara API error:", error); return NextResponse.json({ error: error.message }, { status: 500 }); } } ``` [81 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]