# Project export: mandatory apple redistribution

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 12.0
- Tagline: A for funsies hack that intercepts video rendering calls made by any process, replaces displayed frames with Bad Apple (a meme to get this music video to play on anything, now you can only play it)
- Devpost: https://devpost.com/software/gpu-hijack
- GitHub: https://github.com/emmettlsc/mandatory-apple-redistribution
- Video: https://www.youtube.com/embed/g1PGuWuTxwA?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — Emmett Cocke (11 commits)

## Devpost submission (written by the team)

### Inspiration

We wanted to hack something pointless but technically cool - like getting doom to run on a calculator. A few popular libs do all the heavy lifting for video but no one messes with them directly. So we thought: what if we could hijack any video and force it to play Bad Apple instead (ie imagine you go to Canvas to watch the latest recorded lecture drop for the class you keep sleeping through, you click on the video and it seems like your professor uploaded Bad Apple?? Then you go to YouTube to watch the latest KRAZAM drop, an ad pops up and it's Bad Apple, you think "what are the chances of that happening" then you click skip ad and it seems like KRAZAM uploaded a music video too????). This is also a less popular meme than the "play doom on X" but if you search youtube you will find hundreds of videos of playing bad apple on X (where X is something like a pregnancy test or an apple), well now you have no choice to play bad apple on everything

### What it does

Hooks into ffmpeg, the video decoding lib that pretty much everything uses - chrome, firefox, vlc, you name it. We intercept the video data right as it's about to be decoded and swap in bad apple frames instead. Works with ffplay, vlc, and nonsandboxed browsers - basically anything using ffmpeg (which is most things) becomes a bad apple player whether it wants to or not

### How we built it

A target application (like ffplay) requests the gpu to render a frame. Our injected shared library intercepts that call before it reaches the gpu. We swap or modify the video data on the fly. The result is displayed as if nothing unusual happened

### Challenges we ran into

Browser security issues, finagling browser configs to choose to use ffmpeg decoding libs, some segfault-ing, finding the common denominator of video rendering calls

### Accomplishments we're proud of

Getting bad apple to play over any ffmpeg-based video player without modifying the actual player

### What we learned

Browser security features are annoying

### What's next

Originally we thought this project only had meme value but after working on it we think it would be way cooler to make the system more seamless and configurable and dare I say useful. We plan to replace the current LD_PRELOAD approach with an eBPF-based hook, allowing the interception to work transparently across different applications without manual setup The potential we see is from adding features to any video player regardless of what they support. Stuff like this pops into our minds: dynamic overlays: layering content in real time on top of existing video streams like a subway surfer vid to look at while watching lectures real time captioning: intercepting audio, transcribing it using Whisper, and rendering subtitles directly into the video output universal compatibility: identifying common gpu/va-api and ffmpeg call paths to support the vast majority of browsers, media players, and streaming apps out-of-the-box library mode: turning this into a lightweight and configurable c++ lib for creative or accessibility focused video manipulation that is system wide ==> we just think enabling developers to programmatically remix, annotate, and transform visual streams system wide in flight is a cool idea

## README (from the GitHub repository)

## Mandatory Apple Redistribution 

A for funsies hack that intercepts video rendering calls made by any process, replaces displayed frames with Bad Apple (a meme to get this music video to play on anything, now you can only play it)

## Demo
https://youtu.be/g1PGuWuTxwA

## Inspiration
We wanted to hack something pointless but technically cool - like getting doom to run on a calculator. A few popular libs do all the heavy lifting for video but no one messes with them directly. So we thought: what if we could hijack any video and force it to play Bad Apple instead (ie imagine you go to Canvas to watch the latest recorded lecture drop for the class you keep sleeping through, you click on the video and it seems like your professor uploaded Bad Apple?? Then you go to YouTube to watch the latest KRAZAM drop, an ad pops up and it's Bad Apple, you think "what are the chances of that happening" then you click skip ad and it seems like KRAZAM uploaded a music video too????). This is also a less popular meme than the "play doom on X" but if you search youtube you will find hundreds of videos of playing bad apple on X (where X is something like a pregnancy test or an apple), well now you have no choice to play bad apple on everything

## What it does
Hooks into ffmpeg, the video decoding lib that pretty much everything uses - chrome, firefox, vlc, you name it. We intercept the video data right as it's about to be decoded and swap in bad apple frames instead. Works with ffplay, vlc, and nonsandboxed browsers - basically anything using ffmpeg (which is most things) becomes a bad apple player whether it wants to or not

## How we built it
A target application (like ffplay) requests the gpu to render a frame. Our injected shared library intercepts that call before it reaches the gpu. We swap or modify the video data on the fly. The result is displayed as if nothing unusual happened

## Challenges we ran into
Browser security issues, finagling browser configs to choose to use ffmpeg decoding libs, some segfault-ing, finding the common denominator of video rendering calls 

## Accomplishments that we're proud of
Getting bad apple to play over any ffmpeg-based video player without modifying the actual player

## What we learned
Browser security features are annoying

## What's next
Originally we thought this project only had meme value but after working on it we think it would be way cooler to make the system more seamless and configurable and dare I say useful. We plan to replace the current LD_PRELOAD approach with an eBPF-based hook, allowing the interception to work transparently across different applications without manual setup
 
The potential we see is from adding features to any video player regardless of what they support. Stuff like this pops into our minds: 
- dynamic overlays: layering content in real time on top of existing video streams like a subway surfer vid to look at while watching lectures
- real time captioning: intercepting audio, transcribing it using Whisper, and rendering subtitles directly into the video output
- universal compatibility: identifying common gpu/va-api and ffmpeg call paths to support the vast majority of browsers, media players, and streaming apps out-of-the-box
- library mode: turning this into a lightweight and configurable c++ lib for creative or accessibility focused video manipulation that is system wide

==> we just think enabling developers to programmatically remix, annotate, and transform visual streams system wide in flight is a cool idea


## Detected evidence (automated analysis)

Indexed codebase: 3 recognized source files, 17 KB.
- C (language) — detected in the code
- C++ (language) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (6 of 6)

```
.gitignore
ffmpeg_hook.c
Makefile
notes.txt
README.md
vaapi-logger.c
```

### Dependencies

No dependency index available.

### Recent commits (newest first)

- demo link
- Revise header for Mandatory Apple Redistribution
- readme
- Merge pull request #1 from emmettlsc/ffmpeg-hook
- segfault fixed - hoping this works for bad apple too
- attempt at fixing segfault...
- debugging
- static frame attempt
- added ffmpeg hook - pls work
- removed emacs artifact
- update to get rid of emacs artifacts
- adding makefile for logging testing
- adding basic logging to libav hook
- .
- init

## Key source files (fetched from GitHub, selected and truncated for size)

### ffmpeg_hook.c

```c
#define _GNU_SOURCE
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>

// real AVFrame layout from ffmpeg
// dont use the typedef, just access the fields we need by offset
typedef struct {
    uint8_t *data[8];
    int linesize[8];
    // extended_data comes here
    uint8_t **extended_data;
    int width, height;
    int nb_samples;
    int format;
    // more fields but we dont care
} AVFrame;

// counters
static int packet_count = 0;
static int frame_count = 0;
static int replaced_count = 0;

// bad apple info
// found the vid on gh, forgot the link...
static uint8_t* bad_apple_data = NULL;
static size_t bad_apple_size = 0;
static int bad_apple_width = 640;
static int bad_apple_height = 480;
static int bad_apple_frame_size = 0;
static int bad_apple_num_frames = 0;

// function pointers to real functions
static int (*real_avcodec_send_packet)(void*, void*) = NULL;
static int (*real_avcodec_receive_frame)(void*, void*) = NULL;

// hook avcodec_send_packet -> called when compressed video packet is sent to decoder
int avcodec_send_packet(void* avctx, void* avpkt) {
    if (!real_avcodec_send_packet) {
        real_avcodec_send_packet = dlsym(RTLD_NEXT, "avcodec_send_packet");
        if (!real_avcodec_send_packet) {
            fprintf(stderr, "failed to find real avcodec_send_packet\n");
            return -1;
        }
    }

    packet_count++;
    printf("[ffmpeg hook] packet #%d sent to decoder\n", packet_count);

    return real_avcodec_send_packet(avctx, avpkt);
}

// load bad apple frames
void load_bad_apple_frames() {
    FILE* f = fopen("./badapple/badapple_raw.yuv", "rb");
    if (!f) {
        printf("[ERROR] cant open badapple_raw.yuv\n");
        return;
    }

    fseek(f, 0, SEEK_END);
    bad_apple_size = ftell(f);
    fseek(f, 0, SEEK_SET);

    bad_apple_data = malloc(bad_apple_size);
    if (!bad_apple_data) {
        printf("[ERROR] cant allocate memory for bad apple\n");
        fclose(f);
        return;
    }

    size_t read = fread(bad_apple_data, 1, bad_apple_size, f);
    fclose(f);

    // yuv420p frame size = width * height * 1.5
    bad_apple_frame_size = bad_apple_width * bad_apple_height * 3 / 2;
    bad_apple_num_frames = bad_apple_size / bad_apple_frame_size;

    printf("[BAD APPLE] loaded %zu bytes, %d frames at %dx%d\n",
           bad_apple_size, bad_apple_num_frames, bad_apple_width, bad_apple_height);
}

// replace frame with bad apple frame
void replace_with_bad_apple(AVFrame* frame) {
    if (!frame->data[0] || frame->width <= 0 || frame->height <= 0) {
        return;
    }

    if (!bad_apple_data) {
        printf("[DEBUG] no bad apple data loaded, skipping\n");
        return;
    }

    // cycle through bad apple frames 
    int ba_frame_idx = replaced_count % bad_apple_num_frames;
    uint8_t* ba_frame = bad_apple_data + (ba_frame_idx * bad_apple_frame_size);

    // bad apple is 640x480, source might be different
    // for now just copy what fits
    int copy_width = frame->width < bad_apple_width ? frame->width : bad_apple_width;
    int copy_height = frame->height < bad_apple_height ? frame->height : bad_apple_height;

    // Y plane
    uint8_t* ba_y = ba_frame;
    for (int y = 0; y < copy_height; y++) {
        memcpy(frame->data[0] + y * frame->linesize[0],
               ba_y + y * bad_apple_width,
               copy_width);
    }

    // U plane
    if (frame->data[1]) {
        uint8_t* ba_u = ba_frame + (bad_apple_width * bad_apple_height);
        int uv_copy_width = copy_width / 2;
        int uv_copy_height = copy_height / 2;
        for (int y = 0; y < uv_copy_height; y++) {
            memcpy(frame->data[1] + y * frame->linesize[1],
                   ba_u + y * (bad_apple_width / 2),
                   uv_copy_width);
        }
    }

    // V plane
    if (frame->data[2]) {
        uint8_t* ba_v = ba_frame + (bad_apple_width * bad_apple_height) + (bad_apple_width * bad_apple_height / 4);
        int uv_copy_width = copy_width / 2;
        int uv_copy_height = copy_height / 2;
        for (int y = 0; y < uv_copy_height; y++) {
            memcpy(frame->data[2] + y * frame->linesize[2],
                   ba_v + y * (bad_apple_width / 2),
                   uv_copy_width);
        }
    }
    //YUVVVV
    replaced_count++;
}

// hook avcodec_receive_frame -> called when decoder outputs a frame
int avcodec_receive_frame(void* avctx, void* frame) {
    printf("[HOOK] avcodec_receive_frame called\n");

    if (!real_avcodec_receive_frame) {
        printf("[HOOK] finding real avcodec_receive_frame...\n");
        real_avcodec_receive_frame = dlsym(RTLD_NEXT, "avcodec_receive_frame");
        if (!real_avcodec_receive_frame) {
            fprintf(stderr, "failed to find real avcodec_receive_frame\n");
            return -1;
        }
        printf("[HOOK] found real function at %p\n", real_avcodec_receive_frame);
    }

    printf("[HOOK] calling real avcodec_receive_frame...\n");
    int ret = real_avcodec_receive_frame(avctx, frame);
    printf("[HOOK] real function returned: %d\n", ret);

    if (ret == 0) {
        frame_count++;
        printf("\n=== [HOOK] FRAME #%d DECODED - REPLACING NOW ===\n", frame_count);

        // replace the frame data
        replace_with_bad_apple((AVFrame*)frame);

        printf("=== [HOOK] REPLACEMENT COMPLETE ===\n\n");
    } else {
        printf("[HOOK] ret != 0, not replacing (ret=%d)\n", ret);
    }

    return ret;
}

__attribute__((constructor))
void init() {
    printf("\n=== ffmpeg hook loaded ===\n");
    printf("pid: %d\n", getpid());
    printf("watching avcodec_send_packet and avcodec_receive_frame\n");
    printf("========================\n\n");

    load_bad_apple_frames();
}

__attribute__((destructor))
void cleanup() {
    printf("\n=== ffmpeg hook unloading ===\n");
    printf("packets: %d\n", packet_count);
    printf("frames: %d\n", frame_count);
    printf("frames replaced: %d\n", replaced_count);
    printf("===========
[truncated — 87 more characters]
```

### vaapi-logger.c

```c
#define _GNU_SOURCE
#include <dlfcn.h>
#include <va/va.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <time.h>

// func pointer to original vaRenderPicture
static VAStatus (*real_vaRenderPicture)(VADisplay, VAContextID, VABufferID*, int) = NULL;

// counters for logging
static int render_call_count = 0;
static int total_buffers_seen = 0;

// func for buffer type name
const char* get_buffer_type_name(VABufferType type) {
    switch (type) {
        case VAPictureParameterBufferType: return "PictureParameter";
        case VAIQMatrixBufferType: return "IQMatrix";
        case VABitPlaneBufferType: return "BitPlane";
        case VASliceGroupMapBufferType: return "SliceGroupMap";
        case VASliceParameterBufferType: return "SliceParameter";
        case VASliceDataBufferType: return "SliceData";
        case VAMacroblockParameterBufferType: return "MacroblockParameter";
        case VAResidualDataBufferType: return "ResidualData";
        case VADeblockingParameterBufferType: return "DeblockingParameter";
        case VAImageBufferType: return "Image"; // ??? is this the one we want
        case VAProtectedSliceDataBufferType: return "ProtectedSliceData";
        case VAQMatrixBufferType: return "QMatrix";
        case VAHuffmanTableBufferType: return "HuffmanTable";
        case VAProbabilityBufferType: return "Probability";
        default: return "Unknown";
    }
}

// dump buffer data as hex
void dump_buffer_hex(const void* data, size_t size, const char* prefix) {
    const unsigned char* bytes = (const unsigned char*)data;
    size_t dump_size = size > 64 ? 64 : size;  // Only dump first 64 bytes
    
    printf("%s: ", prefix);
    for (size_t i = 0; i < dump_size; i++) {
        printf("%02x ", bytes[i]);
        if ((i + 1) % 16 == 0) {
            printf("\n%s: ", prefix);
        }
    }
    if (dump_size < size) {
        printf("... (%zu more bytes)", size - dump_size);
    }
    printf("\n");
}

// hooked vaRenderPicture function
VAStatus vaRenderPicture(VADisplay dpy, VAContextID context, 
                        VABufferID *buffers, int num_buffers) {
    
    // get the real function if we still need to
    if (!real_vaRenderPicture) {
        real_vaRenderPicture = dlsym(RTLD_NEXT, "vaRenderPicture");
        if (!real_vaRenderPicture) {
            printf("ERROR: Could not find real vaRenderPicture function!\n");
            return VA_STATUS_ERROR_UNIMPLEMENTED;
        }
    }
    
    // log the call
    render_call_count++;
    total_buffers_seen += num_buffers;
    
    printf("\n=== VA-API HOOK: vaRenderPicture Call #%d ===\n", render_call_count);
    printf("Display: %p, Context: %d, Buffers: %d\n", dpy, context, num_buffers);
    
    // examine each buffer
    for (int i = 0; i < num_buffers; i++) {
        VABufferID buffer_id = buffers[i];
        printf("\n--- Buffer %d/%d (ID: %d) ---\n", i + 1, num_buffers, buffer_id);
        
        // get buffer info
        VABufferType buffer_type;
        unsigned int buffer_size;
        
        VAStatus info_status = vaBufferInfo(dpy, buffer_id, &buffer_type, &buffer_size);
        if (info_status != VA_STATUS_SUCCESS) {
            printf("WARNING: Could not get buffer info (status: %d)\n", info_status);
            continue;
        }
        
        printf("Type: %s (%d), Size: %u bytes\n", 
               get_buffer_type_name(buffer_type), buffer_type, buffer_size);
        
        // map buffer to read its contents
        void* buffer_data = NULL;
        VAStatus map_status = vaMapBuffer(dpy, buffer_id, &buffer_data);
        if (map_status != VA_STATUS_SUCCESS) {
            printf("WARNING: Could not map buffer (status: %d)\n", map_status);
            continue;
        }
        
        if (buffer_data && buffer_size > 0) {
            // Log some interesting details based on buffer type
            if (buffer_type == VASliceDataBufferType) {
                printf("*** SLICE DATA BUFFER (compressed video data) ***\n");
                dump_buffer_hex(buffer_data, buffer_size, "DATA");
                
                // Look for common video codec signatures
                unsigned char* data = (unsigned char*)buffer_data;
                if (buffer_size >= 4) {
                    if (data[0] == 0x00 && data[1] == 0x00 && data[2] == 0x00 && data[3] == 0x01) {
                        printf("DETECTED: H.264 NAL unit start code\n");
                    } else if (data[0] == 0x00 && data[1] == 0x00 && data[2] == 0x01) {
                        printf("DETECTED: H.264 short start code\n");
                    }
                }
            } else if (buffer_type == VASliceParameterBufferType) {
                printf("*** SLICE PARAMETER BUFFER (decode instructions) ***\n");
                dump_buffer_hex(buffer_data, buffer_size, "PARAM");
            } else if (buffer_type == VAPictureParameterBufferType) {
                printf("*** PICTURE PARAMETER BUFFER (frame info) ***\n");
                dump_buffer_hex(buffer_data, buffer_size, "PIC");
            } else {
                printf("Buffer contents (first 32 bytes):\n");
                dump_buffer_hex(buffer_data, buffer_size > 32 ? 32 : buffer_size, "MISC");
            }
        }
        
        // Unmap the buffer
        vaUnmapBuffer(dpy, buffer_id);
    }
    
    printf("\n=== Calling real vaRenderPicture ===\n");
    
    // Call the real function
    VAStatus result = real_vaRenderPicture(dpy, context, buffers, num_buffers);
    
    printf("Real vaRenderPicture returned: %d\n", result);
    printf("=== End of vaRenderPicture Call #%d ===\n\n", render_call_count);
    
    return result;
}

// Optional: Hook other interesting VA-API functions for context
VAStatus vaCreateContext(VADisplay dpy, VAConfigID config_id, int picture_width,
                        int picture_height, int flag, VASurfaceID *render_targets,
                        int num_render_targets, VAContextID *context) {
[truncated — 1894 more characters]
```