# Project export: Parrot on your Hand

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2024
- Tagline: Steer and Control your Parrot Anafi Drone JUST by Using your Apple watch and Hand Gestures!
- Devpost: https://devpost.com/software/parrot-on-your-hand
- GitHub: https://github.com/Aneeshers/ParrotOnYourHand
- Video: https://www.youtube.com/embed/ArSJQXZwJDI?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — Aneesh Muppidi (11 commits)

## Devpost submission (written by the team)

### Overview

Ever imagine controlling your Parrot drone from just the gestures of your hand? Parrot on Your Hand is a Remote Controller for the Anafi Parrot Drone using Your Apple Watch. A Stanford Treehacks 2024 🌳 Project! Video Demonstration YouTube Link Implementation I use Core Motion from Apple WatchOS to derive and record hand gestures -- rotate right (across pitch axis) to move right, rotate left (across pitch axis) to move left, move hand up (rotate up on roll axis) to move the drone up. Tap the land button to land the drone. I use Core Motion from Apple WatchOS to derive and record hand gestures -- rotate right (across pitch axis) to move right, rotate left (across pitch axis) to move left, move hand up (rotate up on roll axis) to move the drone up. Tap the land button to land the drone. I then route these commands to Firebase using the low-level networking libraries from Apple Watch (note this was particularly difficult as Apple Watch supposedly no longer supports Firebase Realtime Database (link to issue)). I pilot the Parrot drone via the Olympe SDK -- which connects to our Firebase Realtime Database and listens to actions from the Apple Watch. I then route these commands to Firebase using the low-level networking libraries from Apple Watch (note this was particularly difficult as Apple Watch supposedly no longer supports Firebase Realtime Database (link to issue)). I pilot the Parrot drone via the Olympe SDK -- which connects to our Firebase Realtime Database and listens to actions from the Apple Watch. Usage Install Olympe (link to original API reference) Install Firebase_admin (original) and set up a Firebase Realtime Database (tutorial) Get the correct service account JSON file and change here: python cred_path = '' databaseURL = '' Connect the drone to the correct IP: DRONE_IP = "10.202.0.1" Connect the drone to the correct IP: You will need to connect your antenna to the node hosting the Olympe controller, as this connects to the Parrot drone while maintaining an internet connection to the Firebase server. You will need to connect your antenna to the node hosting the Olympe controller, as this connects to the Parrot drone while maintaining an internet connection to the Firebase server. Then build and deploy the WatchOS application (note you cannot do this on a simulator as you need access to a gyroscope). Then build and deploy the WatchOS application (note you cannot do this on a simulator as you need access to a gyroscope). Then run python3 firebaseDrone.py Then run python3 firebaseDrone.py

## README (from the GitHub repository)


# Parrot on Your Hand 🦜👋⌚
*Ever imagine controlling your Parrot drone from just the gestures of your hand?*  
**Parrot on Your Hand is a Remote Controller for the Anafi Parrot Drone using Your Apple Watch.**

*A Stanford Treehacks 2024 🌳 Project!*

## Video Demonstration
[![Video Demonstration](https://s5.ezgif.com/tmp/ezgif-5-a21f821d22.gif)](https://youtu.be/ArSJQXZwJDI)
[YouTube Link](https://youtu.be/ArSJQXZwJDI)
## Implementation
- I use Core Motion from Apple WatchOS to derive and record hand gestures -- rotate right (across pitch axis) to move right, rotate left (across pitch axis) to move left, move hand up (rotate up on roll axis) to move the drone up. Tap the land button to land the drone.

- I then route these commands to Firebase using the low-level networking libraries from Apple Watch (note this was particularly difficult as Apple Watch supposedly no longer supports Firebase Realtime Database ([link to issue](https://firebase.google.com/docs/ios/learn-more))). I pilot the Parrot drone via the Olympe SDK -- which connects to our Firebase Realtime Database and listens to actions from the Apple Watch.

## Usage
1. Install Olympe ([link to original API reference](https://developer.parrot.com/docs/olympe/index.html))
2. Install Firebase_admin ([original](https://pypi.org/project/firebase-admin/)) and set up a Firebase Realtime Database ([tutorial](https://firebase.google.com/docs/database))
3. Get the correct service account JSON file and change here:  
   ```python
   cred_path = ''
   databaseURL = ''
   ```
4. Connect the drone to the correct IP:  
   ```python
   DRONE_IP = "10.202.0.1"
   ```

5. You will need to connect your antenna to the node hosting the Olympe controller, as this connects to the Parrot drone while maintaining an internet connection to the Firebase server.

6. Then build and deploy the WatchOS application (note you cannot do this on a simulator as you need access to a gyroscope).

7. Then run `python3 firebaseDrone.py`
```


## Detected evidence (automated analysis)

Indexed codebase: 10 recognized source files, 13 KB.
- Python (language) — detected in the code
- Swift (language) — detected in the code
- Firebase (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (22 of 22)

```
.gitattributes
firebaseDrone.py
LICENSE
ParrotOnHand.xcodeproj/project.pbxproj
ParrotOnHand.xcodeproj/project.xcworkspace/contents.xcworkspacedata
ParrotOnHand.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist
ParrotOnHand.xcodeproj/project.xcworkspace/xcuserdata/aneesh.xcuserdatad/UserInterfaceState.xcuserstate
ParrotOnHand.xcodeproj/xcuserdata/aneesh.xcuserdatad/xcschemes/xcschememanagement.plist
README.md
socket_ss.py
test4 Watch App/Assets.xcassets/AccentColor.colorset/Contents.json
test4 Watch App/Assets.xcassets/AppIcon.appiconset/Contents.json
test4 Watch App/Assets.xcassets/Contents.json
test4 Watch App/ContentView.swift
test4 Watch App/GyroscopeViewModel.swift
test4 Watch App/oldCode.swift
test4 Watch App/Preview Content/Preview Assets.xcassets/Contents.json
test4 Watch App/test4App.swift
test4 Watch AppTests/test4_Watch_AppTests.swift
test4 Watch AppUITests/test4_Watch_AppUITests.swift
test4 Watch AppUITests/test4_Watch_AppUITestsLaunchTests.swift
watchdrone-dfe04-firebase-adminsdk-4hm2l-a0ec3ab47e.json
```

### Dependencies

No dependency index available.

### Recent commits (newest first)

- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- anony
- Apple watch Firebase compatibilty
- added landing function
- bug fix
- droneKeys
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### socket_ss.py

```python
import asyncio
import websockets

async def echo(websocket, path):
    async for message in websocket:
        print(f"Received message: {message}")
        await websocket.send(f"Echo: {message}")

async def start_server():
    async with websockets.serve(echo, "localhost", 12345):
        await asyncio.Future()  # Run forever

if __name__ == "__main__":
    print("Starting server")
    asyncio.run(start_server())

```

### firebaseDrone.py

```python
import firebase_admin
from firebase_admin import credentials
from firebase_admin import db
import olympe
from olympe.messages.ardrone3.Piloting import TakeOff, moveBy, Landing
from olympe.messages.ardrone3.PilotingState import FlyingStateChanged

DRONE_IP = "10.202.0.1"
drone = olympe.Drone(DRONE_IP)

cred_path = ''
databaseURL = 'https://watchdrone-dfe04-default-rtdb.firebaseio.com/'
cred = credentials.Certificate(cred_path)
firebase_admin.initialize_app(cred, {
    'databaseURL': databaseURL
})
action_ref = db.reference('action')

def control_drone(action, value):
    if action == "left":
        drone(moveBy(0, -value, 0, 0))
    elif action == "right":
        drone(moveBy(0, value, 0, 0))
    elif action == "up":
        drone(moveBy(0, 0, -value, 0))
    elif action == "down":
        drone(moveBy(0, 0, value, 0))
    elif action == "land":
        drone(Landing()).wait().success()

def listener(event):
    action_value = event.data["action"]
    action, value = action_value.split()
    value = float(value)
    print(f'Action: {action}, Value: {value}')
    control_drone(action, value)

if __name__ == "__main__":
    drone.connect()
    assert drone(TakeOff() >> FlyingStateChanged(state="hovering", _timeout=5)).wait().success()
    action_ref.listen(listener)
    input('Listening for changes. Press Enter to exit.\n')
    assert drone(Landing()).wait().success()
    drone.disconnect()

```

### test4 Watch App/test4App.swift

```swift
//
//  test4App.swift
//  test4 Watch App
//
//  Created by Aneesh Chennareddy Muppidi on 2/17/24.
//

import SwiftUI

@main
struct test4_Watch_AppApp: App {
    var body: some Scene {
        WindowGroup {
            ContentView()
        }
    }
}

```

### test4 Watch App/ContentView.swift

```swift
import SwiftUI

struct ContentView: View {
    @StateObject private var viewModel = GyroscopeViewModel()
    
    var body: some View {
        VStack {
            Text(viewModel.attitudeData)
                .padding()
            
            Button("Land") {
                viewModel.sendLandCommand()
            }
            .padding()
            .background(Color.blue)
            .foregroundColor(.white)
            .cornerRadius(10)
        }
    }
}

struct ContentView_Previews: PreviewProvider {
    static var previews: some View {
        ContentView()
    }
}

```

### test4 Watch AppUITests/test4_Watch_AppUITestsLaunchTests.swift

```swift
//
//  test4_Watch_AppUITestsLaunchTests.swift
//  test4 Watch AppUITests
//
//  Created by Aneesh Chennareddy Muppidi on 2/17/24.
//

import XCTest

final class test4_Watch_AppUITestsLaunchTests: XCTestCase {

    override class var runsForEachTargetApplicationUIConfiguration: Bool {
        true
    }

    override func setUpWithError() throws {
        continueAfterFailure = false
    }

    func testLaunch() throws {
        let app = XCUIApplication()
        app.launch()

        // Insert steps here to perform after app launch but before taking a screenshot,
        // such as logging into a test account or navigating somewhere in the app

        let attachment = XCTAttachment(screenshot: app.screenshot())
        attachment.name = "Launch Screen"
        attachment.lifetime = .keepAlways
        add(attachment)
    }
}

```

### test4 Watch AppTests/test4_Watch_AppTests.swift

```swift
//
//  test4_Watch_AppTests.swift
//  test4 Watch AppTests
//
//  Created by Aneesh Chennareddy Muppidi on 2/17/24.
//

import XCTest
@testable import test4_Watch_App

final class test4_Watch_AppTests: XCTestCase {

    override func setUpWithError() throws {
        // Put setup code here. This method is called before the invocation of each test method in the class.
    }

    override func tearDownWithError() throws {
        // Put teardown code here. This method is called after the invocation of each test method in the class.
    }

    func testExample() throws {
        // This is an example of a functional test case.
        // Use XCTAssert and related functions to verify your tests produce the correct results.
        // Any test you write for XCTest can be annotated as throws and async.
        // Mark your test throws to produce an unexpected failure when your test encounters an uncaught error.
        // Tests marked async will run the test method on an arbitrary thread managed by the Swift runtime.
    }

    func testPerformanceExample() throws {
        // This is an example of a performance test case.
        self.measure {
            // Put the code you want to measure the time of here.
        }
    }

}

```

### test4 Watch AppUITests/test4_Watch_AppUITests.swift

```swift
//
//  test4_Watch_AppUITests.swift
//  test4 Watch AppUITests
//
//  Created by Aneesh Chennareddy Muppidi on 2/17/24.
//

import XCTest

final class test4_Watch_AppUITests: XCTestCase {

    override func setUpWithError() throws {
        // Put setup code here. This method is called before the invocation of each test method in the class.

        // In UI tests it is usually best to stop immediately when a failure occurs.
        continueAfterFailure = false

        // In UI tests it’s important to set the initial state - such as interface orientation - required for your tests before they run. The setUp method is a good place to do this.
    }

    override func tearDownWithError() throws {
        // Put teardown code here. This method is called after the invocation of each test method in the class.
    }

    func testExample() throws {
        // UI tests must launch the application that they test.
        let app = XCUIApplication()
        app.launch()

        // Use XCTAssert and related functions to verify your tests produce the correct results.
    }

    func testLaunchPerformance() throws {
        if #available(macOS 10.15, iOS 13.0, tvOS 13.0, watchOS 7.0, *) {
            // This measures how long it takes to launch your application.
            measure(metrics: [XCTApplicationLaunchMetric()]) {
                XCUIApplication().launch()
            }
        }
    }
}

```

### test4 Watch App/oldCode.swift

```swift
/*import SwiftUI

struct OldContentView: View {
    private let baseUrl = "https://watchdrone-dfe04-default-rtdb.firebaseio.com/"
    
    var body: some View {
        VStack {
            HStack {
                Text("Left")
                    .onTapGesture {
                        updateActionInFirebase(action: "left", number: 1)
                    }
                Spacer()
                Text("Right")
                    .onTapGesture {
                        updateActionInFirebase(action: "right", number: 1)
                    }
            }
            HStack {
                Text("Down")
                    .onTapGesture {
                        updateActionInFirebase(action: "down", number: 1)
                    }
                Spacer()
                Text("Up")
                    .onTapGesture {
                        updateActionInFirebase(action: "up", number: 1)
                    }
            }
        }
        .padding()
    }
    
    func updateActionInFirebase(action: String, number: Int) {
        let actionValue = "\(action) \(number)"
        guard let url = URL(string: "\(baseUrl)action.json") else {
            return
        }
        var request = URLRequest(url: url)
        request.httpMethod = "PUT"
        request.httpBody = "{\"action\": \"\(actionValue)\"}".data(using: .utf8)
        
        let task = URLSession.shared.dataTask(with: request) { data, response, error in
            if let error = error {
                print("Error updating action: \(error)")
                return
            }
            print("Action updated to \(actionValue)")
        }
        task.resume()
    }
}


struct ContentView_Previews: PreviewProvider {
    static var previews: some View {
        ContentView()
    }
}
*/

```

### test4 Watch App/GyroscopeViewModel.swift

```swift
import Foundation
import CoreMotion
import Combine

class GyroscopeViewModel: ObservableObject {
    private let motionManager = CMMotionManager()
    private var timer: Timer?
    private let baseUrl = "https://watchdrone-dfe04-default-rtdb.firebaseio.com/"

    @Published var attitudeData: String = "Pitch: 0.0, Roll: 0.0, Yaw: 0.0"
    
    init() {
        startDeviceMotion()
    }
    
    private func startDeviceMotion() {
        if motionManager.isDeviceMotionAvailable {
            motionManager.deviceMotionUpdateInterval = 1.0 / 50.0
            motionManager.startDeviceMotionUpdates(using: .xMagneticNorthZVertical)
            
            // Configure a timer to fetch the motion data.
            timer = Timer.scheduledTimer(withTimeInterval: 1.0 / 50.0, repeats: true) { [weak self] _ in
                guard let self = self, let data = self.motionManager.deviceMotion else { return }
                // Get the attitude relative to the magnetic north reference frame.
                let pitch = data.attitude.pitch
                let roll = data.attitude.roll
                let yaw = data.attitude.yaw
                
                // Update the attitude data
                DispatchQueue.main.async {
                    self.attitudeData = String(format: "Pitch: %.2f, Roll: %.2f, Yaw: %.2f", pitch, roll, yaw)
                    self.checkCommands(pitch: pitch, roll: roll)
                }
            }
        } else {
            attitudeData = "Device motion is not available"
        }
    }
    
    private func checkCommands(pitch: Double, roll: Double) {
        if pitch < -0.9 {
            sendLeftCommand()
        } else if pitch > 0.9 {
            sendRightCommand()
        }
         else if roll < -0.9 {
            sendUpCommand()
        }
        
    }
    
    private func sendLeftCommand() {
        print("Sending Left Command")
        updateActionInFirebase(action: "left", number: 1)
    }
    
    private func sendRightCommand() {
        print("Sending Right Command")
        updateActionInFirebase(action: "right", number: 1)
    }
    
    private func sendUpCommand() {
        print("Sending Up Command")
        updateActionInFirebase(action: "up", number: 1)
    }
    func sendLandCommand() {
        print("landing!")
        updateActionInFirebase(action: "land", number: 1)
    }
    
    private func sendDownCommand() {
        print("Sending Down Command")
        updateActionInFirebase(action: "down", number: 1)
    }
    private func updateActionInFirebase(action: String, number: Int) {
            let actionValue = "\(action) \(number)"
            guard let url = URL(string: "\(baseUrl)action.json") else {
                return
            }
            var request = URLRequest(url: url)
            request.httpMethod = "PUT"
            request.httpBody = "{\"action\": \"\(actionValue)\"}".data(using: .utf8)
            
            let task = URLSession.shared.dataTask(with: request) { data, response, error in
                if let error = error {
                    print("Error updating action: \(error)")
                    return
                }
                print("Action updated to \(actionValue)")
            }
            task.resume()
        }
    
    deinit {
        motionManager.stopDeviceMotionUpdates()
        timer?.invalidate()
    }
}

```