├── screen-sharing ├── USB.md └── USB+AIRPLAY.md ├── LICENSE ├── camera ├── QVH+SBC-UVC.md ├── QVH+WEBCAMOID.md ├── CAMTWIST.md └── OBS+CAMTWIST.md └── README.md /screen-sharing/USB.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Andrew Odri 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /screen-sharing/USB+AIRPLAY.md: -------------------------------------------------------------------------------- 1 | When you are using an iOS device as a camera, and additionally want to share anothert iOS devices screen, Zoom will automatically choose which device screen is shared. In my experience, this is always the iPhone, and that invariably breaks the connection with the camera software, and you're worse of off then when you started. 2 | 3 | If you want to share your iPad's screen _and_ audio, then this may be an acceptable solution. 4 | 5 | ## On your iPad... 6 | 7 | 1. Enable Airplane mode, turn off Wi-fi, and turn off notifications _(This is so that you don't get notifications popping up while screen sharing, and don't have any unexpected updates downloading in the background choking up your connection.)_ 8 | 2. Connect your iPad to your Mac via USB 9 | 10 | ## On your Mac... 11 | 12 | 1. Open System Preferences, select Sharing, and check Internet Sharing 13 | 2. On the Internet Sharing options screen, select the device that supplies your internet for "Share your connection from", and select your "[iPad name] USB" from "To computers using" 14 | 15 | Even though your iPad is in Airplane mode with Wi-fi disabled, applications will still be able to connect to the internet (and your local network) via internet connection sharing over USB. 16 | 17 | The next steps take place once your Zoom meeting is established, and you wish to share your iPad's screen: 18 | 19 | 1. Click on the Share Screen button from the meeting window 20 | 2. Check thre "Share computer sound" checkbox, and then select iPhone/iPad via AirPlay 21 | 3. On your iPad, open the control center, and select Screen Mirroring 22 | 4. Select the "Zoom-[Mac name]" target 23 | 24 | You iPad screen and audio will now be shared within the meeting. 25 | -------------------------------------------------------------------------------- /camera/QVH+SBC-UVC.md: -------------------------------------------------------------------------------- 1 | First, download and unzip Raspbian: 2 | 3 | ``` 4 | mkdir assets 5 | curl -o assets/raspbian_2019-07-10.zip http://director.downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2019-07-12/2019-07-10-raspbian-buster-lite.zip 6 | unzip -p assets/raspbian_2019-07-10.zip > assets/raspbian_2019-07-10.img 7 | 8 | curl -o assets/raspbian_2020-02-05.zip http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2020-02-07/2020-02-05-raspbian-buster-lite.zip 9 | unzip -p assets/raspbian_2020-02-05.zip > assets/raspbian_2020-02-05.img 10 | 11 | curl -o assets/raspbian_2020-02-13.zip http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2020-02-14/2020-02-13-raspbian-buster-lite.zip 12 | unzip -p assets/raspbian_2020-02-13.zip > assets/raspbian_2020-02-13.img 13 | ``` 14 | 15 | Next, flash the image to a TF card: 16 | 17 | ``` 18 | diskutil list 19 | diskutil unmountDisk /dev/disk4 20 | sudo dd bs=1m if=assets/raspbian_2019-07-10.img of=/dev/rdisk4 21 | ``` 22 | 23 | Upload the `wpa_supplicant.conf` file to the Pi Zero's boot partition, enable SSH, and disable all the cruft we don't need: 24 | 25 | ``` 26 | echo "country=US 27 | ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev 28 | 29 | network={ 30 | ssid=\"SSID\" 31 | scan_ssid=1 32 | psk=\"password\" 33 | key_mgmt=WPA-PSK 34 | }" > /Volumes/boot/wpa_supplicant.conf 35 | 36 | cp cmdline.txt /Volumes/boot/cmdline.txt 37 | cp config.txt /Volumes/boot/config.txt 38 | touch /Volumes/boot/ssh 39 | ``` 40 | 41 | Unmount the TF card and insert it in the Pi Zero. The following command will unmount the TF card: 42 | 43 | ``` 44 | diskutil unmountDisk /dev/disk4 45 | ``` 46 | 47 | Power up the Pi Zero, and connect to to the Pi Zero via SSH using the following command: _(The password is `raspberry` by default)_ 48 | 49 | ``` 50 | ssh-keygen -R raspberrypi && ssh pi@raspberrypi 51 | sudo apt-get install -y v4l2loopback-dkms 52 | sudo modprobe v4l2loopback 53 | sudo modprobe g_webcam 54 | curl -L -o uvc.gz https://github.com/jdonald/uvc-gadget/files/1443504/uvc-gadget.gz && gunzip uvc.gz && chmod 755 uvc 55 | sudo ./uvc -d -r 1 -s 2 -u /dev/video1 -v /dev/video0 56 | ``` -------------------------------------------------------------------------------- /camera/QVH+WEBCAMOID.md: -------------------------------------------------------------------------------- 1 | Low latency, low sync proven working with Webcamoid: 2 | 3 | ``` 4 | go run main.go gstreamer --pipeline "queue name=audio_target ! wavparse ignore-length=true ! audioconvert ! faac ! aacparse ! mux. 5 | queue name=video_target ! h264parse ! vtdec ! videoconvert ! aspectratiocrop aspect-ratio=16/9 ! videoscale ! video/x-raw,width=1280,height=720 ! videorate ! video/x-raw,framerate=15/1 ! x264enc tune=zerolatency speed-preset=superfast ! h264parse ! mux. 6 | mpegtsmux name=mux ! tcpserversink host=localhost port=7000 sync=false" 7 | ``` 8 | 9 | High latency, high sync proven working with Webcamoid: 10 | 11 | ``` 12 | go run main.go gstreamer --pipeline "queue name=audio_target ! wavparse ignore-length=true ! audioconvert ! faac ! aacparse ! queue2 ! mux. 13 | queue name=video_target ! h264parse ! vtdec ! videoconvert ! aspectratiocrop aspect-ratio=16/9 ! videoscale ! video/x-raw,width=1280,height=720 ! videorate ! video/x-raw,framerate=15/1 ! x264enc tune=zerolatency speed-preset=superfast ! h264parse ! queue2 ! mux. 14 | mpegtsmux name=mux ! tcpserversink host=localhost port=7000 recover-policy=keyframe sync-method=latest-keyframe" 15 | ``` 16 | 17 | Hyperlapse capture with colour correction: 18 | 19 | ``` 20 | go run main.go gstreamer --pipeline "queue name=audio_target ! wavparse ignore-length=true ! audioconvert ! faac ! aacparse ! mux. 21 | queue name=video_target ! h264parse ! vtdec ! videoconvert ! aspectratiocrop aspect-ratio=16/9 ! videobalance hue=-0.1 ! videoscale ! video/x-raw,width=1280,height=720 ! videorate ! video/x-raw,framerate=30/1 ! x264enc tune=zerolatency speed-preset=superfast ! h264parse ! mux. 22 | mpegtsmux name=mux ! tcpserversink host=localhost port=7000 sync=false" 23 | ``` 24 | 25 | Camera app capture with cropping on iPhone 11: 26 | 27 | ``` 28 | go run main.go gstreamer --pipeline "queue name=audio_target ! wavparse ignore-length=true ! audioconvert ! faac ! aacparse ! mux. 29 | queue name=video_target ! h264parse ! vtdec ! videoconvert ! videoflip method=counterclockwise ! videocrop left=242 right=446 ! videoscale ! video/x-raw,width=640,height=480 ! videorate ! video/x-raw,framerate=30/1 ! x264enc tune=zerolatency speed-preset=superfast ! h264parse ! mux. 30 | mpegtsmux name=mux ! tcpserversink host=localhost port=7000 sync=false" 31 | ``` 32 | -------------------------------------------------------------------------------- /camera/CAMTWIST.md: -------------------------------------------------------------------------------- 1 | This method is crazy simple... The only hiccup you might encounter is having to [jump through some of macOS Catalina's new security hoops](https://support.apple.com/en-ca/HT202491). Here are the requirements: 2 | 3 | * **macOS**: Download and install [Zoom.us](https://zoom.us/download) 4 | * **macOS**: Download and install [CamTwist Studio](http://camtwiststudio.com/download/) 5 | * **iOS**: Download and install [Hyperlapse](https://apps.apple.com/us/app/hyperlapse-from-instagram/id740146917) _(Why Hyperlapse? It is very very well supported, has great image stabilization for older iPhones, and most of all, it has a very simple uncluttered interface for screen sharing. It will also prevent your iPhone from automatically locking after a specific period of time.)_ 6 | 7 | ## On your iPhone... 8 | 9 | 1. Enable Airplane mode, turn off Wi-fi, and turn off notifications _(This is so that you don't get notifications popping up over your camera, and don't have any unexpected updates downloading in the background choking up your connection.)_ 10 | 2. Open Hyperlapse, and configure it to point at whatever you want to show up on your camera 11 | 3. Connect your iPhone to your Mac via USB 12 | 4. When prompted with "Trust This Computer?", select "Trust" _(This is what allows the screen to be shared natively via USB with macOS.)_ 13 | 14 | ## On your Mac... 15 | 16 | Follow these instructions to set up CamTwist: 17 | 18 | 1. Open CamTwist, go to the Preferences... menu, and select the General tab 19 | 2. Select Custom from the Video Size select list, and enter 1920x1080 as the resolution 20 | 3. Open CamTwist, from Video Sources select Webcam, then click on the Select button 21 | 4. From Settings, go to Camera, and select your iPhone 22 | 23 | Follow these instructions to set up Zoom: 24 | 25 | 1. Open Zoom, go to Preferences..., and select Video 26 | 2. From Camera, select CamTwist 27 | 28 | That is all you need for bare-bones setup with Zoom! However, you may have a mid-week and week-end meeting where you have to look nice and act appropriately. It you close CamTwist at this point, you'll notice the camera has a cool looking test pattern, and an ugly looking logo on the top. Want that logo to go away? Here's how: 29 | 30 | 1. Open Terminal 31 | 2. Type in the following command, then press enter: `sudo zsh -c 'echo iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII= | base64 -D -o /Library/CoreMediaIO/Plug-Ins/DAL/CamTwist.plugin/Contents/Resources/splash.png'` _(This overwrites the "splash" image the camera uses with a 1x1px transparent PNG that was decoded from a base64 string.)_ 32 | 3. Type in your password when prompted 33 | -------------------------------------------------------------------------------- /camera/OBS+CAMTWIST.md: -------------------------------------------------------------------------------- 1 | This is just-as-if-not-more-so janky as the previous method, but it relies on a video stream overe USB from the iPhone rather than screen sharing, which is then processed by some real broadcast software. What does that mean? It'll chew up way more resources, but you have options for much higher resolution video (i.e. 4K), and you won't have issues with notifications or extra artifacts in your feed. Here are the requirements: 2 | 3 | * **Hardware**: Purchase and install an [HDMI dummy connector](https://www.amazon.ca/gp/product/B0788HYC1S/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&th=1): **~$10.99** _(Fast delivery despite COVID-19!)_ 4 | * **iOS**: Download and install [Camera for OBS Studio](https://apps.apple.com/ca/app/camera-for-obs-studio/id1352834008): **$21.99** 5 | * **macOS**: Download and install [OBS Studio](https://obsproject.com/download) 6 | * **macOS**: Download and install [iOS Camera plugin for OBS Studio](https://obs.camera/docs/getting-started/ios-camera-plugin-usb/) 7 | * **macOS**: Download and install [Zoom.us](https://zoom.us/download) 8 | * **macOS**: Download and install [CamTwist Studio](http://camtwiststudio.com/download/) 9 | 10 | ## On your iPhone... 11 | 12 | 1. Download and install Camera for OBS Studio 13 | 2. Open Camera for OBS Studio 14 | 3. Connect your iPhone to your Mac via USB 15 | 4. When prompted with "Trust This Computer?", select "Trust" _(This is what allows the screen to be shared natively via USB with macOS.)_ 16 | 17 | ## On your Mac... 18 | 19 | Go to System Preferences, select Mission Control, then uncheck Displays have separate Spaces. You will need to log out, then log back in again. This removes the menu bar on the dummy display, so that we can extract the native 1080p video that is getting streamed to it without any unwanted visual artifacts. It also prevents apps from opening on second "display". 20 | 21 | The command below should do the same thing, although your milage may vary: 22 | 23 | ``` 24 | sudo defaults write com.apple.spaces spans-displays 1 25 | ``` 26 | 27 | Follow these instructions to set up OBS: 28 | 29 | 1. Plug in your HDMI dummy plug into a spare HDMI port _(You might need an extra dongle for this one.)_ 30 | 2. Set your dummy display to separate display, and set the resolution of the dummy display to whatever you want your camera resolution to be 31 | 3. Open OBS Studio 32 | 4. Set up the iOS camera 33 | 5. Right click on the preview window, expand the Fullscreen Projector (Source) sub menu, then select the 1920x1080 display _(Likely "Display 2: 1920x1080 @ ..." or something to that effect.)_ 34 | 6. Go to the OBS menu, select Preferences..., and select the General tab 35 | 7. Go to the Projectors heading, and check Save projectors on exit 36 | 8. Right click on the preview window, and uncheck the Enable preview menu item _(Optional: This will conserve resources.)_ 37 | 38 | This will ensure that configuration consumes the least amount of resources, and also opens up pre-configured by default. 39 | 40 | Follow these instructions to set up CamTwist: 41 | 42 | 1. Open CamTwist, go to the Preferences... menu, and select the General tab 43 | 2. Select Custom from the Video Size select list, and enter 1920x1080 as the resolution _(This is so that no scaling is required when the 1080p desktop is captured.)_ 44 | 3. Open CamTwist, from Video Sources select Display, then click on the Select button 45 | 4. Go to the Settings section, then select the additional display from the Screen select list 46 | 5. Go to the Adjust settings section, click on the Save Setup button, and give the setup and appropriate name 47 | 6. Go to the Saved Setups select box, select the newly created setup, and then click on the Auto load button so that the name of the setup is bolded 48 | 49 | This will ensure that the virtual camera is configured to pull from the secondary display automatically when it is opened. 50 | 51 | Follow these instructions to set up Zoom: 52 | 53 | 1. Open Zoom, go to Preferences..., and select Video 54 | 2. From Camera, select CamTwist 55 | 56 | That is all you need for bare-bones setup with Zoom! However, you may have a mid-week and week-end meeting where you have to look nice and act appropriately. It you close CamTwist at this point, you'll notice the camera has a cool looking test pattern, and an ugly looking logo on the top. Want that logo to go away? Here's how: 57 | 58 | 1. Open Terminal 59 | 2. Type in the following command, then press enter: `sudo zsh -c 'echo iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII= | base64 -d -o /Library/CoreMediaIO/Plug-Ins/DAL/CamTwist.plugin/Contents/Resources/splash.png'` _(This overwrites the "splash" image the camera uses with a 1x1px transparent PNG that was decoded from a base64 string.)_ 60 | 3. Type in your password when prompted 61 | 62 | ## Troubleshooting 63 | 64 | ### Camera for OBS Studio won't connect. 65 | 66 | This is a bit of annoying one... Not sure what is causing it, but here is the process I've found you need to follow in order to get it to connect: Disconnect the USB cable, then force close OBS Camera on the device, and close OBS Studio on your Mac. After that open OBS Camera on iOS, orient the display appropriately, plug in the USB cable, then open OBS Studio on your Mac. You may need to open the iOS Camera source settings and reconnect the camera. 67 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # iPhone to macOS camera 2 | 3 | If you're anything like me, you've probably got an iPhone that is a couple of generations old sitting unused in a box. Now that you're locked in your home becuase of the little SARS-CoV-2 virii floating around these days, you may need video conferencing _and_ screen sharing on the same call; the days of flubbing it with your iPad are over. Of course, you MacBook has a camera built in, but it sucks, and more importantly, opening your MacBook will mess up you windows on your big monitor. 4 | 5 | The solution? Buy a webcam on Amazon... But delivery is now over 1 month becuase Amazon only delivers toilet paper and pasta, and [apparently can't afford to pay their driver's sick leave](https://www.snopes.com/fact-check/amazon-donations-sick-leave/)? Whatever. 6 | 7 | You know there is a better way... You know you that 12-megapixel Sony Exmos RS camera with it's six-element lens on your old iPhone 7 is a gazillion times better than whatever top-of-line webcam Logitech is trying to sell you. Also, you're webcam now has resale value and it's 2GHz+ quad-core processor. If that is your predicament, then this repo is for you. 8 | 9 | _**Note**: I wrote this for my own use... And right now I only use Zoom. This won't work with 1st-party apps like Facetime. Your milage may vary with other 3rd-party apps._ 10 | 11 | _**Note**: All of these techniques try to take advantage of direct connections between the device and the host computer. This reduces the liklihood of connection errors, latency issues, and bottlenecks due to router performance or internet bandwidth limitations._ 12 | 13 | ## Configuring your iPhone as a macOS camera 14 | 15 | This is intended to be become a list of all various ways you can configure an iOS device as a webcam/camera. Hopefully it'll inspire the development of some more elegant solutions as time goes on. 16 | 17 | You may also be thinking what I did the first couple of times I saw this... I'm on macOS Catalina 10.15.3, so CamTwist will not work due to code signing and SIP. It does, and it's really easy. I suspect that part of the solution is install these apps the old-school way; i.e. not using the App Store. If you haven't install an app this way for a while, it might seem like macOS gives you a hard no when you attempt to open one of these apps. **You will need to [go to System Preferences, Security & Privacy, General, and select Open Anyway](https://support.apple.com/en-ca/HT202491) _after_ getting denied in order to proceed.** After you've done that, you'll be able to open the app normally moving forward. 18 | 19 | * [Less configuration, less polished result](camera/CAMTWIST.md) (Using only CamTwist) 20 | * [More configuration, more polished result](camera/OBS+CAMTWIST.md) (Using OBS Studio and CamTwist) 21 | 22 | ## Screen sharing your iPad via Zoom for macOS 23 | 24 | This is another one of my specific use cases... I have a application that only has an iOS client, which I need to share during Zoom meetings. These are the methods you can use to share your iPad's screen: 25 | 26 | * [Over USB](https://support.zoom.us/hc/en-us/articles/201379235-iOS-Screen-Sharing-with-the-Zoom-Desktop-Client) (When an iOS camera **is not** being utilized) 27 | * [Over USB via AirPlay](screen-sharing/USB+AIRPLAY.md) (When an iOS camera **is** being utilized) 28 | 29 | ## Solution imagineering 30 | 31 | The techniques below are high-level notes on a few existing and possibly compatible approaches that could used to put together a dedicated solution with the lowest possible overhead. 32 | 33 | The headings below will eventually get moved into the following documents: 34 | 35 | * [CoreMediaIO DAL Plug-in approach](camera/QVH+WEBCAMOID.md) 36 | * [Single-board-computer UVC gadget approach](camera/QVH+SBC-UVC.md) 37 | 38 | ### Capturing video and audio from iOS via USB 39 | 40 | The best (i.e. most grokkable) implementation of video and audio capture I've seen to date is the [quicktime video hack](https://github.com/danielpaulus/quicktime_video_hack) project by [Daniel Paulus](https://github.com/danielpaulus). It's written in [Go](https://golang.org/), and outputs to [gstreamer](https://gstreamer.freedesktop.org/documentation/plugins_doc.html?gi-language=c), which makes output manipulation _extremely_ easy and flexible. The process it employs along with a changelog of challenges encountered and conquered is also available. 41 | 42 | From what I can tell, the things you need to do are: 43 | 44 | 1. You need to enable ["Quicktime Capture" mode](https://github.com/danielpaulus/quicktime_video_hack/blob/master/doc/technical_documentation.md#13-hidden-configuration) via USB, 45 | 2. [Work your way back from the gstreamer targets](https://github.com/danielpaulus/quicktime_video_hack/blob/master/screencapture/gstadapter/gst_adapter_macos.go) to get the least processed usable streams you need for integration 46 | 47 | ### Creating a macOS virtual camera 48 | 49 | Virtual cameras are actually a bona-fide thing in macOS. They utilize [CoreMedia API "Device Abstraction Layer" (DAL) plug-ins](https://developer.apple.com/library/archive/samplecode/CoreMediaIO/Introduction/Intro.html), as opposed to the Hardware Abstraction Layer (HAL). If you have some installed, you can poke around here: `/Library/CoreMediaIO/Plug-Ins/DAL/YourFancyVirtualCamera.plugin/` 50 | 51 | While SIP prevents unsigned cameras from being used in 1st party apps in newer versions of macOS, they can be used with all 3rd party apps under Catalina (as far as I've tested at the time of writing anyway). 52 | 53 | [Webcamoid](https://github.com/webcamoid/webcamoid/) provides this functionality... It's clunky and leans hard into QT, so I'd probably need to convinced for using that code without heavy refactoring for a macOS specific solution. But it's the only open source thing out there that works. The code is pretty tough (for me) to grok, but from what I can tell, the party starts in the [IpcBridge](https://github.com/webcamoid/webcamoid/blob/4000735bd2f5678153b44d6133d1a9307964772a/libAvKys/Plugins/VirtualCamera/src/dshow/VCamIPC/src/ipcbridge.cpp). 54 | 55 | ### Creating an SBC UVC camera device 56 | 57 | I feel like this could be a really slick solution... Provided performance was adequate, if a modified QVH could be run on a small SBC, like a Raspberry Pi Zero, and register as UVC camera, then you could have a sub-$20 adapter that registers as a hardware camera and gives macOS (and every other OS for that matter) a trusted camera device to work with. 58 | 59 | The only gotcha there is that there is very little high-level code that handles registration/de-registration as a gadget... This is a probably the place to start: https://stackoverflow.com/questions/42895950/usb-gadget-device-mode-configfs-uvc-and-mass-storage-on-single-configurat 60 | --------------------------------------------------------------------------------