├── README.md └── images ├── IMG_20201128_170914.jpg ├── Screenshot from 2020-11-28 15-52-31.png └── empty /README.md: -------------------------------------------------------------------------------- 1 | # HdmiPi-Streaming 2 | Streaming using a cheap HDMI capture card and a Raspberry Pi 4 to an RTMP Receiver. 3 | 4 | 5 | 6 | ![Dodgy Banner](/images/IMG_20201128_170914.jpg) 7 | 8 | 9 | 10 | After maybe a month of pulling my hair out, I finally got cheap USB HDMI capture cards work well with the Hardware encoder on the Pi and stream it to Twitch. For best performance your Raspberry Pi needs to be overclocked and have adequate cooling. These are the settings that worked for me. 11 | 12 | Here's what you need: 13 | 14 | - OC'ed Raspberry Pi 4 With decent PSU and cooling (I use a passive metal case, thats sufficient) 15 | - USB HDMI Capture card (more details below) 16 | - (Optional) HDMI Splitter 17 | - (Optional) Ethernet Connection to the Pi (5Ghz results in occasional dropped frames) 18 | 19 | HDMI Splitter isnt mandatory, but there is a 10s delay between the local stream and whats being broadcast to twitch, so I split my HDMI signal from my Video Game consoles to my Screen and to the PI Capture card. 20 | 21 | 22 | # Just tell me the code dammit: 23 | 24 | If you're in a rush, here's the code to Stream to twitch. You'll need the RTMP url and you might need to modify the command depending on how the USB device was detected on your system: 25 | 26 | ``` 27 | v4l2-ctl --set-fmt-video=width=1280,height=720 && ffmpeg -f v4l2 -thread_queue_size 384 -input_format mjpeg -framerate 30 -i /dev/video0 -f alsa -thread_queue_size 4096 -i plughw:1,0 -acodec pcm_s16le -ac 1 -ar 96000 -copytb 1 -use_wallclock_as_timestamps 1 -c:a aac -b:a 128k -ar 44100 -b:v 4M -c:v h264_omx -f flv rtmp://live.twitch.tv/app/XXXXXXXXXXXXXXXXXXXXXXX 28 | ``` 29 | 30 | 31 | # Find the right Capture card: 32 | Look up an HDMI Capture card with UVC (Usb Video Class) and UAC (Usb Audio Class). Typing in "USB Hdmi Capture UVC" should return appropriate results. Make sure that the description lists UVC and UAC. For this use case, there isnt much difference between USB 2 and USB 3, the bottleneck is the Raspberry Pi CPU/GPU. 33 | 34 | The description should look similar to below: 35 | 36 | ______ 37 | 38 | ![Ebay Description](/images/Screenshot%20from%202020-11-28%2015-52-31.png) 39 | 40 | 41 | 42 | 43 | # Raspberry Pi Setup 44 | 45 | ______________ 46 | 47 | 48 | # Raspberry Pi OC settings (/boot/config.txt): 49 | ``` 50 | arm_freq=2000 51 | enable_tvout 52 | #hdmi_enable_4kp60=1 53 | core_freq=550 54 | #gpu_freq=800 55 | h264_freq=850 56 | over_voltage=6 57 | disable_splash=1 58 | force_turbo=1 #Voids Warranty! (uncomment to avoid CPU scaling down to 600Mhz) 59 | boot_delay=1 #helps to avoid sdcard corruption when force_turbo is enabled. 60 | #sdram_freq=500 #uncomment to test. Works only with some boards. 61 | ``` 62 | 63 | # Identifying capture card and setting resolution: 64 | 65 | Set up your Pi, making sure it is up to date, and ensuring that `ffmpeg` and `v4l-utils` is installed. You can do so with the command `sudo apt install ffmpeg v4l-utils`. Plug in the device, wait a moment and type into the terminal (local or SSH), `v4l2-ctl --list-devices`. You should get something similar to this: 66 | 67 | ``` 68 | pi@capturepi:~ $ v4l2-ctl --list-devices 69 | bcm2835-codec-decode (platform:bcm2835-codec): 70 | /dev/video10 71 | /dev/video11 72 | /dev/video12 73 | 74 | bcm2835-isp (platform:bcm2835-isp): 75 | /dev/video13 76 | /dev/video14 77 | /dev/video15 78 | /dev/video16 79 | 80 | UVC Camera (534d:2109): USB Vid (usb-0000:01:00.0-1.4): 81 | /dev/video0 82 | /dev/video1 83 | 84 | ``` 85 | 86 | The last listing is what we're interested in. Make a quick note of where the device is (in my case **/dev/video0**) The next step will indicate how to decide between video0 and video1. Type in `v4l2-ctl -d /dev/video0 --list-formats-ex` 87 | 88 | ``` 89 | pi@capturepi:~ $ v4l2-ctl -d /dev/video0 --list-formats-ex 90 | ioctl: VIDIOC_ENUM_FMT 91 | Type: Video Capture 92 | 93 | [0]: 'MJPG' (Motion-JPEG, compressed) 94 | Size: Discrete 1920x1080 95 | Interval: Discrete 0.033s (30.000 fps) 96 | Interval: Discrete 0.040s (25.000 fps) 97 | Interval: Discrete 0.050s (20.000 fps) 98 | Interval: Discrete 0.100s (10.000 fps) 99 | Interval: Discrete 0.200s (5.000 fps) 100 | Size: Discrete 1600x1200 101 | Interval: Discrete 0.033s (30.000 fps) 102 | Interval: Discrete 0.040s (25.000 fps) 103 | Interval: Discrete 0.050s (20.000 fps) 104 | Interval: Discrete 0.100s (10.000 fps) 105 | Interval: Discrete 0.200s (5.000 fps) 106 | Size: Discrete 1360x768 107 | Interval: Discrete 0.033s (30.000 fps) 108 | Interval: Discrete 0.040s (25.000 fps) 109 | Interval: Discrete 0.050s (20.000 fps) 110 | Interval: Discrete 0.100s (10.000 fps) 111 | Interval: Discrete 0.200s (5.000 fps) 112 | Size: Discrete 1280x1024 113 | Interval: Discrete 0.033s (30.000 fps) 114 | Interval: Discrete 0.040s (25.000 fps) 115 | Interval: Discrete 0.050s (20.000 fps) 116 | Interval: Discrete 0.100s (10.000 fps) 117 | Interval: Discrete 0.200s (5.000 fps) 118 | Size: Discrete 1280x960 119 | Interval: Discrete 0.020s (50.000 fps) 120 | Interval: Discrete 0.033s (30.000 fps) 121 | Interval: Discrete 0.050s (20.000 fps) 122 | Interval: Discrete 0.100s (10.000 fps) 123 | Interval: Discrete 0.200s (5.000 fps) 124 | Size: Discrete 1280x720 125 | Interval: Discrete 0.017s (60.000 fps) 126 | Interval: Discrete 0.020s (50.000 fps) 127 | Interval: Discrete 0.033s (30.000 fps) 128 | Interval: Discrete 0.050s (20.000 fps) 129 | Interval: Discrete 0.100s (10.000 fps) 130 | Size: Discrete 1024x768 131 | Interval: Discrete 0.017s (60.000 fps) 132 | Interval: Discrete 0.020s (50.000 fps) 133 | Interval: Discrete 0.033s (30.000 fps) 134 | Interval: Discrete 0.050s (20.000 fps) 135 | Interval: Discrete 0.100s (10.000 fps) 136 | Size: Discrete 800x600 137 | Interval: Discrete 0.017s (60.000 fps) 138 | Interval: Discrete 0.020s (50.000 fps) 139 | Interval: Discrete 0.033s (30.000 fps) 140 | Interval: Discrete 0.050s (20.000 fps) 141 | Interval: Discrete 0.100s (10.000 fps) 142 | Size: Discrete 720x576 143 | Interval: Discrete 0.017s (60.000 fps) 144 | Interval: Discrete 0.020s (50.000 fps) 145 | Interval: Discrete 0.033s (30.000 fps) 146 | Interval: Discrete 0.050s (20.000 fps) 147 | Interval: Discrete 0.100s (10.000 fps) 148 | Size: Discrete 720x480 149 | Interval: Discrete 0.017s (60.000 fps) 150 | Interval: Discrete 0.020s (50.000 fps) 151 | Interval: Discrete 0.033s (30.000 fps) 152 | Interval: Discrete 0.050s (20.000 fps) 153 | Interval: Discrete 0.100s (10.000 fps) 154 | Size: Discrete 640x480 155 | Interval: Discrete 0.017s (60.000 fps) 156 | Interval: Discrete 0.020s (50.000 fps) 157 | Interval: Discrete 0.033s (30.000 fps) 158 | Interval: Discrete 0.050s (20.000 fps) 159 | Interval: Discrete 0.100s (10.000 fps) 160 | [1]: 'YUYV' (YUYV 4:2:2) 161 | Size: Discrete 1920x1080 162 | Interval: Discrete 0.200s (5.000 fps) 163 | Size: Discrete 1600x1200 164 | Interval: Discrete 0.200s (5.000 fps) 165 | Size: Discrete 1360x768 166 | Interval: Discrete 0.125s (8.000 fps) 167 | Size: Discrete 1280x1024 168 | Interval: Discrete 0.125s (8.000 fps) 169 | Size: Discrete 1280x960 170 | Interval: Discrete 0.125s (8.000 fps) 171 | Size: Discrete 1280x720 172 | Interval: Discrete 0.100s (10.000 fps) 173 | Size: Discrete 1024x768 174 | Interval: Discrete 0.100s (10.000 fps) 175 | Size: Discrete 800x600 176 | Interval: Discrete 0.050s (20.000 fps) 177 | Interval: Discrete 0.100s (10.000 fps) 178 | Interval: Discrete 0.200s (5.000 fps) 179 | Size: Discrete 720x576 180 | Interval: Discrete 0.040s (25.000 fps) 181 | Interval: Discrete 0.050s (20.000 fps) 182 | Interval: Discrete 0.100s (10.000 fps) 183 | Interval: Discrete 0.200s (5.000 fps) 184 | Size: Discrete 720x480 185 | Interval: Discrete 0.033s (30.000 fps) 186 | Interval: Discrete 0.050s (20.000 fps) 187 | Interval: Discrete 0.100s (10.000 fps) 188 | Interval: Discrete 0.200s (5.000 fps) 189 | Size: Discrete 640x480 190 | Interval: Discrete 0.033s (30.000 fps) 191 | Interval: Discrete 0.050s (20.000 fps) 192 | Interval: Discrete 0.100s (10.000 fps) 193 | Interval: Discrete 0.200s (5.000 fps) 194 | pi@capturepi:~ $ v4l2-ctl -d /dev/video1 --list-formats-ex 195 | ioctl: VIDIOC_ENUM_FMT 196 | Type: Video Capture 197 | 198 | ``` 199 | 200 | In my Instance **video0** lists a bunch of formats whilst **video1** doesnt. You could try to open **video0** in VLC or ffplay at this moment but you'll notice that the frame rate is very low. Thats because it defaults to the "YUYV" stream instead of "MJPG" one. We will need to use the "MJPG" stream in this instance. Take note of **/dev/video0** or whatever it may be in your case. 201 | 202 | 203 | Now, we need to sort out audio. List the audio devices using the command "cat /proc/asound/devices" : 204 | 205 | ``` 206 | pi@capturepi:~ $ cat /proc/asound/devices 207 | 0: [ 0] : control 208 | 16: [ 0- 0]: digital audio playback 209 | 32: [ 1] : control 210 | 33: : timer 211 | 56: [ 1- 0]: digital audio capture 212 | ``` 213 | 214 | The last one looks like what we're after. Take note of the **"[ 1- 0]"** or what it might be in your setup. In the script, it will be formatted as **1,0**. If it was listed as **"[ 2- 0]"**, it would be formatted as **2,0**. 215 | 216 | 217 | Now, we have the video stream **/dev/video0** and the audio stream **1,0**. However, the PI (As of me writing this) cannot convert 1080p footage at anything above 8-10fps. I suspect that this is due to Software conversion happening for the colour values in the MJPG stream. You'll need to reduce the resolution of the capture device. This is temporary, and will reset on device reboot, or if you change it manually. To change the resolution to 720P, you'll need to invoke the following command: 218 | 219 | ```v4l2-ctl --set-fmt-video=width=1280,height=720``` 220 | 221 | # ffmpeg Command explaination 222 | 223 | The command to stream the video **/dev/video0** and audio **1,0** to ffmpeg, convert it using the Raspberry Pi inbuilt HW encoder, then stream it to twitch is as followss: 224 | 225 | ``` 226 | ffmpeg -f v4l2 -thread_queue_size 384 -input_format mjpeg -framerate 30 -i /dev/video0 -f alsa -thread_queue_size 4096 -i plughw:1,0 -acodec pcm_s16le -ac 1 -ar 96000 -copytb 1 -use_wallclock_as_timestamps 1 -c:a aac -b:a 128k -ar 44100 -b:v 4M -c:v h264_omx -f flv rtmp://live.twitch.tv/app/XXXXXXXXXXXXXXXXXXXXXXX 227 | ``` 228 | 229 | Lets break it down: 230 | 231 | - ffmpeg : The program we are using for conversion 232 | - "-f v4l2 -thread_queue_size 384 -input_format mjpeg -framerate 30 -i /dev/video0" : Use the 30fps MJPEG stream at **/dev/video0** and give it some buffer. 233 | - "-f alsa -thread_queue_size 4096 -i plughw:1,0 -acodec pcm_s16le -ac 1 -ar 96000 -copytb 1 -use_wallclock_as_timestamps 1" : Use the audio from **1,0**, mono audio (Capture card limitation), codec pcm_161e, with a sample of 96kHz. 234 | - "-c:a aac -b:a 128k -ar 44100 -b:v 8M -c:v h264_omx" : Convert Audio to Birtate 128K, sample rate of 44.1kHz. Convert Video using Hardware H264 encoder, bitrate of 4Mb/s. 235 | 236 | - "-f flv rtmp://live.twitch.tv/app/XXXXXXXXXXXXXXXXXXXXXXX " Mux the convereted Video and Audio into an FLV container and send it to your stream destination. 237 | 238 | 239 | You should see your stream show up on your platform of choice, however there will be a delay. On twitch, I get a roughly 10s delay between gameplay and whats on twitch. 240 | 241 | 242 | # Benefits : 243 | 244 | - USB HDMI Capture cards are inexpensive compared to other HDMI capture solutions. 245 | - Raspberry Pi is relatively inexpensive and has little power comsumption compared to dedicated capture PC 246 | - If capturing a PC display, less conversion load on host PC. 247 | - Minimal software prerequesites, Raspberry Pi OS Lite + `ffmpeg` + `v4l-utils` is basically all you need. 248 | 249 | 250 | # Caveats 251 | 252 | - Audio is only at mono (I think this is a limitation of the capture card themselves) 253 | - Resolution is capped at 720P for 25+ FPS stream, due to software conversion of the MJPEG colourspace (I think, I might be wrong) 254 | - No GUI (at present) 255 | - Some proficency at command line needed. 256 | 257 | 258 | ## Acknowledgements 259 | 260 | My knowledge in linux is bare minimal, I take no credit whatsoever apart from documenting what worked in my case. 261 | 262 | [Arch Wiki, Ofcourse](https://wiki.archlinux.org/index.php/Streaming_to_twitch.tv) 263 | 264 | [FFmpeg Documentation](https://trac.ffmpeg.org/wiki/EncodingForStreamingSites) 265 | 266 | [ffmpeg IRC Channel](https://ffmpeg.org/contact.html) (the person who helped me didnt wish to be credited) 267 | 268 | https://github.com/pikvm/ustreamer (I used this for LOTS of debugging) 269 | 270 | Big shoutout to [Will Usher and his Blog](https://www.willusher.io/general/2020/11/15/hw-accel-encoding-rpi4), for alerting me to his blog as well as a new encoder. As of this moment, the Raspberry Pi is still constrained by its CPU for a filtering step, however, I will be looking into improving performance. 271 | -------------------------------------------------------------------------------- /images/IMG_20201128_170914.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PrawnMan/HdmiPi-Streaming/573020a8785315578080b07706174605d3a27a16/images/IMG_20201128_170914.jpg -------------------------------------------------------------------------------- /images/Screenshot from 2020-11-28 15-52-31.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PrawnMan/HdmiPi-Streaming/573020a8785315578080b07706174605d3a27a16/images/Screenshot from 2020-11-28 15-52-31.png -------------------------------------------------------------------------------- /images/empty: -------------------------------------------------------------------------------- 1 | . 2 | --------------------------------------------------------------------------------