├── LICENSE ├── README-cn.md ├── README.md ├── VFIO ├── Configure Libvirt cn.md ├── Configure Libvirt.md ├── Debugging and other features.md ├── Libvirt Hooks cn.md ├── Libvirt Hooks.md ├── Setting up Passthrough cn.md ├── Setting up Passthrough.md ├── Setting up a basic KVM cn.md ├── Setting up a basic KVM.md └── libvirt_hooks_nvidia-cn.md ├── libvirt-hooks ├── qemu ├── vfio-startup.sh └── vfio-teardown.sh ├── libvirt-nvidia-hooks ├── qemu ├── vfio-startup.sh └── vfio-teardown.sh └── resources ├── iommuamd.sh └── wechatpay.jpg /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (C) 2021 liucreator. 2 | Permission is granted to copy, distribute and/or modify this document 3 | under the terms of the GNU Free Documentation License, Version 1.3 4 | or any later version published by the Free Software Foundation; 5 | with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. 6 | A copy of the license is included in the section entitled "GNU 7 | Free Documentation License". -------------------------------------------------------------------------------- /README-cn.md: -------------------------------------------------------------------------------- 1 | # ledis的单显卡直通教程 2 | [英文版本](README.md) 3 | 4 | 这是我的单显卡直通配置过程;我在多个不同配置的电脑上都试过，这个方法都成功了。
5 | 如果有问题可以去B站上找我, 视频教程链接：*还没做完*https://space.bilibili.com/589560036
6 | 7 | - [![Followers](https://bilistats.lonelyion.com/followers?uid=589560036)](https://space.bilibili.com/589560036/channel/seriesdetail?sid=2031728) 8 | - [![YouTube Channel Subscribers](https://img.shields.io/youtube/channel/subscribers/UCKXFTVfYRA8Ho71bAT5tfVA?style=social)](https://www.youtube.com/channel/UCKXFTVfYRA8Ho71bAT5tfVA?sub_confirmation=1) 9 | 10 |
11 | 我这里讲的是AMD处理器（英特尔的也差不多，个别参数会不一样）; 系统是Arch, 如果是其它发行版一些软件名和文件位置可能会不太一样。
12 | 13 | --- 14 | 更新日期：2021年12月26日 15 | 添加了更适合英伟达显卡直通的脚本 16 | --- 17 | 18 | **我的电脑配置** 19 | - 处理器: AMD 锐龙 3900x 12核 20 | - 显卡: 微星 Radeon RX 560 4GT LP 超频 21 | - 主版：玩家国度 ROG Strix B550-A GAMING 吹雪 22 | - 内存: 32GB 3200Mhz 双通道 23 | - 系统: Arch Linux 5.11 24 | - 桌面环境: KDE Plasma 5.21 X11 25 | 26 | * * * 27 | **步骤** 28 | 1. [配置一个简单的KVM（非直通）](VFIO/Setting%20up%20a%20basic%20KVM%20cn.md) 29 | 2. [Libvirt 钩子](VFIO/Libvirt%20Hooks%20cn.md) 30 | 3. [QEMU和Libvirt的配置](VFIO/Configure%20Libvirt%20cn.md) 31 | 4. [显卡直通](VFIO/Setting%20up%20Passthrough%20cn.md) 32 | 5. [some other things](VFIO/Debugging%20and%20other%20features.md) 33 | 34 | --- 35 | 36 | 还在编辑中，请考虑帮我更新添加更多内容！ 37 | 38 | 本人是一名半工半读的学生，如果你觉得我的视频和这些教程有用，请支持我！ 39 | 40 | 可以通过Paypal捐赠： 41 | 42 | [![Paypal](https://www.paypalobjects.com/zh_XC/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/donate/?hosted_button_id=HVU7NRQMZGMNN) 43 | 44 | 或者可以选择在在微信上给我打赏（标注VFIO），非常感谢！ 45 | 46 | ![微信支付码](resources/wechatpay.jpg) 47 | 48 | --- 49 | 50 | **其它资源** 51 | - Arch Wiki [PCI passthrough](https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF) 52 | - GitLab [RisingPrismTV's script](https://gitlab.com/risingprismtv/single-gpu-passthrough) 53 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Single GPU Passthrough on Ryzen CPU 2 | 3 | https://liucreator.gitlab.io/zh/posts/0x0b-single-gpu-passthrough/main/ 4 | 5 | [中文版本](README-cn.md)
6 | These are my steps to set up QEMU/KVM with GPU passthrough as of 2021 March, 7 | it works for me as I tried many times on different systems. 8 | 9 | You might need to change a few things, and feel free to ask me if you have any questions! 10 | 11 | My current setup is on Ryzen CPU with Radeon GPU, and sorry my English is not the best. 12 | 13 | - [![Followers](https://bilistats.lonelyion.com/followers?uid=589560036)](https://space.bilibili.com/589560036/channel/seriesdetail?sid=2031728) 14 | - [![YouTube Channel Subscribers](https://img.shields.io/youtube/channel/subscribers/UCKXFTVfYRA8Ho71bAT5tfVA?style=social)](https://www.youtube.com/channel/UCKXFTVfYRA8Ho71bAT5tfVA?sub_confirmation=1) 15 | 16 | **My Specs:** 17 | - CPU: AMD Ryzen 3900x 12 cores 18 | - GPU: MSI Radeon RX 560 4GT LP OC 19 | - RAM: 32GB 3200Mhz dual channels 20 | - OS: Arch Linux 5.11 21 | - DE: KDE Plasma 5.21 22 | 23 | 24 | * * * 25 | **Procedure:** 26 | 1. [Setting up an basic KVM on linux without VFIO](/VFIO/Setting%20up%20a%20basic%20KVM.md) 27 | 2. [Libvirt Hooks and Scripts](/VFIO/Libvirt%20Hooks.md) 28 | 3. [QEMU+Libvirt Configuration](/VFIO/Configure%20Libvirt.md) 29 | 4. [GPU passthrough](/VFIO/Setting%20up%20Passthrough.md) 30 | 5. [some other things](/VFIO/Debugging%20and%20other%20features.md) 31 | 32 | * * * 33 | 34 | --- 35 | 36 | Please considering of supporting me if you find my guides or videos helpful! 37 | 38 | [![Paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/donate/?hosted_button_id=A47HPSXTPD5HE) 39 | 40 | --- 41 | 42 | **Credits:** 43 | - Arch Wiki [PCI passthrough](https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF) 44 | - GitLab [RisingPrismTV's script](https://gitlab.com/risingprismtv/single-gpu-passthrough) 45 | - SubReddit [r/VFIO](https://www.reddit.com/r/VFIO/) 46 | -------------------------------------------------------------------------------- /VFIO/Configure Libvirt cn.md: -------------------------------------------------------------------------------- 1 | # 配置Libvirt 2 | 3 | **编辑libvirt.conf**
4 | `sudo nano /etc/libvirt/libvirtd.conf` 5 | 6 | 找到这两行，把前面的`#`删掉： 7 | ``` 8 | unix_sock_group = "libvirt" 9 | unix_sock_rw_perms = "0770" 10 | ``` 11 | 可以选择把这两行添加到文件的最后面（日志文件）: 12 | ``` 13 | log_filters="1:qemu" 14 | log_outputs="1:file:/var/log/libvirt/libvirtd.log" 15 | ``` 16 | * * * 17 | **编辑qemu.conf**
18 | `sudo nano /etc/libvirt/qemu.conf`
19 | 20 | 把 `#user = "root"` 改成 `user = "yourusername"`,
21 | `#group = "root"` 改成 `group = "libvirt"`
22 | 23 | 把自己添加到libvirt组里面：
24 | `sudo usermod -a -G libvirt 用户名` 25 | * * * 26 | 27 | **重启电脑或者重启libvirt** 28 | ``` 29 | sudo systemctl restart libvirtd.service 30 | sudo systemctl restart virtlogd.socket 31 | ``` 32 | 33 | [教程主页](../README-cn.md) -------------------------------------------------------------------------------- /VFIO/Configure Libvirt.md: -------------------------------------------------------------------------------- 1 | # Configuring Libvirt 2 | 3 | edit libvirt.conf:
4 | `sudo nano /etc/libvirt/libvirtd.conf` 5 | 6 | uncomment these two lines by removing the `#` in front of them: 7 | ``` 8 | unix_sock_group = "libvirt" 9 | unix_sock_rw_perms = "0770" 10 | ``` 11 | 12 | add these two lines to the end of the file for logs: 13 | ``` 14 | log_filters="1:qemu" 15 | log_outputs="1:file:/var/log/libvirt/libvirtd.log" 16 | ``` 17 | * * * 18 | edit qemu.conf: 19 | 20 | change `#user = "root"` to `user = "yourusername"` 21 | change `#group = "root"` to `group = "libvirt"` 22 | 23 | add yourself to the libvirt group:
24 | `sudo usermod -a -G libvirt yourusername` 25 | * * * 26 | 27 | **Restart computer or restart libvirt:** 28 | ``` 29 | sudo systemctl restart libvirtd.service 30 | sudo systemctl restart virtlogd.socket 31 | ``` 32 | -------------------------------------------------------------------------------- /VFIO/Debugging and other features.md: -------------------------------------------------------------------------------- 1 | **You can find Logs at:**
2 | **win10.log** in /var/log/libvirt/qemu
3 | **custom_hooks.log** in/var/log/libvirt
4 | **libvirtd.log** in /var/log/libvirt/
5 | also check `journalctl` and `dmesg` if necessary 6 | 7 | * * * 8 | **to do:** 9 | - [ ] CPU performance tweak 10 | - [ ] virtio drivers 11 | - [ ] sound passsthrough 12 | - [ ] physical drive passthrough 13 | - [ ] game controller passthrough 14 | - [ ] nested VM in guest 15 | 16 | * * * 17 | **Bugs and fixes**
18 | you can sent me any issues you have, if we have a solution I can post them here. 19 | 20 | 21 | - [ ] -------------------------------------------------------------------------------- /VFIO/Libvirt Hooks cn.md: -------------------------------------------------------------------------------- 1 | # 配置Libvirt脚本 2 | [英伟达显卡请看这里](libvirt_hooks_nvidia-cn.md) 3 | 4 | 在Libirt里面创建一个hooks文件夹
5 | `sudo mkdir /etc/libvirt/hooks` 6 | 7 | 把[这里](../libvirt-hooks)的三个文件放到hooks文件夹里面 8 | 9 | 都变成可执行的文件
10 | `sudo chmod +x /etc/libvirt/hooks/*`
11 | 12 | 把两个脚本软链接到根目录的bin文件夹 13 | ``` 14 | sudo ln -s /etc/libvirt/hooks/vfio-startup.sh /bin/vfio-startup.sh 15 | sudo ln -s /etc/libvirt/hooks/vfio-teardown.sh /bin/vfio-teardown.sh 16 | ``` 17 | 18 | * * * 19 | **防止Libvirt在运行时休眠**
20 | `sudo nano /etc/systemd/system/libvirt-nosleep@.service` 21 | 22 | 在libvirt-nosleep@.service里面添加： 23 | ``` 24 | [Unit] 25 | Description=Preventing sleep while libvirt domain "%i" is running 26 | 27 | [Service] 28 | Type=simple 29 | ExecStart=/usr/bin/systemd-inhibit --what=sleep --why="Libvirt domain \"%i\" is running" --who=%U --mode=block sleep infinity 30 | ``` 31 | 更改权限：
32 | `sudo chmod 644 -R /etc/systemd/system/libvirt-nosleep@.service`
33 | 34 | 变成系统文件
35 | `sudo chown root:root /etc/systemd/system/libvirt-nosleep@.service` 36 | 37 | [教程主页](../README-cn.md) -------------------------------------------------------------------------------- /VFIO/Libvirt Hooks.md: -------------------------------------------------------------------------------- 1 | # Setup libvirt hooks 2 | ``` 3 | sudo mkdir /etc/libvirt/hooks 4 | sudo nano /etc/libvirt/hooks/qemu 5 | ``` 6 | 7 | in qemu: 8 | ``` 9 | #!/bin/bash 10 | 11 | OBJECT="$1" 12 | OPERATION="$2" 13 | 14 | if [[ $OBJECT == "win10" ]]; then 15 | case "$OPERATION" in 16 | "prepare") 17 | systemctl start libvirt-nosleep@"$OBJECT" 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 18 | /bin/vfio-startup.sh 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 19 | ;; 20 | 21 | "release") 22 | systemctl stop libvirt-nosleep@"$OBJECT" 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 23 | /bin/vfio-teardown.sh 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 24 | ;; 25 | esac 26 | fi 27 | ``` 28 | then make it executable:
29 | `sudo chmod +x /etc/libvirt/hooks/qemu` 30 | 31 | * * * 32 | **edit start up script:**
33 | `sudo nano /etc/libvirt/hooks/vfio-startup.sh` 34 | 35 | in vfio-startup.sh: 36 | ``` 37 | #!/bin/bash 38 | # Helpful to read output when debugging 39 | set -x 40 | 41 | long_delay=10 42 | medium_delay=5 43 | short_delay=1 44 | echo "Beginning of startup!" 45 | 46 | function stop_display_manager_if_running { 47 | # Stop dm using systemd 48 | if command -v systemctl; then 49 | if systemctl is-active --quiet "$1.service" ; then 50 | echo $1 >> /tmp/vfio-store-display-manager 51 | systemctl stop "$1.service" 52 | fi 53 | 54 | while systemctl is-active --quiet "$1.service" ; do 55 | sleep "${medium_delay}" 56 | done 57 | 58 | return 59 | fi 60 | 61 | # Stop dm using runit 62 | if command -v sv; then 63 | if sv status $1 ; then 64 | echo $1 >> /tmp/vfio-store-display-manager 65 | sv stop $1 66 | fi 67 | fi 68 | } 69 | 70 | 71 | # Stop currently running display manager 72 | if test -e "/tmp/vfio-store-display-manager" ; then 73 | rm -f /tmp/vfio-store-display-manager 74 | fi 75 | 76 | stop_display_manager_if_running sddm 77 | stop_display_manager_if_running gdm 78 | stop_display_manager_if_running lightdm 79 | stop_display_manager_if_running lxdm 80 | stop_display_manager_if_running xdm 81 | stop_display_manager_if_running mdm 82 | stop_display_manager_if_running display-manager 83 | 84 | sleep "${medium_delay}" 85 | 86 | # Unbind VTconsoles if currently bound (adapted from https://www.kernel.org/doc/Documentation/fb/fbcon.txt) 87 | if test -e "/tmp/vfio-bound-consoles" ; then 88 | rm -f /tmp/vfio-bound-consoles 89 | fi 90 | for (( i = 0; i < 16; i++)) 91 | do 92 | if test -x /sys/class/vtconsole/vtcon${i}; then 93 | if [ `cat /sys/class/vtconsole/vtcon${i}/name | grep -c "frame buffer"` \ 94 | = 1 ]; then 95 | echo 0 > /sys/class/vtconsole/vtcon${i}/bind 96 | echo "Unbinding console ${i}" 97 | echo $i >> /tmp/vfio-bound-consoles 98 | fi 99 | fi 100 | done 101 | 102 | # Unbind EFI-Framebuffer 103 | if test -e "/tmp/vfio-is-nvidia" ; then 104 | rm -f /tmp/vfio-is-nvidia 105 | fi 106 | 107 | if lsmod | grep "nvidia" &> /dev/null ; then 108 | echo "true" >> /tmp/vfio-is-nvidia 109 | echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind 110 | fi 111 | 112 | echo "End of startup!" 113 | ``` 114 | make it executable:
115 | ```sudo chmod +x /etc/libvirt/hooks/vfio-startup.sh``` 116 | 117 | soft link it to /bin:
118 | `sudo ln -s /etc/libvirt/hooks/vfio-startup.sh /bin/vfio-startup.sh` 119 | 120 | * * * 121 | **edit tear down script:**
122 | `sudo nano /etc/libvirt/hooks/vfio-teardown.sh` 123 | 124 | in vfio-teardown.sh: 125 | ``` 126 | #!/bin/bash 127 | set -x 128 | 129 | echo "Beginning of teardown!" 130 | 131 | sleep 10 132 | 133 | # Restart Display Manager 134 | input="/tmp/vfio-store-display-manager" 135 | while read displayManager; do 136 | if command -v systemctl; then 137 | systemctl start "$displayManager.service" 138 | else 139 | if command -v sv; then 140 | sv start $displayManager 141 | fi 142 | fi 143 | done < "$input" 144 | 145 | # Rebind VT consoles (adapted from https://www.kernel.org/doc/Documentation/fb/fbcon.txt) 146 | input="/tmp/vfio-bound-consoles" 147 | while read consoleNumber; do 148 | if test -x /sys/class/vtconsole/vtcon${consoleNumber}; then 149 | if [ `cat /sys/class/vtconsole/vtcon${consoleNumber}/name | grep -c "frame buffer"` \ 150 | = 1 ]; then 151 | echo "Rebinding console ${consoleNumber}" 152 | echo 1 > /sys/class/vtconsole/vtcon${consoleNumber}/bind 153 | fi 154 | fi 155 | done < "$input" 156 | 157 | # Rebind framebuffer for nvidia 158 | if test -e "/tmp/vfio-is-nvidia" ; then 159 | echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/bind 160 | fi 161 | 162 | echo "End of teardown!" 163 | ``` 164 | make it executable: 165 | `sudo chmod +x /etc/libvirt/hooks/vfio-teardown.sh` 166 | 167 | soft link it to /bin: 168 | `sudo ln -s /etc/libvirt/hooks/vfio-teardown.sh /bin/vfio-teardown.sh` 169 | 170 | * * * 171 | **edit libvirt no sleep service**
172 | `sudo nano /etc/systemd/system/libvirt-nosleep@.service` 173 | 174 | in libvirt-nosleep@.service: 175 | ``` 176 | [Unit] 177 | Description=Preventing sleep while libvirt domain "%i" is running 178 | 179 | [Service] 180 | Type=simple 181 | ExecStart=/usr/bin/systemd-inhibit --what=sleep --why="Libvirt domain \"%i\" is running" --who=%U --mode=block sleep infinity 182 | ``` 183 | change permission:
184 | `sudo chmod 644 -R /etc/systemd/system/libvirt-nosleep@.service` 185 | 186 | change ownership:
187 | `sudo chown root:root /etc/systemd/system/libvirt-nosleep@.service` 188 | -------------------------------------------------------------------------------- /VFIO/Setting up Passthrough cn.md: -------------------------------------------------------------------------------- 1 | # 直通显卡 2 | 3 | **导出你的显卡的VGA BIOS** 4 | - 最简单的办法: 如果电脑是Windows双系统，可以用[GPU-Z](https://www.techpowerup.com/gpuz/) 5 | - 去[Tech Power Up](https://www.techpowerup.com/vgabios/)下载一个 6 | - 在Linux上使用[NVIDIA](https://www.techpowerup.com/download/nvidia-nvflash/) or [AMD](https://www.techpowerup.com/download/ati-atiflash/) 7 | - 其它命令行方法[Arch Wiki](https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#UEFI_(OVMF)_compatibility_in_VBIOS) 8 | 9 | **VBIOS补丁- 只有英伟达显卡需要** 10 | 1. 用十六进制编辑器打开导出的rom(我个人喜欢Bless) 11 | 2. 查找字符"VIDEO"
12 | 把`VIDEO`前面第一个大写`U.`(十六进制是55)之前的所有的headers都删掉. 13 | 4. 保存到某个地方
14 | *我在Arch上保存到个人主目录文件夹是没问题的*
15 | Arch、Fedora可以保存到`/var/lib/libvirt/vbios`里面，
16 | Ubuntu/OpenSUSE/Mint(用AppArmour的发行版)可以保存到`/usr/share/vgabios`里面。 17 | 5. 修改权限：
18 | `sudo chmod -R 660 vbios文件.rom` 19 | 6. 更改拥有者 20 | `sudo chown 你的用户名:users vbios文件.rom` 21 | 22 | * * * 23 | 24 | **配置qemu虚拟机** 25 | 1. 打开虚拟机管理器，把显卡的所有pci部分都添加到虚拟机里面。 26 | 2. 启用XML编辑, 在显卡的所有pci设备里面, 在``之后添加:
27 | `` 28 | 3. 把信道，usb转接，显卡qxl,触摸板那些东西删掉 29 | 4. 添加你的USB设备，比如鼠标、键盘和USB耳机。 30 | 31 | **绕过英伟达Geforce显卡防虚拟化检测（恶名昭著的错误43）**
32 | *2021四月之后好像不需要了*
33 | 编辑XML: 34 | - 在``之前添加（value应该等于8-12个字母）:
35 | ``
36 | AMD显卡可以用"`AuthenticAMD`"作为id但不需要，因为AMD不会拦着你显卡虚拟化。 37 | 38 | - 在`` 和 `` 之间添加: 39 | ``` 40 | 41 | 42 | 43 | ``` 44 | 那么现在试试你的虚拟机吧！第一次开机可能会有点慢，因为要装显卡驱动，不过等等就可以了。 45 | 46 | [教程主页](../README-cn.md) -------------------------------------------------------------------------------- /VFIO/Setting up Passthrough.md: -------------------------------------------------------------------------------- 1 | # Setting up passthrough 2 | 3 | **dump GPU VBIOS:** 4 | - easiest way: use GPU-Z on a temporary windows 10 5 | - or download it from [tech power up vga bios collection](https://www.techpowerup.com/vgabios/) 6 | - dump in linux using [NVIDIA](https://www.techpowerup.com/download/nvidia-nvflash/) or [AMD](https://www.techpowerup.com/download/ati-atiflash/) 7 | - or by command line 8 | 9 | **patch VBIOS - NVIDIA only** 10 | 1. open the rom with a hex editor (I use Okteta or Bless) 11 | 2. look for CHAR "video" 12 | Remove everything before `U.`(55 in hex). 13 | 4. save it somewhere 14 | *home directory works for Arch Linux* 15 | Arch/Fedora: `sudo mkdir /var/lib/libvirt/vbios` 16 | Ubuntu/OpenSUSE/Mint(Distros with AppArmour): `sudo mkdir /usr/share/vgabios` 17 | 18 | 5. add read and execute permission 19 | `sudo chmod -R 660 patched.rom` 20 | 6. own it 21 | `sudo chown yourusername:users patched.rom` 22 | 23 | **configure VM** 24 | 1. go to virt-manager and add all pcie part of the graphic card to vm 25 | 2. enable XML editing, in pci device, after `` add: 26 | `` 27 | 3. remove things like channel spice, tablet, usb redirection, video QXL etc. 28 | 4. add usb devices like mouse, keyboard, usb headsets 29 | 30 | **Bypass Nvidia VM detection(not required anymore as of April 2021?)** 31 | Edit XML: 32 | - add the following before ``, value should be 8 to 12 characters: 33 | `` 34 | AMD gpu can use id "`AuthenticAMD`". 35 | 36 | - inside of `` and `` add: 37 | ``` 38 | 39 | 40 | 41 | ``` 42 | -------------------------------------------------------------------------------- /VFIO/Setting up a basic KVM cn.md: -------------------------------------------------------------------------------- 1 | # 配置一个简单的KVM 2 | 3 | **启用IOMMU**
4 | 比如Systemd启动器：
5 | `nano /boot/loader/entries/arch.conf`
6 | 把 `amd_iommu=on iommu=pt` 添加到options后面
7 | 保存，重启系统 8 | 9 | **检查IOMMU是否已启用**
10 | `sudo dmesg | grep -e DMAR -e IOMMU`
11 | 找找有没有amd_IOMMU:Detected" 12 | 13 | **检查IOMMU组是否有效**
14 | [iommuamd.sh](../resources/iommuamd.sh) 15 | 运行这个脚本，看看显卡是不是在自己的一个组里面。 16 | ``` 17 | IOMMU group 16 18 | 0a:00.0 VGA compatible controller \[0300\]: Advanced Micro Devices, Inc. \[AMD/ATI\] Baffin \[Radeon RX 550 640SP / RX 560/560X\] \[1002:67ff\] (rev cf) 19 | Driver: amdgpu 20 | 0a:00.1 Audio device \[0403\]: Advanced Micro Devices, Inc. \[AMD/ATI\] Baffin HDMI/DP Audio \[Radeon RX 550 640SP / RX 560/560X\] \[1002:aae0\] 21 | Driver: snd\_hda\_intel 22 | 23 | ``` 24 | 复制一下，先保存下来。 25 | 26 | **添加VFIO到内核模块**
27 | `sudo nano /etc/modprobe.d/vfio.conf`
28 | 在vfio.conf里面添加
29 | ``` 30 | options vfio-pci ids=1002:67ff,1002:aae0 31 | options vfio-pci disable_idle_d3=1 32 | options vfio-pci disable_vga=1 33 | ``` 34 | *ids=后面应该是刚才查看的显卡的那些id,每个值之间用`,`隔开* 35 | 36 | 37 | **安装需要的软件
38 | `sudo pacman -S qemu libvirt edk2-ovmf virt-manager dnsmasq ebtables iptables bridge-utils` 39 | 40 | **开机启用服务**
41 | ``` 42 | sudo systemctl enable libvirtd 43 | sudo systemctl enable virtlogd.socket 44 | ``` 45 | 46 | **开始Libvirt**
47 | ``` 48 | "sudo systemctl start libvirtd" 49 | `sudo systemctl start virtlogd.socket 50 | ``` 51 | 52 | **启用虚拟网络**
53 | ``` 54 | sudo virsh net-start default 55 | sudo virsh net-autostart default 56 | ``` 57 | 58 | **先打开Virt-Manager安装一个没有直通的虚拟机，然后去下一步** 59 | 60 | [教程主页](../README-cn.md) -------------------------------------------------------------------------------- /VFIO/Setting up a basic KVM.md: -------------------------------------------------------------------------------- 1 | # Setting up a Basic KVM 2 | 3 | **Enable IOMMU in Bootloader**
4 | `nano /boot/loader/entries/arch.conf` 5 | add `amd_iommu=on iommu=pt` to the end of the options 6 | Save and Reboot 7 | 8 | **check IOMMU**
9 | `sudo dmesg | grep -e DMAR -e IOMMU` 10 | look for amd_IOMMU:Detected" 11 | 12 | **check PCI iommu groups**
13 | [iommuamd.sh](../_resources/5276daf6236346479e15f96ce0afe812.sh) 14 | run this script, look for your GPU, make sure they are in their own group 15 | ``` 16 | IOMMU group 16 17 | 0a:00.0 VGA compatible controller \[0300\]: Advanced Micro Devices, Inc. \[AMD/ATI\] Baffin \[Radeon RX 550 640SP / RX 560/560X\] \[1002:67ff\] (rev cf) 18 | Driver: amdgpu 19 | 0a:00.1 Audio device \[0403\]: Advanced Micro Devices, Inc. \[AMD/ATI\] Baffin HDMI/DP Audio \[Radeon RX 550 640SP / RX 560/560X\] \[1002:aae0\] 20 | Driver: snd\_hda\_intel 21 | 22 | ``` 23 | copy it and save it somewhere. 24 | 25 | **add modprobe module:**
26 | `sudo nano /etc/modprobe.d/vfio.conf`
27 | in vfio.conf add:
28 | ``` 29 | options vfio-pci ids=1002:67ff,1002:aae0 30 | options vfio-pci disable_idle_d3=1 31 | options vfio-pci disable_vga=1 32 | ``` 33 | *ids should be your pci parts ids with `,` in between* 34 | 35 | 36 | **Install required packages**
37 | `sudo pacman -S qemu libvirt edk2-ovmf virt-manager dnsmasq ebtables iptables bridge-utils` 38 | 39 | **Enable them in init system**
40 | ``` 41 | sudo systemctl enable libvirtd 42 | sudo systemctl enable virtlogd.socket 43 | ``` 44 | 45 | **Start services**
46 | ``` 47 | "sudo systemctl start libvirtd" 48 | `sudo systemctl start virtlogd.socket 49 | ``` 50 | 51 | **enable virtual network**
52 | ``` 53 | sudo virsh net-start default 54 | sudo virsh net-autostart default 55 | ``` 56 | 57 | **open virt-manager and setting up an basic virtual machine without passthrough** 58 | -------------------------------------------------------------------------------- /VFIO/libvirt_hooks_nvidia-cn.md: -------------------------------------------------------------------------------- 1 | # 配置Libvirt脚本（针对宿主机在使用闭源驱动的英伟达显卡） 2 | [AMD显卡请看这里](Libvirt%20Hooks%20cn.md) 3 | 4 | 在Libirt里面创建一个hooks文件夹
5 | `sudo mkdir /etc/libvirt/hooks` 6 | 7 | 把[这里](../libvirt-nvidia-hooks)的三个文件放到hooks文件夹里面 8 | 9 | 记得之前保存的这个吗？（我这拿AMD显卡举例子） 10 | ``` 11 | IOMMU group 16 12 | 0a:00.0 VGA compatible controller \[0300\]: Advanced Micro Devices, Inc. \[AMD/ATI\] Baffin \[Radeon RX 550 640SP / RX 560/560X\] \[1002:67ff\] (rev cf) 13 | Driver: amdgpu 14 | 0a:00.1 Audio device \[0403\]: Advanced Micro Devices, Inc. \[AMD/ATI\] Baffin HDMI/DP Audio \[Radeon RX 550 640SP / RX 560/560X\] \[1002:aae0\] 15 | Driver: snd\_hda\_intel 16 | 17 | ``` 18 | 19 | 看到前面的`0a:00.0`和`0a:00.1`吗？把它看成`AB:CD.E`. 20 | 打开**vfio-startup.sh**和**vfio-teardown.sh**, 21 | 把里面的`pci_0000_AB_CD_E`按字母替换了，你的显卡有几个部分，就需要几行。 22 | 23 | 然后要根据你的显示管理器，把`your-display-manager` 改成你的显示管理器的服务，我用的是SDDM 24 | 不知道的可以尝试用`file /etc/systemd/system/display-manager.service` 命令查看。 25 | 26 | 比如这是我的**vfio-startup.sh**，从： 27 | ``` 28 | ... 29 | systemctl stop your-display-manager 30 | ... 31 | virsh nodedev-detach pci_0000_AB_CD_E 32 | ... 33 | ``` 34 | 35 | 改成了： 36 | ``` 37 | ... 38 | systemctl stop sddm 39 | ... 40 | virsh nodedev-detach pci_0000_0a_00_0 41 | virsh nodedev-detach pci_0000_0a_00_1 42 | ... 43 | ``` 44 | 45 | 46 | 别忘了也要改**vfio-teardown.sh**! 47 | 48 | --- 49 | 50 | 都变成可执行的文件
51 | `sudo chmod +x /etc/libvirt/hooks/*`
52 | 53 | 把两个脚本软链接到根目录的bin文件夹 54 | ``` 55 | sudo ln -s /etc/libvirt/hooks/vfio-startup.sh /bin/vfio-startup.sh 56 | sudo ln -s /etc/libvirt/hooks/vfio-teardown.sh /bin/vfio-teardown.sh 57 | ``` 58 | 59 | * * * 60 | **防止Libvirt在运行时休眠**
61 | `sudo nano /etc/systemd/system/libvirt-nosleep@.service` 62 | 63 | 在libvirt-nosleep@.service里面添加： 64 | ``` 65 | [Unit] 66 | Description=Preventing sleep while libvirt domain "%i" is running 67 | 68 | [Service] 69 | Type=simple 70 | ExecStart=/usr/bin/systemd-inhibit --what=sleep --why="Libvirt domain \"%i\" is running" --who=%U --mode=block sleep infinity 71 | ``` 72 | 更改权限：
73 | `sudo chmod 644 -R /etc/systemd/system/libvirt-nosleep@.service`
74 | 75 | 变成系统文件
76 | `sudo chown root:root /etc/systemd/system/libvirt-nosleep@.service` 77 | 78 | [教程主页](../README-cn.md) -------------------------------------------------------------------------------- /libvirt-hooks/qemu: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | OBJECT="$1" 4 | OPERATION="$2" 5 | 6 | if [[ $OBJECT == "win10" ]]; then 7 | case "$OPERATION" in 8 | "prepare") 9 | systemctl start libvirt-nosleep@"$OBJECT" 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 10 | /bin/vfio-startup.sh 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 11 | ;; 12 | 13 | "release") 14 | systemctl stop libvirt-nosleep@"$OBJECT" 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 15 | /bin/vfio-teardown.sh 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 16 | ;; 17 | esac 18 | fi -------------------------------------------------------------------------------- /libvirt-hooks/vfio-startup.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Helpful to read output when debugging 3 | set -x 4 | 5 | long_delay=10 6 | medium_delay=5 7 | short_delay=1 8 | echo "Beginning of startup!" 9 | 10 | function stop_display_manager_if_running { 11 | # Stop dm using systemd 12 | if command -v systemctl; then 13 | if systemctl is-active --quiet "$1.service" ; then 14 | echo $1 >> /tmp/vfio-store-display-manager 15 | systemctl stop "$1.service" 16 | fi 17 | 18 | while systemctl is-active --quiet "$1.service" ; do 19 | sleep "${medium_delay}" 20 | done 21 | 22 | return 23 | fi 24 | 25 | # Stop dm using runit 26 | if command -v sv; then 27 | if sv status $1 ; then 28 | echo $1 >> /tmp/vfio-store-display-manager 29 | sv stop $1 30 | fi 31 | fi 32 | } 33 | 34 | 35 | # Stop currently running display manager 36 | if test -e "/tmp/vfio-store-display-manager" ; then 37 | rm -f /tmp/vfio-store-display-manager 38 | fi 39 | 40 | stop_display_manager_if_running sddm 41 | stop_display_manager_if_running gdm 42 | stop_display_manager_if_running lightdm 43 | stop_display_manager_if_running lxdm 44 | stop_display_manager_if_running xdm 45 | stop_display_manager_if_running mdm 46 | stop_display_manager_if_running display-manager 47 | 48 | sleep "${medium_delay}" 49 | 50 | # Unbind VTconsoles if currently bound (adapted from https://www.kernel.org/doc/Documentation/fb/fbcon.txt) 51 | if test -e "/tmp/vfio-bound-consoles" ; then 52 | rm -f /tmp/vfio-bound-consoles 53 | fi 54 | for (( i = 0; i < 16; i++)) 55 | do 56 | if test -x /sys/class/vtconsole/vtcon${i}; then 57 | if [ `cat /sys/class/vtconsole/vtcon${i}/name | grep -c "frame buffer"` \ 58 | = 1 ]; then 59 | echo 0 > /sys/class/vtconsole/vtcon${i}/bind 60 | echo "Unbinding console ${i}" 61 | echo $i >> /tmp/vfio-bound-consoles 62 | fi 63 | fi 64 | done 65 | 66 | # Unbind EFI-Framebuffer 67 | if test -e "/tmp/vfio-is-nvidia" ; then 68 | rm -f /tmp/vfio-is-nvidia 69 | fi 70 | 71 | if lsmod | grep "nvidia" &> /dev/null ; then 72 | echo "true" >> /tmp/vfio-is-nvidia 73 | echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind 74 | fi 75 | 76 | echo "End of startup!" -------------------------------------------------------------------------------- /libvirt-hooks/vfio-teardown.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -x 3 | 4 | echo "Beginning of teardown!" 5 | 6 | sleep 10 7 | 8 | # Restart Display Manager 9 | input="/tmp/vfio-store-display-manager" 10 | while read displayManager; do 11 | if command -v systemctl; then 12 | systemctl start "$displayManager.service" 13 | else 14 | if command -v sv; then 15 | sv start $displayManager 16 | fi 17 | fi 18 | done < "$input" 19 | 20 | # Rebind VT consoles (adapted from https://www.kernel.org/doc/Documentation/fb/fbcon.txt) 21 | input="/tmp/vfio-bound-consoles" 22 | while read consoleNumber; do 23 | if test -x /sys/class/vtconsole/vtcon${consoleNumber}; then 24 | if [ `cat /sys/class/vtconsole/vtcon${consoleNumber}/name | grep -c "frame buffer"` \ 25 | = 1 ]; then 26 | echo "Rebinding console ${consoleNumber}" 27 | echo 1 > /sys/class/vtconsole/vtcon${consoleNumber}/bind 28 | fi 29 | fi 30 | done < "$input" 31 | 32 | # Rebind framebuffer for nvidia 33 | if test -e "/tmp/vfio-is-nvidia" ; then 34 | echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/bind 35 | fi 36 | 37 | echo "End of teardown!" -------------------------------------------------------------------------------- /libvirt-nvidia-hooks/qemu: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | OBJECT="$1" 4 | OPERATION="$2" 5 | 6 | if [[ $OBJECT == "win10" ]]; then 7 | case "$OPERATION" in 8 | "prepare") 9 | systemctl start libvirt-nosleep@"$OBJECT" 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 10 | /bin/vfio-startup.sh 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 11 | ;; 12 | 13 | "release") 14 | systemctl stop libvirt-nosleep@"$OBJECT" 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 15 | /bin/vfio-teardown.sh 2>&1 | tee -a /var/log/libvirt/custom_hooks.log 16 | ;; 17 | esac 18 | fi -------------------------------------------------------------------------------- /libvirt-nvidia-hooks/vfio-startup.sh: -------------------------------------------------------------------------------- 1 | set -x 2 | 3 | systemctl stop your-display-manager 4 | 5 | echo 0 > /etc/class/vtconsole/vtcon0/bind 6 | echo 0 > /etc/class/vtconsole/vtcon1/bind 7 | 8 | echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind 9 | 10 | sleep 15 11 | 12 | modprobe -r nvidia_drm 13 | modprobe -r nvidia_uvm 14 | modprobe -r nvidia_modeset 15 | modprobe -r drm_kms_helper 16 | modprobe -r nvidia 17 | modprobe -r i2c_nvidia_gpu 18 | modprobe -r drm 19 | 20 | virsh nodedev-detach pci_0000_AB_CD_E 21 | 22 | modprobe vfio_pci 23 | modprobe vfio 24 | modprobe vfio_iommu_type1 25 | modprobe vfio_virqfd -------------------------------------------------------------------------------- /libvirt-nvidia-hooks/vfio-teardown.sh: -------------------------------------------------------------------------------- 1 | set -x 2 | 3 | sleep 10 4 | 5 | modprobe -r vfio_pci 6 | modprobe -r vfio 7 | modprobe -r vfio_iommu_type1 8 | modprobe -r vfio_virqfd 9 | 10 | virsh nodedev-reattach pci_0000_AB_CD_E 11 | 12 | echo 1 > /etc/class/vtconsole/vtcon0/bind 13 | echo 1 > /etc/class/vtconsole/vtcon1/bind 14 | 15 | echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind 16 | 17 | modprobe nvidia 18 | modprobe nvidia_modeset 19 | modprobe nvidia_uvm 20 | modprobe nvidia_drm 21 | modprobe drm_kms_helper 22 | modprobe i2c_nvidia_gpu 23 | modprobe drm 24 | 25 | sleep 10 26 | 27 | systemctl start your-display-manager -------------------------------------------------------------------------------- /resources/iommuamd.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | useColors=true 4 | usePager=true 5 | 6 | usage() { 7 | echo "\ 8 | Usage: $(basename $0) [OPTIONS] 9 | Shows information about IOMMU groups relevant for working with PCI-passthrough 10 | 11 | -c -C enables/disables colored output, respectively 12 | -p -P enables/disables pager (less), respectively 13 | 14 | -h display this help message" 15 | } 16 | 17 | color() { 18 | if ! $useColors; then 19 | cat 20 | return 21 | fi 22 | 23 | rset=$'\E[0m' 24 | case "$1" in 25 | black) colr=$'\E[22;30m' ;; 26 | red) colr=$'\E[22;31m' ;; 27 | green) colr=$'\E[22;32m' ;; 28 | yellow) colr=$'\E[22;33m' ;; 29 | blue) colr=$'\E[22;34m' ;; 30 | magenta) colr=$'\E[22;35m' ;; 31 | cyan) colr=$'\E[22;36m' ;; 32 | white) colr=$'\E[22;37m' ;; 33 | intenseBlack) colr=$'\E[01;30m' ;; 34 | intenseRed) colr=$'\E[01;31m' ;; 35 | intenseGreen) colr=$'\E[01;32m' ;; 36 | intenseYellow) colr=$'\E[01;33m' ;; 37 | intenseBlue) colr=$'\E[01;34m' ;; 38 | intenseMagenta) colr=$'\E[01;35m' ;; 39 | intenseCyan) colr=$'\E[01;36m' ;; 40 | intenseWhite) colr=$'\E[01;37m' ;; 41 | esac 42 | 43 | sed "s/^/$colr/;s/\$/$rset/" 44 | } 45 | 46 | indent() { 47 | sed 's/^/\t/' 48 | } 49 | 50 | pager() { 51 | if $usePager; then 52 | less -SR 53 | else 54 | cat 55 | fi 56 | } 57 | 58 | while getopts cCpPh opt; do 59 | case $opt in 60 | c) 61 | useColors=true 62 | ;; 63 | C) 64 | useColors=false 65 | ;; 66 | p) 67 | usePager=true 68 | ;; 69 | P) 70 | usePager=false 71 | ;; 72 | h) 73 | usage 74 | exit 75 | ;; 76 | esac 77 | done 78 | 79 | iommuGroups=$(find '/sys/kernel/iommu_groups/' -maxdepth 1 -mindepth 1 -type d) 80 | 81 | if [ -z "$iommuGroups" ]; then 82 | echo "No IOMMU groups found. Are you sure IOMMU is enabled?" 83 | exit 84 | fi 85 | 86 | for iommuGroup in $iommuGroups; do 87 | echo "IOMMU group $(basename "$iommuGroup")" | color red 88 | 89 | for device in $(ls -1 "$iommuGroup/devices/"); do 90 | devicePath="$iommuGroup/devices/$device/" 91 | 92 | # Print pci device 93 | lspci -nns "$device" | color blue 94 | 95 | # Print drivers 96 | driverPath=$(readlink "$devicePath/driver") 97 | if [ -z "$driverPath" ]; then 98 | echo "Driver: none" 99 | else 100 | echo "Driver: $(basename $driverPath)" 101 | fi | indent | color cyan 102 | 103 | # Print usb devices 104 | usbBuses=$(find $devicePath -maxdepth 2 -path '*usb*/busnum') 105 | for usb in $usbBuses; do 106 | echo 'Usb bus:' | color cyan 107 | lsusb -s $(cat "$usb"): | indent | color green 108 | done | indent 109 | 110 | # Print block devices 111 | blockDevices=$(find $devicePath -mindepth 5 -maxdepth 5 -name 'block') 112 | for blockDevice in $blockDevices; do 113 | echo 'Block device:' | color cyan 114 | echo "Model: $(cat "$blockDevice/../model")" | indent | color green 115 | lsblk -no NAME,SIZE,MOUNTPOINT "/dev/$(ls -1 $blockDevice)" | indent | color green 116 | done | indent 117 | done | indent 118 | done | pager 119 | -------------------------------------------------------------------------------- /resources/wechatpay.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ledisthebest/LEDs-single-gpu-passthrough/1b7df961107d83231b7b4cd2eb6eaf3f3521ff65/resources/wechatpay.jpg --------------------------------------------------------------------------------