├── iommu_groups_alt.bash ├── iommu_groups.bash ├── etc └── libvirt │ └── hooks │ ├── qemu.d │ └── win10 │ │ ├── release │ │ └── end │ │ │ └── stop.bash │ │ └── prepare │ │ └── begin │ │ └── start.bash │ └── qemu └── README.md /iommu_groups_alt.bash: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | shopt -s nullglob 4 | for d in /sys/kernel/iommu_groups/*/devices/*; do 5 | n=${d#*/iommu_groups/*}; n=${n%%/*} 6 | printf 'IOMMU Group %s ' "$n" 7 | lspci -nns "${d##*/}" 8 | done; 9 | -------------------------------------------------------------------------------- /iommu_groups.bash: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | while IFS= read -rd '' iommu_group; do 4 | echo "IOMMU group $(basename "$iommu_group")"; 5 | for device in "$iommu_group"/devices/*; do 6 | echo -n $'\t'; 7 | lspci -nns "$(basename "$device")"; 8 | done; 9 | done< <(sort -nzt '/' -k 5 <(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d -print0)) 10 | -------------------------------------------------------------------------------- /etc/libvirt/hooks/qemu.d/win10/release/end/stop.bash: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # stop display manager 4 | systemctl stop display-manager 5 | 6 | # unbind virtual console from intel gpu 7 | echo 0 > /sys/class/vtconsole/vtcon1/bind 8 | 9 | # disable intel Xorg config 10 | rm -f /etc/X11/xorg.conf.d/20-intel.conf 11 | 12 | # re-attach gpu 13 | virsh nodedev-reattach pci_0000_01_00_1 14 | virsh nodedev-reattach pci_0000_01_00_0 15 | 16 | # run nvidia-xconfig to fix gpu 17 | nvidia-xconfig --query-gpu-info > /dev/null 2>&1 18 | 19 | # rebind virtual console to nvidia gpu 20 | echo 1 > /sys/class/vtconsole/vtcon1/bind 21 | 22 | # rebind EFI-framebuffer 23 | echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/bind 24 | 25 | # start display manager 26 | systemctl start display-manager 27 | -------------------------------------------------------------------------------- /etc/libvirt/hooks/qemu: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Author: Sebastiaan Meijer (sebastiaan@passthroughpo.st) 4 | # 5 | # Copy this file to /etc/libvirt/hooks, make sure it's called "qemu". 6 | # After this file is installed, restart libvirt. 7 | # From now on, you can easily add per-guest qemu hooks. 8 | # Add your hooks in /etc/libvirt/hooks/qemu.d/vm_name/hook_name/state_name. 9 | # For a list of available hooks, please refer to https://www.libvirt.org/hooks.html 10 | # 11 | 12 | GUEST_NAME="$1" 13 | HOOK_NAME="$2" 14 | STATE_NAME="$3" 15 | MISC="${@:4}" 16 | 17 | BASEDIR="$(dirname $0)" 18 | 19 | HOOKPATH="$BASEDIR/qemu.d/$GUEST_NAME/$HOOK_NAME/$STATE_NAME" 20 | 21 | set -e # If a script exits with an error, we should as well. 22 | 23 | if [ -f "$HOOKPATH" ]; then 24 | eval "$HOOKPATH" "$@" 25 | elif [ -d "$HOOKPATH" ]; then 26 | find -L "$HOOKPATH" -maxdepth 1 -type f -executable -exec {} "$@" \; 27 | fi 28 | 29 | -------------------------------------------------------------------------------- /etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.bash: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | sddm_uid="995" 4 | 5 | # stop display manager 6 | systemctl stop display-manager.service 7 | 8 | # wait for all sddm processes to exit, because sddm doesn't wait for its helper process 9 | while systemctl -q is-active "user@$sddm_uid.service"; do 10 | sleep 1 11 | done 12 | 13 | # unbind virtual console 14 | echo 0 > /sys/class/vtconsole/vtcon1/bind 15 | 16 | # unbind EFI-framebuffer 17 | echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/unbind 18 | 19 | # detach GPU 20 | virsh nodedev-detach pci_0000_01_00_0 21 | virsh nodedev-detach pci_0000_01_00_1 22 | 23 | # bind virtual console to available gpu (only intel available) 24 | echo 1 > /sys/class/vtconsole/vtcon1/bind 25 | 26 | # enable Xorg intel config (so Xorg only detects the intel gpu) 27 | cat > /etc/X11/xorg.conf.d/20-intel.conf <<- EOM 28 | Section "Device" 29 | Identifier "Intel Graphics" 30 | Driver "intel" 31 | BusID "PCI:0:2:0" 32 | EndSection 33 | EOM 34 | 35 | # start display manager 36 | systemctl start display-manager.service 37 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # gpu-passthrough 2 | A collection of links regarding gpu passthrough 3 | 4 | PCIe device power cycle: https://unix.stackexchange.com/questions/73908/how-to-reset-cycle-power-to-a-pcie-device 5 | Pass primary GPU to VM + patch vBIOS: https://github.com/joeknock90/Single-GPU-Passthrough 6 | Pass GPU to VM and back without X restart using DRI3 drivers: https://arseniyshestakov.com/2016/03/31/how-to-pass-gpu-to-vm-and-back-without-x-restart/ 7 | GPU passthrough without restart: https://www.reddit.com/r/VFIO/comments/93c4s7/passing_nvidia_card_between_host_and_guest_and_to/ 8 | Passing primary nvidia gpu to vm (proprietary drivers) with X restart: https://bbs.archlinux.org/viewtopic.php?id=233959 9 | Collection of information and scripts regarding gpu passthrough: https://gitlab.com/YuriAlek/vfio 10 | Arch Wiki article: https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF 11 | GPU hotplug: https://gist.github.com/ArseniyShestakov/dc152d080c65ebaa6781 12 | 13 | ### amd reset bug 14 | The AMD reset bug is an issue in qemu with many AMD cards. 15 | Apparently qemu fails to correctly reset the card on guest shutdown/reboot which causes the card to not be usable anymore until host reboot. 16 | https://vfio.blogspot.com/2015/04/progress-on-amd-front.html 17 | https://www.reddit.com/r/VFIO/comments/5tf62v/who_still_experiences_the_amd_reset_bug/ 18 | https://www.reddit.com/r/VFIO/comments/52ij8j/the_amdresetbug_will_it_ever_be_fixedwhere_to/ 19 | https://www.reddit.com/r/VFIO/comments/angluw/restart_vs_shutdown_of_vm/ 20 | 21 | Possible solution: 22 | https://forum.level1techs.com/t/linux-host-windows-guest-gpu-passthrough-reinitialization-fix/121097 23 | https://forums.unraid.net/topic/69373-graphics-card-reset-issue-amd/?tab=comments#comment-635252 24 | https://github.com/inga-lovinde/RadeonResetBugFix 25 | 26 | ### could be interesting 27 | | command | comment | source | 28 | | --- | --- | --- | 29 | | `cat /sys/kernel/debug/dri/1/amdgpu_gpu_recover` | gpu recovery | [1](https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver/issues/11#issuecomment-450696825), [2](https://dri.freedesktop.org/docs/drm/gpu/amdgpu.html) | 30 | 31 | ### kvm_set_msr_common 32 | When launching some games (e.g. L.A. Noire) in a VM with CPU model set to 'host-model' the game may crash and the following errors get logged by the kernel: https://gist.github.com/jkhsjdhjs/f45046d7dbec02d5a0dc9136319f222a 33 | This is caused by the NVIDIA driver (apparently only on Windows 10) trying to use some MSRs (Model Specific Registers) of the host CPU the VCPU doesn't support. 34 | `ignore_msrs` is already set to 1 here (check with `cat /sys/module/kvm/parameters/ignore_msrs`). 35 | It could be fixed by changing the CPU model. However I haven't had the time to test which CPU model works yet. 36 | https://www.reddit.com/r/VFIO/comments/4kagkv/nvidia_system_unstable_windows_10_qemukvmvgapass/d3dldyj/ 37 | EDIT: Changing the CPU model results either in the VM not booting or the errors still occuring. 38 | --------------------------------------------------------------------------------