├── iommu.sh ├── start.sh ├── LICENSE └── README.md /iommu.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # get a list of devices and respective IOMMU group numbers 4 | # (forgot where this script came from) 5 | 6 | shopt -s nullglob 7 | for d in /sys/kernel/iommu_groups/*/devices/*; do 8 | n=${d#*/iommu_groups/*}; n=${n%%/*} 9 | printf 'IOMMU Group %s ' "$n" 10 | lspci -nns "${d##*/}" 11 | done; 12 | -------------------------------------------------------------------------------- /start.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # start.sh 4 | # enable or disable OPTS as needed 5 | 6 | OPTS="" 7 | OPTS="$OPTS -serial none -parallel none" 8 | OPTS="$OPTS -nodefaults" 9 | OPTS="$OPTS -name windows" 10 | OPTS="$OPTS -rtc clock=host,base=localtime" 11 | # Basic CPU settings. 12 | #OPTS="$OPTS -cpu host,kvm=off,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_vendor_id=Nvidia43FIX" 13 | # remove hv_time to see if anything happens 14 | #OPTS="$OPTS -cpu host,kvm=off,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_vendor_id=Nvidia43FIX" 15 | OPTS="$OPTS -cpu host,kvm=off" 16 | OPTS="$OPTS -smp 8,sockets=1,cores=4,threads=2" 17 | # Enable KVM full virtualization support. 18 | OPTS="$OPTS -enable-kvm" 19 | # Assign memory to the VM. Hugepages requires additional configuration. 20 | OPTS="$OPTS -m 16G" 21 | OPTS="$OPTS -mem-path /dev/hugepages" 22 | OPTS="$OPTS -mem-prealloc" 23 | # VFIO GPU and GPU sound passthrough. 24 | OPTS="$OPTS -device vfio-pci,host=02:00.0,multifunction=on" 25 | OPTS="$OPTS -device vfio-pci,host=02:00.1" 26 | # Supply OVMF (general UEFI bios, needed for EFI boot support with GPT disks). 27 | OPTS="$OPTS -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd" 28 | OPTS="$OPTS -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_VARS.fd" 29 | 30 | # Load our created VM image as a harddrive. 31 | # 32 | # NOTE: Giving the VM *raw* disk access can lead to unintended things 33 | # happening. Do this only if you're giving the VM dedicated and 34 | # exclusive access to the HDD. 35 | # 36 | OPTS="$OPTS -device virtio-scsi-pci,id=scsi" 37 | OPTS="$OPTS -drive file=/dev/disk/by-id/ata-YOUR_DRIVE_HERE,cache=none,if=virtio,format=raw" 38 | 39 | # Load our OS setup image e.g. ISO file. 40 | #OPTS="$OPTS -cdrom $(pwd)/en_windows_10_education_version_1607_updated_jul_2016_x64_dvd_9055880.iso" 41 | # load virtio drivers 42 | #OPTS="$OPTS -cdrom $(pwd)/virtio-win.iso" 43 | # Use the following emulated video device (use none for disabled). 44 | OPTS="$OPTS -vga none" 45 | # running from the shell 46 | OPTS="$OPTS -nographic" 47 | # Redirect QEMU's console input and output. 48 | OPTS="$OPTS -monitor stdio" 49 | 50 | # (qemu-2.10+) Use QEMU's XHCI host adapter support for USB 1.1, 2, 3 51 | # This has the added benefit of not requiring the user to specify the 52 | # bus. 53 | # See https://git.qemu.org/?p=qemu.git;a=blob;f=docs/usb2.txt;h=172614d3a7e0566c2cdd988d72a1674b73f879fe;hb=HEAD 54 | #OPTS="$OPTS -device qemu-xhci" 55 | 56 | # Otherwise, use the other XHCI controller (USB 1.1, 2, 3) if you're 57 | # running qemu < 2.10: 58 | # https://en.wikibooks.org/wiki/QEMU/Devices/USB/Root 59 | OPTS="$OPTS -device nec-usb-xhci,id=xhci" 60 | 61 | # Or if you need USB 2.0 support only 62 | #OPTS="$OPTS -device usb-ehci,id=ehci" 63 | 64 | # Passthrough USB devices. 65 | OPTS="$OPTS -usb" 66 | # USB mouse 67 | OPTS="$OPTS -device usb-host,bus=xhci.0,vendorid=0xdead,productid=0xbeef" 68 | # USB keyboard 69 | OPTS="$OPTS -device usb-host,bus=xhci.0,vendorid=0xdead,productid=0xbeef" 70 | 71 | # Network configuration 72 | # 73 | # See qemu documentation for more details, but '-net user' will put the 74 | # VM in its own subnet that only the host can access. The Windows VM 75 | # will be able to initiate communications on the LAN and WAN. 76 | # 77 | # Passing the 'smb=/path/' option will start QEMU's Samba server on the 78 | # host that the guest can access. It does have read/write access to the 79 | # shared folder (depending on the permissions or mount options you've 80 | # set). If you need a more complex configuration, then you'll probably 81 | # want to set up your own Samba server. 82 | # 83 | #OPTS="$OPTS -net nic -net user" 84 | OPTS="$OPTS -net nic -net user,smb=/shared/" 85 | 86 | # first two options are related to audio configuration 87 | # 'taskset' pins qemu to certain CPUs for more consistent performance 88 | # (otherwise qemu seems switch to whichever cores it feels like). 89 | QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128 taskset -c 21-31 qemu-system-x86_64 $OPTS 90 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | CC0 1.0 Universal 2 | 3 | Statement of Purpose 4 | 5 | The laws of most jurisdictions throughout the world automatically confer 6 | exclusive Copyright and Related Rights (defined below) upon the creator and 7 | subsequent owner(s) (each and all, an "owner") of an original work of 8 | authorship and/or a database (each, a "Work"). 9 | 10 | Certain owners wish to permanently relinquish those rights to a Work for the 11 | purpose of contributing to a commons of creative, cultural and scientific 12 | works ("Commons") that the public can reliably and without fear of later 13 | claims of infringement build upon, modify, incorporate in other works, reuse 14 | and redistribute as freely as possible in any form whatsoever and for any 15 | purposes, including without limitation commercial purposes. These owners may 16 | contribute to the Commons to promote the ideal of a free culture and the 17 | further production of creative, cultural and scientific works, or to gain 18 | reputation or greater distribution for their Work in part through the use and 19 | efforts of others. 20 | 21 | For these and/or other purposes and motivations, and without any expectation 22 | of additional consideration or compensation, the person associating CC0 with a 23 | Work (the "Affirmer"), to the extent that he or she is an owner of Copyright 24 | and Related Rights in the Work, voluntarily elects to apply CC0 to the Work 25 | and publicly distribute the Work under its terms, with knowledge of his or her 26 | Copyright and Related Rights in the Work and the meaning and intended legal 27 | effect of CC0 on those rights. 28 | 29 | 1. Copyright and Related Rights. A Work made available under CC0 may be 30 | protected by copyright and related or neighboring rights ("Copyright and 31 | Related Rights"). Copyright and Related Rights include, but are not limited 32 | to, the following: 33 | 34 | i. the right to reproduce, adapt, distribute, perform, display, communicate, 35 | and translate a Work; 36 | 37 | ii. moral rights retained by the original author(s) and/or performer(s); 38 | 39 | iii. publicity and privacy rights pertaining to a person's image or likeness 40 | depicted in a Work; 41 | 42 | iv. rights protecting against unfair competition in regards to a Work, 43 | subject to the limitations in paragraph 4(a), below; 44 | 45 | v. rights protecting the extraction, dissemination, use and reuse of data in 46 | a Work; 47 | 48 | vi. database rights (such as those arising under Directive 96/9/EC of the 49 | European Parliament and of the Council of 11 March 1996 on the legal 50 | protection of databases, and under any national implementation thereof, 51 | including any amended or successor version of such directive); and 52 | 53 | vii. other similar, equivalent or corresponding rights throughout the world 54 | based on applicable law or treaty, and any national implementations thereof. 55 | 56 | 2. Waiver. To the greatest extent permitted by, but not in contravention of, 57 | applicable law, Affirmer hereby overtly, fully, permanently, irrevocably and 58 | unconditionally waives, abandons, and surrenders all of Affirmer's Copyright 59 | and Related Rights and associated claims and causes of action, whether now 60 | known or unknown (including existing as well as future claims and causes of 61 | action), in the Work (i) in all territories worldwide, (ii) for the maximum 62 | duration provided by applicable law or treaty (including future time 63 | extensions), (iii) in any current or future medium and for any number of 64 | copies, and (iv) for any purpose whatsoever, including without limitation 65 | commercial, advertising or promotional purposes (the "Waiver"). Affirmer makes 66 | the Waiver for the benefit of each member of the public at large and to the 67 | detriment of Affirmer's heirs and successors, fully intending that such Waiver 68 | shall not be subject to revocation, rescission, cancellation, termination, or 69 | any other legal or equitable action to disrupt the quiet enjoyment of the Work 70 | by the public as contemplated by Affirmer's express Statement of Purpose. 71 | 72 | 3. Public License Fallback. Should any part of the Waiver for any reason be 73 | judged legally invalid or ineffective under applicable law, then the Waiver 74 | shall be preserved to the maximum extent permitted taking into account 75 | Affirmer's express Statement of Purpose. In addition, to the extent the Waiver 76 | is so judged Affirmer hereby grants to each affected person a royalty-free, 77 | non transferable, non sublicensable, non exclusive, irrevocable and 78 | unconditional license to exercise Affirmer's Copyright and Related Rights in 79 | the Work (i) in all territories worldwide, (ii) for the maximum duration 80 | provided by applicable law or treaty (including future time extensions), (iii) 81 | in any current or future medium and for any number of copies, and (iv) for any 82 | purpose whatsoever, including without limitation commercial, advertising or 83 | promotional purposes (the "License"). The License shall be deemed effective as 84 | of the date CC0 was applied by Affirmer to the Work. Should any part of the 85 | License for any reason be judged legally invalid or ineffective under 86 | applicable law, such partial invalidity or ineffectiveness shall not 87 | invalidate the remainder of the License, and in such case Affirmer hereby 88 | affirms that he or she will not (i) exercise any of his or her remaining 89 | Copyright and Related Rights in the Work or (ii) assert any associated claims 90 | and causes of action with respect to the Work, in either case contrary to 91 | Affirmer's express Statement of Purpose. 92 | 93 | 4. Limitations and Disclaimers. 94 | 95 | a. No trademark or patent rights held by Affirmer are waived, abandoned, 96 | surrendered, licensed or otherwise affected by this document. 97 | 98 | b. Affirmer offers the Work as-is and makes no representations or warranties 99 | of any kind concerning the Work, express, implied, statutory or otherwise, 100 | including without limitation warranties of title, merchantability, fitness 101 | for a particular purpose, non infringement, or the absence of latent or 102 | other defects, accuracy, or the present or absence of errors, whether or not 103 | discoverable, all to the greatest extent permissible under applicable law. 104 | 105 | c. Affirmer disclaims responsibility for clearing rights of other persons 106 | that may apply to the Work or any use thereof, including without limitation 107 | any person's Copyright and Related Rights in the Work. Further, Affirmer 108 | disclaims responsibility for obtaining any necessary consents, permissions 109 | or other rights required for any use of the Work. 110 | 111 | d. Affirmer understands and acknowledges that Creative Commons is not a 112 | party to this document and has no duty or obligation with respect to this 113 | CC0 or use of the Work. 114 | 115 | For more information, please see 116 | 117 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # qemu-windows-10 2 | 3 | Some notes and a starter script on getting a Windows 10 guest running on 4 | a Debian 10 host using QEMU 3.1.0. 5 | 6 | *Note*: Still a work in progress. See [start.sh](./start.sh) for the 7 | script. 8 | 9 | ## Table of Contents 10 | 11 | 12 | 1. [Hardware Requirements](#hardware-requirements) 13 | 1. [Software Requirements](#software-requirements) 14 | 1. [Software Configuration](#software-configuration) 15 | 1. [Configuring the VM](#configuring-the-vm) 16 | 1. [USB Devices](#usb-devices) 17 | 1. [Audio](#audio) 18 | 1. [CPU-pinning with `taskset`](#cpu-pinning-with-taskset) 19 | 1. [Some Possible Performance Improvements](#some-possible-performance-improvements) 20 | 1. [hugepages](#hugepages) 21 | 1. [Force Windows to use MSI on GPU](#force-windows-to-use-msi-on-gpu) 22 | 1. [Running the VM](#running-the-vm) 23 | 1. [TODO](#todo) 24 | 1. [File Sharing with Samba](#file-sharing-with-samba) 25 | 1. [BIOS/UEFI boot priority](#bios/uefi-boot-priority) 26 | 1. [Resources](#resources) 27 | 1. [LICENSE](#license) 28 | 29 | 30 | ## Hardware Requirements 31 | 32 | The main thing you need is VT-d support on your processor. 33 | 34 | I have a dedicated GPU and HDD for Windows. 35 | If you plan on sharing the GPU or using a filesystem container, tweak 36 | the configuration as needed. 37 | 38 | It's useful to have a separate keyboard/mouse for Windows' exclusive 39 | use, but not necessary. 40 | If you only have one kb/m, the Windows guest will "take" it while 41 | it's running, and then "release" it once it's shutdown. 42 | This can be inconvenient if you're still tweaking the config. 43 | 44 | ## Software Requirements 45 | 46 | * Linux kernel 4.1+ (vfio, iommu support) 47 | * `qemu-kvm` 48 | * `ovmf` (latest tarball from Fedora's website?) 49 | * `hugepages` 50 | * [virtio-win.iso][virtio-win.iso] 51 | 52 | [virtio-win.iso]: https://fedoraproject.org/wiki/Windows_Virtio_Drivers 53 | 54 | Some guides and tutorials will use `virt-manager`. 55 | Since my Linux install is rather lean, I've done all of the 56 | configuration by hand (and won't cover virt-manager here). 57 | 58 | ## Software Configuration 59 | 60 | *Note*: section is incomplete. 61 | 62 | `/etc/default/grub` (then run `update-grub`): 63 | ``` 64 | GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on" 65 | ``` 66 | 67 | `/etc/modules`: 68 | ``` 69 | vfio 70 | vfio_iommu_type1 71 | vfio_pci 72 | kvm 73 | kvm_intel 74 | ``` 75 | 76 | Generally your GPU will have two device IDs under `lspci`, the "VGA 77 | Compatible Controller" and an associated "Audio device". Passing both is 78 | useful, especially if you want to use audio passthrough with HDMI or 79 | DisplayPort. 80 | 81 | `/etc/modprobe.d/vfio-pci.conf`: 82 | ``` 83 | options vfio-pci ids=YOUR_IDS_HERE 84 | ``` 85 | 86 | ## Configuring the VM 87 | 88 | **Configure the options for your VM in [start.sh](./start.sh) before 89 | attempting to run it.** 90 | 91 | This script is highly dependent on your hardware and host OS, and it's 92 | advisable to go through it line-by-line to understand what options 93 | you're passing to QEMU. 94 | 95 | ### USB Devices 96 | 97 | The old way of using `-usbdevice` has been deprecated. See the script or 98 | [QEMU/USB Quick Start][qemu-usb-qs] for more details on specifying USB 99 | devices. 100 | 101 | Ideally, use the `qemu-xhci` controller device (qemu-2.10+) to minimize 102 | CPU overhead, but the current version on Debian Stretch is qemu-2.8, 103 | requiring the use of `nec-usb-xhci`--not sure of what the performance 104 | impact is. 105 | 106 | If you don't specify a USB host controller, QEMU defaults to a slower (I 107 | think) one. 108 | 109 | This can be problematic in non-obvious ways: for example, if you wanted 110 | to connect and use an Xbox Controller for Windows, Windows 10 might 111 | recognize and install drivers for the Xbox Controller but won't 112 | initialize it properly (Code 10). This can be resolved by specifying an 113 | XHCI or EHCI controller and attaching it to that bus (on this particular 114 | [Reddit thread](https://old.reddit.com/r/Windows10/comments/7v4jc2/xbox_one_wireless_adapter_for_windows_10_code_10/), 115 | an EHCI/USB-2.0 only controller needed to be specified). 116 | 117 | **References**: 118 | * [QEMU/USB Quick Start][qemu-usb-qs] 119 | * [QEMU USB Controllers](https://en.wikibooks.org/wiki/QEMU/Devices/USB/Root) 120 | 121 | [qemu-usb-qs]: https://git.qemu.org/?p=qemu.git;a=blob;f=docs/usb2.txt;h=172614d3a7e0566c2cdd988d72a1674b73f879fe;hb=HEAD 122 | 123 | ### Audio 124 | 125 | Getting audio working on the guest was a bit tricky. You might face some 126 | issues with scratchy, delayed, or even loss of sound, most of which I 127 | won't cover here since I did not experience it. 128 | 129 | There are different ways to do audio: 130 | 131 | 1. guest passes through audio to host setup: I attempted to set this up, 132 | but this solution seemed to introduce additional problems. It might 133 | work well if you have audio working properly on the host OS. 134 | 2. PCIe soundcard passthrough: untested. 135 | 3. HDMI/DP-passthrough (via graphics card): seems to work well, but I am 136 | using the DVI output of my graphics card. 137 | 4. USB headset or USB sound card passthrough: this worked the best and 138 | what I currently use. 139 | 140 | **Note**: Having multiple sound devices (#3 and #4) seemed to result in 141 | audio intermittently working, with either lots of lag or no sound at 142 | all. 143 | 144 | There is some issue with drivers, latency, audio buffers, and timing, 145 | but what I found solved this issue was to have only *one* device enabled 146 | in the start script at one time. In my case, I disabled passing through 147 | the audio device on the graphics card and only passed through the USB 148 | headset: all works well now. 149 | 150 | ### CPU-pinning with `taskset` 151 | 152 | Set aside various cores for the guest OS. 153 | 154 | You'll want to be mindful of any additional computations on the 155 | host or other VMs that aren't pinned. 156 | For example, if you run a heavy job on the host without specifying 157 | which CPUs it should or shouldn't use, you'll start sharing with the 158 | Windows 10 guest, leading to terrible performance on the guest. 159 | 160 | See the script for some details. 161 | 162 | ### Some Possible Performance Improvements 163 | 164 | The following sections are some configuration settings that I use. 165 | These are likely optional, with perhaps some minor increases in 166 | performance (less than 10%?). 167 | 168 | Documentation here is a bit incomplete. 169 | 170 | #### hugepages 171 | 172 | We can use `hugepages` to reserve a certain amount of RAM for the 173 | guest's exclusive use. 174 | 175 | ``` 176 | apt install libhugetlbfs-bin 177 | hugeadm --explain 178 | ``` 179 | 180 | Configure number of pages to reserve in `/etc/sysctl.conf`: 181 | 182 | ``` 183 | # If guest has 16GiB of memory, 16GiB / 2MiB = 8192 pages. 184 | # Add ~5% overhead (YMMV) => 8600 pages. 185 | # 186 | # If you don't add any overhead, Windows will sometimes blue screen if 187 | # it uses up all of its available memory. 188 | vm.nr_hugepages = 8600 189 | vm.hugetlb_shm_group = 1001 190 | ``` 191 | 192 | Display hugepages configuration, and set recommended settings: 193 | ``` 194 | hugeadm --explain 195 | hugeadm --set-recommended-min_free_kbytes --set-recommended-shmmax 196 | ``` 197 | 198 | #### Force Windows to use MSI on GPU 199 | 200 | Increased performance. Some [documentation][vfio-msi]. 201 | 202 | [vfio-msi]: https://vfio.blogspot.com/2014/09/vfio-interrupts-and-how-to-coax-windows.html 203 | 204 | MSI supported, but not enabled: 205 | 206 | ```console 207 | # lspci -v -s 2:00.0 208 | 02:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] (rev a1) (prog-if 00 [VGA controller]) 209 | ... 210 | Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+ 211 | ... 212 | Kernel driver in use: vfio-pci 213 | Kernel modules: nouveau 214 | ``` 215 | 216 | MSI supported, and enabled in Windows 10 guest: 217 | ``` 218 | # lspci -v -s 2:00.0 219 | 02:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] (rev a1) (prog-if 00 [VGA controller]) 220 | ... 221 | Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+ 222 | ... 223 | Kernel driver in use: vfio-pci 224 | Kernel modules: nouveau 225 | ``` 226 | 227 | ## Running the VM 228 | 229 | [start.sh](./start.sh) 230 | 231 | Run as `root`. There is a way to run it as a less-privileged user, but I 232 | haven't gotten around to configuring that yet. 233 | 234 | Load the Windows 10 installation ISO and virtio-win.iso as options 235 | passed to `-cdrom` for the initial install, then comment those lines out 236 | on subsequent reboots (after the Windows installation is complete). 237 | 238 | In the Windows 10 install process, you might need to load device drivers 239 | before it recognizes your disks. 240 | 241 | ## TODO 242 | 243 | Sections I haven't quite fully documented. 244 | 245 | * file sharing 246 | * BIOS/UEFI boot priority 247 | 248 | ### File Sharing with Samba 249 | 250 | QEMU spins up a temporary Samba allowing access to only the guest using 251 | a configuration located at `/tmp/qemu-smb.${RANDOM}/smb.conf`. 252 | 253 | There doesn't seem to be a good way to configure the Samba share with 254 | anything more complex than one folder. 255 | 256 | Although not recommended, you could modify the ephemeral config after 257 | the guest has started and reload `smbd`. 258 | 259 | If you need finer controls in file sharing, it seems best to run a 260 | proper Samba config and restrict access to the loopback interface for 261 | guest-OS-only access. 262 | 263 | ### BIOS/UEFI boot priority 264 | 265 | If you install Windows directly to another disk (without using a 266 | container), the BIOS/UEFI will detect it as a possible boot device and 267 | attempt to load it. 268 | 269 | My workaround is to ensure that the BIOS setting ignores the hard drive 270 | that Windows is on. On occasion after some hardware changes, Windows 271 | will be loaded directly. 272 | 273 | ## Resources 274 | 275 | * 276 | * 277 | 278 | ## LICENSE 279 | 280 | [`CC0-1.0` / CC0-1.0 Universal](./LICENSE) 281 | --------------------------------------------------------------------------------