GPU Passthrough

Recording a log of my efforts to have proper GPU performance in a windows VM.

Fist, enable some iommu on kernel. Given I use syslinux, I added intel_iommu=on. Also we'll use a new initramfs, but more on that later.

LABEL arch-lts-hypervisor
    MENU LABEL Arch Linux Hypervisor
    LINUX ../vmlinuz-linux-lts
    APPEND root=UUID=1c297142-49a8-421d-a269-b14099ce2f81 rw intel_iommu=on iommu=pt
    INITRD ../intel-ucode.img,../initramfs-linux-lts-hypervisor.img

Also I plugged in my radeon card and started the computer:

Then I listed the IOMMU groups with the script from arch linux manual:

IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Oland XT [Radeon HD 8670 / R7 250/350] [1002:6610] (rev 81)
IOMMU Group 1 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0]
IOMMU Group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation ...

We need to configure vfio module by setting these parameters:

# cat /etc/modprobe.d/vfio.conf

options vfio-pci ids=1002:6610,1002:aab0

Create a new initial ramfs configuration where make sure you load modules and use modconf hook, so you want to match MODULES and also make sure modconf is included in HOOKS.

cat /etc/mkinitcpio-hypervisor.conf

Results:

MODULES=(vfio_pci vfio vfio_iommu_type1 vfio_virqfd xhci-hcd i915)
BINARIES=()
FILES=()
HOOKS=(base udev autodetect modconf block filesystems keyboard fsck)

And re-generated the initramfs:

mkinitcpio -c /etc/mkinitcpio-hypervisor.conf -g /boot/initramfs-linux-lts-hypervisor.img

and rebooted...

Verifying that the configuration worked

# dmesg | grep -i vfio

[    0.681607] VFIO - User Level meta-driver version: 0.3
[    0.683310] vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[    0.710059] vfio_pci: add [1002:6610[ffff:ffff]] class 0x000000/00000000
[    0.740110] vfio_pci: add [1002:aab0[ffff:ffff]] class 0x000000/00000000
[    0.797346] vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none

Or another check (if the above does not put out any device ids):

lspci -nnk -d 1002:6610

Then gave it to my virtual machine, and everything worked as expected.

Links

<qemu:arg value='-cpu'/>
<qemu:arg value='host,hv_time,kvm=off'/>