Proxmox GPU Passthrough
This configuration worked for me, you might need to change things around
Keep in mind I have an AMD CPU and Nvidia GPU, if you have other config, you might have to use different commands
After upgrading Proxmox to 7.2, passthrough wasn't working. To make it work again try resetting your graphics card: Resetting GPU
OR keep reading, the GRUB parameters have to be changed to make it work again with the latest kernel!
Before doing anything make sure virtualization and IOMMU is enabled in your BIOS, you can't do anything bofore that.
If your motherboard doesn't support IOMMU, then you can't pass through PCI(e) devices to your VMs.
Update the Host configuration
Login to the host and open
/etc/default/grub. Find the line
GRUB_CMDLINE_LINUX_DEFAULT and change it from:
GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt nofb nomodeset video=vesafb:off video=efifb:off"
Please note that
efifb:off are assigned to
video in two different statements. Many tutorials mention a shorted version, like
video=vesafb:off,efifb:off but that didn't work for me.
Rather than using "video=vesafb:off video=efifb:off" parameters, try replacing them with "initcall_blacklist=sysfb_init" when using the latest kernel (5.15.64-1-pve), so the GPU passthrough will work again
update-grub to append the grub's content to all linux entries in
/etc/modules and add the followings:
vfio vfio_iommu_type1 vfio_pci vfio_virqfd
After these changes run the below to refresh the
initramfs, then restart your server:
update-initramfs -u -k all
Once it's restarted, run the below commands to check if IOMMU was successfully enabled:
dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
It should display that
IOMMU, Directed I/O or Interrupt Remapping is enabled or something similar, it could be different on your hardware.
Also check that the devices are in different IOMMU groups:
find /sys/kernel/iommu_groups/ -type l
Device passthrough setup
First find the device Ids that you want to passthrough.
which will display all the devices and their Ids in the host. Find yours and write it down.
It looks something like
[1245:4f5a], don't forget the copy the audio device's Id as well
Since we want to use a GPU in our VM, we have to passthrough both the video and sound devices, you can't pass only one of them.
You also have to blacklist your GPU so the host won't utilize it. This is how my
/etc/modprobe.d/pve-blacklist.conf looks like:
# This file contains a list of modules which are not supported by Proxmox VE # nidiafb see bugreport https://bugzilla.proxmox.com/show_bug.cgi?id=701 blacklist nvidiafb #blacklist radeon # Use this line if you have an AMD card #blacklist amdgpu # Use this line if you have an AMD card blacklist nvidia blacklist nouveau
Then in your
options vfio-pci ids=1245:4f5a,1002:aad8 disable_vga=1
Here the two Ids are the ones which you copied previously.
/etc/modprobe.d/kvm.conf with the below content:
options kvm ignore_msrs=1
This will allow to use Nvidia cards on Windows when you set the CPU to host.
Apply these changes:
update-initramfs -u -k all then restart the host.
At this point your host should be ready.
Configuration in a text format:
bios: ovmf bootdisk: scsi0 cores: 4 cpu: kvm64,flags=+aes cpuunits: 3096 efidisk0: container:136/vm-136-disk-1.qcow2,size=128K hostpci0: 07:00,pcie=1,x-vga=1 ide2: none,media=cdrom machine: q35 memory: 12288 name: ubuntu8 net0: virtio=A2:0B:1E:90:B5:B8,bridge=vmbr0,firewall=1 numa: 0 ostype: l26 scsi0: container:136/vm-136-disk-0.qcow2,size=52G scsihw: virtio-scsi-pci smbios1: uuid=b925668d-9785-4941-ab36-4151164248c7 sockets: 1 usb0: host=046d:c05b,usb3=1 usb1: host=413c:2113,usb3=1 vga: none vmgenid: 9aae2c4f-30ff-4a2b-ac56-805e49c670d5
07:00is my GPU set to
- vga has to be set to null
- cpu can be kvm64, it doesn't have to be host
- I set cpuunits to higher than default so proxmox will prioritize this VM