Discard (TRIM) with KVM Virtual Machines... in 2020!
While checking out some logs and google search analytics, I found that my post about Discard (TRIM) with KVM Virtual Machines has been referenced far more than I expected it to be. I decided to take this opportitnity to fact-check and correct that article.
I migrated my server from a 10-year old Dual Xeon X5650 to a significantly more performant Ryzen 1600. I undertook the process of migrating and updating my VMs, ensuring the libvirt definitions were current, switching from NFS to 9p, etc.
virtio vs virtio-scsi
My previous post made this observation:
All of my VMs were using virtio disks. However, they don’t pass discard through. However, the virtio-scsi controller does.
It appears that is no longer entirely true. At some point between October 2015 and March 2020 (when I’m writing this), standard virtio-blk devices gained discard support. Indeed, virtio-blk devices actually support discard out of the box, with no additional configuration required:
[guest] $ sudo lsblk -o NAME,MOUNTPOINT,DISC-MAX,FSTYPE
NAME MOUNTPOINT DISC-MAX FSTYPE
vdb 2G
└─vdb1 [SWAP] 2G swap
vda 2G
└─vda1 / 2G xfs
My research shows this was qemu 4.0 that added this. Fedora 31 has qemu 4.1.
It is no longer required to convert to virtio-scsi.
FWIW, I’m using machine type ‘pc-q35-4.1’. I expect this behaviour is only exposed on machines >= 4.0.
Discard vs unmap/shrink
While the guest will support discards now, that doesn’t actually shrink the disks on the host. I haven’t performed any testing to confirm, but I suspect it is actually performing a passthrough to the underlying storage layer, allowing the SSD to do a proper Garbage Collection, without modifying the file on the host filesystem. That’s all well and good, and a perfectly sane default, but I do still want to shrink files on the host. These VMs are on a somewhat smaller drive, and I wish to reclaim space whenever possible.
To shrink the actual files on the host, the discard mode must be changed to ‘unmap’.
<driver name='qemu' type='qcow2' discard='unmap'/>
This can be done with the same sed
-magic used in my last post. Additionally, the option is actually exposed through virt-manager now:
Enable fstrim
As before, use fstrim to issue discards on-demand
$ sudo fstrim -av
Going forward, you can either add ‘discard’ to the mount options in fstab (which will probably hurt performance), or use fstrim periodically. I opted for fstrim, as it has a systemd timer unit that can be scheduled:
$ sudo systemctl enable --now fstrim.timer