During my most recent server-build, I had to tackle exposing host-managed files to VMs.

Previously, I used NFS. However, that was becoming increasingly troublesome, unfortunately due to IPv6. While I have an IPv6 connection and IPv6 network, my router will stop advertising the IPv6 network if the Internet goes down, causing connectivity issues. Additionally, the uncertainty of whether any connection will come as fe80:: vs my regular routable address was a pain (especially when my IPv6 network changed). Ideally, everything would use fe80:: internally, but it didn’t, and that was likely due to me mucking about with things.

I eventually worked around the quirky issues by forcing IPv4 on the NFS clients.

Fast forward to today, and I decided to investigate using filesystem passthrough using virtio-9p. This is the “built-in” filesystem passthrough in libvirt+qemu.

Simple?

On the surface, it is actually very simple. You select a path, you give it a name, and you mount it in the client.

In practice, a lot more thought and configuration is required.

Options

Driver

There are several options for “Driver”:

• Handle
• Path

Path is the option you want to use, here. I didn’t find a very good explanation of “Handle”.

Mode

Mode is how UIDs and permissions are handled between guest and host. To quote the Qemu 9p documentation wiki:

• Passthrough

  Files on the filesystem are directly created with client-user’s credentials.

  This would be the simplest option if the VMs were run as root. However, Fedora runs the VMs as the qemu user (additionally, you can run VMs as your user). Effectively, this means that while the guest may try to set ownership permissions on a file, the host may block it based on the qemu’s user account.

  I tried to use this option, going so far as making my shared directory permissions an inadvisably-open 777.

  Reading files works as expected, keeping their original UIDs and permissions, as long as you keep in mind that both the guest and host are applying permissions rules. Even root on the guest can’t read a file qemu on the host can’t access.

  Writing doesn’t really work, as the Qemu host process is running as a limited user account. So if you’re user chris in the VM, you can write a file to the share, but be unable to read it. The file was written to the host disk by the qemu-owned VM process, failed to set ownership to chris, leaving you with a file you created, but can not read. You’ll also get errors setting ownership/permissions.

  Every file created will be owned by qemu (which may just be a non-descript UID on your guest). This is less than desirable.

  This would be an option if you were running a VM as your user, and running apps in the VM as the same user.

• Squashed

  It is equivalent to passthrough security model; the only exception is, failure of privileged operation like chown are ignored. This makes a passthrough like security model usable for people who run KVM as non root.

  (Note, the wiki is a little unclear and lists this as “none”. Pretty sure this is correct)

  As described, this is effectively the same as Passthrough, but it pretends ownership and permissions changes worked (even though they didn’t).

  This would be a slightly better option if you were running a VM as your user, and running apps in the VM as the same user.

• Mapped

  Files are created with qemu user credentials and the client-user’s credentials are saved in extended attributes.

  This completely separates host and client permissions.

  On the host, you would chown -R qemu:qemu all the files you intend to share with the client. (the qemu-owned process still needs permissions for these files)

  On the guest, you can chown/chmod as much as you want. Any metadata changes made by the guest are stored in extended attributes. These are global (they’re not per-vm extended attributes), so if you share the same path with two guests, they’ll see the same UIDs.

  You should confirm you’re backing up extended attributes.

I used Mapped. It allows permissions on the host to be handled simply (just make qemu own everything), while permissions on the guest can also be set independently (for whatever user/service is using those files).

Source Path

This is the path on the host you want to export.

ex: /foo/bar

Target Path

This is actually a label, not a path. Use it to identify this share to the guest.

ex: foobar

Export filesystem as readonly mount

This is handy if you want to ensure the guest can not write files back to the host. Particularly handy if you share the same directory to multiple machines, but want to ensure only one can modify the contents.

Configured

Here’s a sample configuration that exports the path /foo/bar with the label foobar, using the mapped security model:

Mounting

And this is how the guest would mount the filesystem:

foobar    /mnt/foobar    9p   trans=virtio,version=9p2000.L,_netdev 0 0

Testing

Note: You may encounter problems here, and may need to skip ahead to the SELinux rules

On the guest, you can change the permissions and ownerships:

$ chown root:root /mnt/foobar
$ chmod 750 /mnt/foobar

$ ls -ld /mnt/foobar
drwxr-x---. 1 root root 96 Mar  3 23:31 /mnt/foobar

Here’s what that looks like on the host. Note the ownership and permissions haven’t changed:

$ ls -ld /foo/bar
drwxr-xr-x. 1 qemu qemu 96 Mar  3 23:31 /foo/bar

The guest values are stored as extended attributes:

$ attr -l /foo/bar
Attribute "virtfs.mode" has a 4 byte value for /foo/bar
Attribute "virtfs.uid" has a 4 byte value for /foo/bar
Attribute "selinux" has a 35 byte value for /foo/bar
Attribute "virtfs.gid" has a 4 byte value for /foo/bar

Note: I haven’t looked for an easy way to read these values. They’re binary, and dumping them doesn’t convert them to a printable character

Times are updated on the files as normal (mtime, etc).

SELinux

Guest

On the guest, these filesystems actually get tagged as nfs_t, so all the standard SELinux bools to allow httpd/samba/etc to work with NFS work as expected.

$ ls -ldZ /mnt/foobar
drwxr-x---. root root system_u:object_r:nfs_t:s0       /mnt/foobar

Host

On the host, SELinux inserts itself into the permissions as well to restrict qemu to only properly tagged files.

I used svirt_image_t (the same as the libvirt images directory). There may be a better tag, but I couldn’t immediately find one.

$ semanage fcontext -a -t svirt_image_t '/foo/bar(/.*)?'
$ restorecon -Rv /foo/bar

$ ls -ldZ /foo/bar
drwxr-xr-x. 1 qemu qemu system_u:object_r:svirt_image_t:s0 96 Mar  9 20:56 /foo/bar

RHEL and CentOS

Oh yeah, RHEL (and therefore CentOS) don’t enable the 9p filesystem driver in their kernel. You’re SOL if you want to stay on the stock kernel.

However, there are two options to add 9p:

• Kernel-plus. This is the stock kernel, but “plus” a bunch of features RHEL disabled. It is available in the centosplus repository, which is available (but disabled by default) on CentOS.

  Install CentOS Plus kernel

• kernel-ml may also include support. I haven’t actually tested 9p on this, but btrfs did work (which it doesn’t on stock).

  ElRepo kernel-ml