Fedora Cloud Image on Linode

Linode offers pre-assembled Fedora VMs, but their environment doesn’t support SELINUX. You also don’t get any notification on when you should reboot for new kernels, etc.

I decided to attempt to adapt Fedora’s stock cloud image on Linode. It was not without it’s own effort, but ultimately I think it’s a better solution than attempting to retro-fit the linode image for booting. Also, I’m using the fedora cloud image on all my other VMs, so I’m familiar with how it’s set up (and can easily spool up a local copy for testing).

This guide also covers setting up a small raid1 btrfs filesystem for my data storage, as well as setting selinux context for httpd.

Preparation

There are some quirks for booting your own kernels on Linode. It does not support your own bootloader – you have to use their supplied version of grub-legacy (called pv_grub). Unfortunately, it seems this version of grub doesn’t seem to like partitions, so we’ve got create our own /boot.

Luckily, Fedora’s Cloud image ships a grub-legacy config that will (almost) work out of the box. After a slight modification, it will happily update correctly when kernel-updates are installed, requiring no future meddling.

Linode also doesn’t have a way to upload disk images, so we need a temporary VM to bootstrap our new system.

  1. We will require a pre-existing Linode VM. Either deploy one, or use your current one.
  2. Create one 4GB raw “root” disk for your new fedora system
  3. Create one 500mb ext4 “boot” disk for /boot. This is important, you need a linode-formatted disk for /boot (possibly a raw disk without a partition table will work too)
  4. Create one 100mb raw “cloud-init” disk for bootstrapping
  5. Create two 5GB raw disks for btrfs storage, to be used later.

Attach the disks to your pre-existing (or temporary) VM. I used the following disk IDs:

  • xvdd -> root
  • xvde -> boot
  • xvdf -> cloud-init

Installation

Boot your VM.

We’re going to need some additional tools that were not available on my pre-existing Fedora 20 system. Install them:

$ yum install dosfstools cloud-utils cloud-utils-growpart genisoimage

After downloading the Fedora cloud image, use dd to put the image on our root disk. Remember, the image is compressed when downloaded.

$ xzcat Fedora-Cloud-Base-20141203-21.x86_64.raw.xz | dd of=/dev/xvdd

Expand the root partition and filesystem to fill the 4GB disk:

$ growpart /dev/xvdd 1
$ e2fsck -f /dev/xvdd1
$ resize2fs /dev/xvdd1

mount it and set up our separate /boot

## Mount new image
$ mkdir /mnt/f21
$ mount /dev/xvdd1 /mnt/f21

## Move boot directory out of the way
$ mv /mnt/f21/boot /mnt/f21/boot-dist

## Make new boot directory
$ mkdir /mnt/f21/boot
$ chmod 000 /mnt/f21/boot

## Mount new boot filesystem (remember: no partitions, and pre-formatted by linode)
$ mount /dev/xvde /mnt/f21/boot

## Copy boot contents
$ cp -a /mnt/f21/boot-dist/* /mnt/f21/boot

Another limitation of pv_grub is that it’s hard-coded to look for the grub configuration in /boot on our bare disk. Since the disk only contains /boot, it won’t find it. We can fix this using a recursive symlink.

$ ln -sfn . /mnt/f21/boot/boot

Since we’re don’t have a partition on our boot device, edit /mnt/f21/boot/grub/menu.lst to use (hd0) instead of (hd0,0)

Update /mnt/f21/etc/fstab. I prefer to use UUIDs for system disks. If I ever need to mount another system’s disk (F22 in the future?), I’d hate to have multiple filesystems labelled “boot” or “fedora”. Additionally, this way I don’t care what xvdX disk each filesystem is connected to.

# Find UUID for our /boot
$ blkid /dev/xvde
    /dev/xvde: UUID="65235d57-6e89-4347-b23e-77d0a365284c" TYPE="ext4"
# Add it to fstab. You can use vi, or echo it. Be sure to >> append, not > replace!
$ echo "UUID=65235d57-6e89-4347-b23e-77d0a365284c /boot ext4 defaults 1 1" >> /mnt/f21/etc/fstab

Make cloud-init disk. I installed dosfstools because cloud-init supposed to support simple vfat disks, and Linode can’t add a cdrom device. vfat doesn’t work, but iso9660 on a block device does. Use cloud-localds create an iso9660-formatted disk. Have your user-data and meta-data files ready.

cloud-localds /dev/xvdf ~/user-data ~/meta-data

We now have all the necessary bits to perform our first-boot on the new Fedora 21 system.

First Boot

Make new VM config for your Fedora 21 system.

  • xvda -> boot
  • xvdb -> root
  • xvdc -> cloud-init

Boot VM, let it do it’s thing. I was monitoring the console from Linode’s LISH, so I could see when cloud-init finished, then I removed cloud init and upgraded the system.

$ yum remove cloud-init
$ yum upgrade

After shutting down my VM, I removed the cloud-init disk from the VM.

Install Software

What good is a server that doesn’t do anything? Install the stuff I need.

$ yum install httpd screen vim btrfs-progs

BTRFS

While we’re editing the VM definition, add our two 5gb raw disks for btrfs. I’m using two disks so btrfs can correct checksum errors. It can detect errors with only one disk, but needs a second copy to actually fix the errornous data.

  • xvdd -> btrfs-disk-1
  • xvde -> btrfs-disk-2

Boot the VM, then make our filesystem with raid1 redundancy. Then mount it.

$ mkfs.btrfs -L linode_storage -d raid1 /dev/xvdd /dev/xvde
$ mkdir -p /mnt/btrfs/linode_storage
$ mount LABEL=linode_storage /mnt/btrfs/linode_storage

I’m going to need some subvolumes for my various storage needs:

$ btrfs subvol create /mnt/btrfs/linode_storage/home
$ btrfs subvol create /mnt/btrfs/linode_storage/sites
$ btrfs subvol create /mnt/btrfs/linode_storage/ssl

Add fstab entries. I like using LABELs for data disks since they’re not machine-specific. I’ll only have one linode_storage, so there’s no conflict issue. We need to add the degraded bit in case one of our disks goes completely south. The last thing you want is systemd sitting at the boot console waiting for you to press CTRL-D because your storage is incomplete (but recoverable).

LABEL=linode_storage                      /mnt/btrfs/linode_storage btrfs degraded                     0 0
LABEL=linode_storage                      /var/www/sites            btrfs degraded,subvol=sites        0 0
LABEL=linode_storage                      /home                     btrfs degraded,subvol=home         0 0
LABEL=linode_storage                      /var/www/ssl              btrfs degraded,subvol=ssl          0 0

Mount the subvolumes. You may need to play some musical chairs with /home, and any other directories you’re replacing.

$ mv /home /home-dist

$ mkdir      /home
$ chmod 0000 /home
$ mkdir      /var/www/ssl
$ chmod 0000 /var/www/ssl
$ mkdir      /var/www/sites
$ chmod 0000 /var/www/sites

$ mount -a
$ mv /home-dist/* /home/

Configure httpd

This is pretty similar to my article covering SELINUX and apache (httpd), so I’m not going to get into the details of configuring httpd. So ensure your ownership, permissions, and config are good at this point.

Permissions aside, we also need to set our SELINUX contexts for the new directories. I’m not using NFS on this system, so I’m actually properly (well, hopefully) configuring selinux instead of setting a blanket context on the mount point.

Use semanage to add a file context type to our new site and ssl directories. Then use restorecon to apply it. You could use chcon directly, but if a system-wide relabel happens, you’ll lose your changes.

# http sites
$ semanage fcontext -a -t httpd_sys_content_t "/var/www/sites(/.*)?"
$ restorecon -R /var/www/sites

# SSL certs
$ semanage fcontext -a -t cert_t "/var/www/ssl(/.*)?"
$ restorecon -R /var/www/ssl

I recommend using firewalld and fail2ban, but I’m not going to cover that here. That said, I will say not to forget to poke a hole for your web server:

$ firewall-cmd --permanent --add-service=http
$ firewall-cmd --permanent --add-service=https
$ firewall-cmd --reload

Start apache, and you should be good.

$ systemctl start httpd

Remember, ausearch -m avc is your best friend to diagnose access errors caused by selinux