Virtualization

QEMU disk mounter

I've been recently upgrading my home server farm to new Intel VT and AMD SVM capable chipsets. The principal benefit is to consolidate numerous random machines onto virtual machines on a few physical servers. The difference in my monthly electricity bill has been marked!

I've been using KVM on Fedora. Since RedHat recently bought KVM I figured Fedora was most likely to have the best integration of the two - though I'm not convinced this is really the case.

Regardless, the one thing I keep needing to do is be able to modify the contents of a VM's virtual disk while the VM is offline - this helps with all sorts of things like backup/restore, to simple identity provisioning based off a golden OS image.

The problem with a raw disk image is that it is not a single filesystem partition, but an entire disk with partition table and multiple partitions. So how to access each partition? (e.g. run fsck offline, copy files on or off).

Initially I kludged something with scripts based on sfdisk to dump the partitions and dmsetup to create logical devices, but it was hard to unwind when things went wrong, and there were/are a limited number of DM devices.

Worse, DM doesn't understand any of the non-raw file formats like VMware's VMDK format or like qcow2 etc. that qemu can use to reduce disk space usage etc. and speed cloning.

So instead I wrote my own utility based using FUSE that will take a disk image, and expose each slice/partition on that image as a file. Those files can then be modified with user level applications (e.g. mke2fs) or loop-back mounted so that data can be copied on and off using the regular filesystem.

Code releases

The code is available (warts 'an all) in tar-ball form.

Current release: qemu-diskp-1.0c.tgz

Quick start reference

This quick start is also available in the README file of each release, and a screenshot of the commands running for real is here

For this I'll assume you're using KVM on Linux, and have created your golden reference disk image - probably the old fashined way of booting and installing from a cdrom ISO image.

Let's assume this disk is called: base-linux.img and is in a raw disk format.

Now we want to create a second VM using the same image, but also copy in a new message of the day file (/etc/motd) for the new machine :-)

First we do a quick clone (and save disk space) by using a qcow2 clone image:


# qemu-img create -b base-linux.img -f qcow2 clone.qcow2

Next we simply mount the clone image as a file system:


# qemu-diskp clone.qcow2 /mnt/fuse -o allow_other

Notes:

  1. If FUSE barfs on the allow_other option, either run this as root, or create a /etc/fuse.conf file with a single line: user_allow_other
  2. The /mnt/fuse mount point is one of your choice... if you have multiple users doing this you might want to script this with a per-use mount point e.g. /mnt/fuse/username.

We can see each of the disk partition images as files in at the mount point:


# ls -l /mnt/fuse/
total 0
-rw-rw-rw- 1 root root 1052803584 2009-03-29 15:35 clone.qcow2.0
-rw-rw-rw- 1 root root 1052835840 2009-03-29 15:35 clone.qcow2.1

Note: each file name is based on the base file name of the image file to avoid confusion. The extra .0, .1 etc. on the filename is the partition number being referred to.

We can then loopback mount one of the partitions as a filesystem locally:


# mount -o loop /mnt/fuse/clone.qcow2.0 /mnt/share

At this point you can treat /mnt/share as a regular file system, and copy files on and off at will ...


# cp /etc/motd /mnt/share/etc

When you're finished working on your disk image, unmount any loop-back mounts. Then, simply unmount the qemu-diskp file system, and the program will exit.


# umount /mnt/fuse

errr....
... that's it.

Warning

It's been implicit in the above description that you're not trying to do anything else with the disk image file while you're using it with qemu-diskp.

Or more explicitly do not try to do anything else with the disk image file while you're using it with qemu-diskp.

Image files are opened using the block driver library from qemu, which doesn't try to open files exclusively, so there's no enforcement to prevent you from opening your disk image file with other programs while it is in use with qemu-diskp.

Modifying the contents of your image file out from under qemu-diskp instead of via qemu-diskp will likely confuse qemu-diskp, and more importantly any OS filesystem if you've loop-back mounted any of the qemu-diskp exported partition files. All sorts of unpredictable behavior will likely ensue, if you're lucky you'll only corrupt the contents of your image file, but quite possibly much worse. So don't do this.


Copyright © A.Saulsbury 2009. All rights reserved.