Alex Bligh's blog

I wrote here about how to build PV drivers for Linux using the unmodified_drivers directory from Xen. The problem with this method is that you need to Xenify an existing kernel. And although the process can be automated a little, it still requires someone to keep Xen patches in sync with the kernel. That’s hard work for someone, and certainly too hard work for me.

So I simplified things. By stripping out only the files that are actually used when building the PV drivers, and putting them into a normal kernel build configuration, I can now build PV drivers for a fully HVM kernel (i.e. a normal kernel), just by adding a patch. This has the great merit  you can use it with a kernel package from an arbitrary distribution.

To install, simply apply this patch to your kernel build tree, and build as normal.

Notes:

• To turn it on, you need to switch on the relevant config file option (either CONFIG_XEN_PVDRIVERS=y or CONFIG_XEN_PVDRIVERS=m).
• The main changes are the location of files.
• I have imported a spaghetti of .h files. These are all included, one way or another. Most of them are pointlessly included. These could be pruned, but that would make them less easy to maintain.
• Supporting building otherwise than as a module (CONFIG_XEN_PVDRIVERS=y) was non-trivial because the code did all sorts of strange and I believe unnecessary things when MODULE was not defined if CONFIG_XEN was false too. I suspect this may not have been tested in recent times. Rather than work out what it was doing, I changed MODULE in the source to another keyword which is always defined as true. Though this now works, it is less tested. It seems not to modprobe the vbd devices correctly; I suspect it may need a MOD_ALIAS_BLOCKDEV_MAJOR in or something.
• I rewrote the Makefile and build system for the pvdrivers.
• I think the remaining code changes boil down to renaming the function ctrl_alt_del(), which is used elsewhere in the kernel which broke non-modular compilation.
• It will not work with CONFIG_XEN turned on. Whilst you don’t need CONFIG_XEN turned on for a fully HVM kernel, I can see why you might want a dual use kernel. The only reason it won’t work is namespace collisions. This is probably fixable at some cost to maintainability.

I was somewhat surprised to find that this is non-trivial.

The xensource hg repository contains a directory called unmodified_drivers. To be polite, the instructions for building these are woefully incomplete. Also, pulling down the entire xen source code to build a couple of drivers is a pain in the backside.

In essence, what you need to do is as follows:

• Grab a recent kernel source from kernel.org. I used 2.6.32.11.
• Find a set of patches to xenify it (I used Andrew Lyon’s Gentoo patches).
• Configure the kernel in non-Xen mode (i.e. a normal kernel, with the options you would want for full HVM).
• Build the kernel.
• Build the PV drivers.

I have simplified this process as follows:

• I wrote a little script to patch the kernel (more accurately, I adopted and adapted a perfectly useful script from Boris Derzhavets).
• I pulled the unmodified_drivers directory out of the rest of the Xen source tree, fixed a bug in a Makefile, wrote a build script, and put it in its own .tgz.

So now what you need to do is as follows (I’m building on Ubuntu Lucid, but it doesn’t much matter):

wget \
  http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.11.tar.bz2
tar xvjf linux-2.6.32.11.tar.bz2
cd linux-2.6.32.11
wget http://www.alex.org.uk/pvdrivers-0.01.tgz
tar xvzf pvdrivers-0.01.tgz
wget \
  http://gentoo-xen-kernel.googlecode.com/files/xen-patches-2.6.32-1.tar.bz2
pvdrivers-0.01/xenify.sh xen-patches-2.6.32-1.tar.bz2
# Use a normal HVM config *NOT* a Xen config
cp /boot/config-`uname -r` .config
make menuconfig
(exit)
make localmodconfig
(accept defaults)
make -j 9
cd pvdrivers-0.01
./build.sh

And that should be it. In case you are wondering where pvdrivers-0.01.tgz came from, here’s roughly how you make it:

mkdir pvdrivers
cd pvdrivers
hg clone http://xenbits.xen.org/xen-4.0-testing.hg
cd xen-4.0-testing.hg
# Fix duff makefile
perl -pi -e 's/ vbd.o$/ vbd.o vcd.o/' \
   unmodified_drivers/linux-2.6/blkfront/Kbuild
# Remove the unneeded Xen Source code
ls -1 | egrep -v 'unmodified|COPYING' | xargs rm -rf
rm -rf .hg
cd ..

You’ll then need to add the two shell scripts I wrote (or your own equivalent). The main surprises here are:

• The XL environment variable needs to point to a kernel source tree with a configured and built xenified kernel in, but one which has not been configured for Xen.
• The XEN environment variable does not need to point to a Xen source tree. Instead it needs to point to include/xen within the Linux tree.

The 3.3-testing PV drivers do not build (though it’s an easy fix, one of the hypervisor includes has moved). However, PV drivers built using this package (from 4.0-testing) seem to work just fine with older versions of Xen, so there’s not a lot of point in bothering.