Virtualization in Debian Etch (Part 3 - Xen)

In Part 1 and Part 2, we have explored running virtual machines using UML and KVM and benchmarked their performance comparing to the real machine. In Part 3, we will investigate Xen. If you are not familiar with Xen, here is the description from KernelNewbies

---

Xen is a hypervisor based virtualization technology originating at the University of Cambridge 
nowadays developed largely by the company XenSource. Xen introduced the concept of
paravirtualization, which allows for extremely high performance virtualization provided that the 
guest virtual machine runs a modified operating system kernel.
...

Follow this step-by-step instructions, you will be able to:

1. install Xen hypervisor and related tools from the Etch repository, without building from source files or compilation of kernel
2. create virtual guest image and install Debian Etch on it
3. start, run and manage Xen virtual guest systems
4. compare the performance of difference virtualization technologies - UML, KVM and Xen

1. Installing Xen kernel, hypervisor and related tools

To install the required software, run:

# apt-get install xen-linux-system-2.6.18-4-xen-686 xen-tools \
  xen-docs-3.0 libc6-xen xen-hypervisor-3.0.3-1-i386-pae \
  xen-linux-system-2.6.18-4-xen-vserver-686 bridge-utils

This will install the patched Xen kernel, the hypervisor, and all other tools that need to manage virtual machines and create virtual machine images. The heart of Xen is the hypervisor, which gets booted by the customized boot loader and controls cpu and memory, sharing them between the administrative domain (Domain 0) and the virtual guest systems.

It is now the right time to reboot. After restart, the system should display that you are running a kernel ready for Xen. That also means you are running Domain 0 (dom0):

Linux punknix-xen 2.6.18-4-xen-686 #1 SMP Thu May 10 03:24:35 UTC 2007 i686 GNU/Linux

A domain management service, Xen Daemon (Xend), performs system management functions related to virtual machines. It forms a central point of control for Domain 0 and can be controlled using an HTTP-based protocol. It must be running in order to start and manage virtual machines.

The first step for configuring xend is to setup bridge network for the guests. Edit /etc/xen/xend-config.sxp:

(network-script network-bridge)
#(network-script network-dummy)

Comment the original "(network-script network-dummy)" and enable "(network-script network-bridge)" line. Xend even allows you to define more comprehensive network setup, but network-bridge should be good enough for beginners.

Then restart Xen Daemon by:

# /etc/init.d/xend restart

2. Creating Debian Etch virtual guest system (domU)

Before we create virtual guest image, edit /etc/xen-tools/xen-tools.conf to define the default values.

#lvm = punknix-xen
dir = /home/xen
debootstrap = 1
size = 4Gb
memory = 128Mb
swap = 256Mb
fs = ext3
dist = etch
image = sparse
kernel = /boot/vmlinuz-2.6.18-4-xen-vserver-686
initrd = /boot/initrd.img-2.6.18-4-xen-vserver-686
mirror = http://ftp.hk.debian.org/debian/

There are two options to store guest system in loop-back file image file or in LVM. In above xen-tools.conf, we will use the loop back image file (dir = /home/xen) to store the guest system. But you may want to consider LVM if you prefer a "standard" way in managing virtual server images. In this section, we will create guest system on a file image, but we will explore LVM in the later section:

Guest systems will be stored in /home/xen directory. Create it if it does not exist:

# mkdir /home/xen

Then, create the guest system images with the following commands:

# xen-create-image --hostname=punknix-xen1 --size=4Gb --swap=256Mb \
  --ip=192.168.1.93 --netmask=255.255.255.0  --gateway=192.168.1.1 \
  --dir=/home/xen \
  --debootstrap --dist=etch --mirror=http://ftp.hk.debian.org/debian/ --passwd

xen-tools package provide handy and useful scripts to ease the creation of fully configured Xen guest domains (domU) which can be listed, updated, or copied easily. The above command is self-explanatory:

1. creates the guest system with hostname punknix-xen1, with 4G root partition and 256MB swap
2. assigns static IP 192.168.1.93, netmask 255.255.255.0 and 192.168.1.1 as default gateway.
3. stores the guest system in /home/xen
4. installs Debain Etch from ftp.hk.debian.org/debian through debootstrap and interactively asks for root password after installation

The following should be displayed while Debian Etch system image is being created:

========================================================
General Information
--------------------
Hostname       :  punknix-xen1
Distribution   :  etch
Fileystem Type :  ext3

Size Information
----------------
Image size     :  4Gb
Swap size      :  256Mb
Image type     :  sparse
Memory size    :  128Mb
Kernel path    :  /boot/vmlinuz-2.6.18-4-xen-vserver-686
Initrd path    :  /boot/initrd.img-2.6.18-4-xen-vserver-686

Networking Information
----------------------
IP Address 1   : 192.168.1.93
Netmask        : 255.255.255.0
Gateway        : 192.168.1.1

Creating swap image: /home/xen/domains/punknix-xen1/swap.img
Done
Creating disk image: /home/xen/domains/punknix-xen1/disk.img
Done
Creating ext3 filesystem on /home/xen/domains/punknix-xen1/disk.img
Done
Installing your system with debootstrap mirror http://ftp.hk.debian.org/debian/
Done
...
========================================================

After a coffee break, root filesystem images and swap image should be created under /home/xen

  /home/xen/domains/punknix-xen1/disk.img
  /home/xen/domains/punknix-xen1/swap.img

3. Starting Xen virtual server

xen-create-image script automatically generates domain configuration files to ease the setup work for users. The domain configuration file of our newly created domain will be saved in /etc/xen/punknix-xen1.cfg. Before starting virtual server, you may want to review or edit the domain configuration.

Another domain management tool, xm, is the primary tool for managing Xen from the console. Use "xm create" to start the virtual server just created:

# xm create /etc/xen/punknix-xen1.cfg

Once the virtual server is started, you can connect a virtual console with command:

# xm console punknix-xen1

To verify how Xen manages the network interface, we run ifconfig on dom0:

vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:384 (384.0 b)  TX bytes:4031 (3.9 KiB)

Since we used bridge network, you can also see that Xen found a bridge interface and add the virtual interface to the bridge automatically:

# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.0016013e2bd6       no              eth1
                                                        eth2
                                                        vif1.0

And you can ping to domU punknix-xen1:

# ping -c 2 192.168.1.93
PING 192.168.1.93 (192.168.1.93) 56(84) bytes of data.
64 bytes from 192.168.1.93: icmp_seq=1 ttl=64 time=0.096 ms
64 bytes from 192.168.1.93: icmp_seq=2 ttl=64 time=0.101 ms

--- 192.168.1.93 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.096/0.098/0.101/0.010 ms

4. Create Xen with LVM

This section demonstrates the use of LVM for virtual images instead of loop-back image.

Before creating guest image, make sure you have free space in the volume group.

root@punknix-xen:~# vgs
  VG          #PV #LV #SN Attr   VSize   VFree
  punknix-xen   1   8   0 wz--n- 233.52G 84.27G

Use the same xen-create-image script to create the image but this time supplied with parameter --lvm=<volume group>.

# xen-create-image --hostname=punknix-xen2 --size=4Gb --swap=256Mb \
  --ip=192.168.1.94 --netmask=255.255.255.0  --gateway=192.168.1.1 \
  --lvm=punknix-xen \
  --debootstrap --dist=etch --mirror=http://ftp.hk.debian.org/debian/ --passwd

The above creates Debian Etch virtual guest image using volume group punknix-xen. After another coffee break, two logical volumes were created:

# lvscan
  ...
  ACTIVE            '/dev/punknix-xen/punknix-xen2-disk' [4.00 GB] inherit
  ACTIVE            '/dev/punknix-xen/punknix-xen2-swap' [256.00 MB] inherit

We now have two Xen virtual guest systems ready!

5. Performance

Again, we benchmark different virtualization technologies by measuring Voyage Linux kernel build time. Do you believe in near native performance in Xen? Here is the result:

	spec	real	user	sys
Host OS	Athlon 64 X2 4400+ Memory: 2GB Ram	8m57.566s	8m13.147s	0m43.487s
UML	Memory: 128MB (use tmpfs) Swap: 256MB Disk Image: 4GB	18m12.380s	6m36.030s	2m5.140s
KVM	Memory: 128MB Swap: 240MB QEMU Disk Image: 4GB	24m5.904s	20m0.371s	3m56.707s
Xen - File Image	Memory: 128MB Swap: 256MB File Image: 4GB	9m15.866s	8m5.950s	1m1.852s
Xen - LVM	Memory: 128MB Swap: 256MB LVM Root: 4GB	9m37.698s	8m3.602s	1m3.780s

It is a bit surprise to see that running guest system on file image is faster than on LVM in a small fraction. Also, the paravirtualization approach and HVM hardware help the virtualization performance a lot.

It is known that Xen also supports full virtualization that allows running unmodified operating systems such as Windows on processors with Intel VT or AMD-V technology. We will investigate full virtualization in Xen in the coming future. Meanwhile, you may refer to this thread for setup example.

Reference:

The below lists additional references for writing up this article:
[1] - http://www.howtoforge.com/debian_etch_xen_from_debian_repository
[2] - http://wiki.kartbuilding.net/index.php/Debian_Etch_Xen_Install
[3] - http://www.cosmocode.de/en/blogs/gohr/20070130123639/index.html
[4] - http://etbe.blogspot.com/2007/01/installing-xen-domu-on-debian-etch.html