Virtualization in Debian Etch (Part 3 - Xen)


In Part 1 and Part 2, we have explored running virtual machines using UML and KVM and benchmarked their performance comparing to the real machine. In Part 3, we will investigate Xen. If you are not familiar with Xen, here is the description from KernelNewbies

Xen is a hypervisor based virtualization technology originating at the University of Cambridge 
nowadays developed largely by the company XenSource. Xen introduced the concept of
paravirtualization, which allows for extremely high performance virtualization provided that the 
guest virtual machine runs a modified operating system kernel.

Follow this step-by-step instructions, you will be able to:

  1. install Xen hypervisor and related tools from the Etch repository, without building from source files or compilation of kernel
  2. create virtual guest image and install Debian Etch on it
  3. start, run and manage Xen virtual guest systems
  4. compare the performance of difference virtualization technologies - UML, KVM and Xen

1. Installing Xen kernel, hypervisor and related tools

To install the required software, run:
# apt-get install xen-linux-system-2.6.18-4-xen-686 xen-tools \
  xen-docs-3.0 libc6-xen xen-hypervisor-3.0.3-1-i386-pae \
  xen-linux-system-2.6.18-4-xen-vserver-686 bridge-utils

This will install the patched Xen kernel, the hypervisor, and all other tools that need to manage virtual machines and create virtual machine images. The heart of Xen is the hypervisor, which gets booted by the customized boot loader and controls cpu and memory, sharing them between the administrative domain (Domain 0) and the virtual guest systems.

It is now the right time to reboot. After restart, the system should display that you are running a kernel ready for Xen. That also means you are running Domain 0 (dom0):

Linux punknix-xen 2.6.18-4-xen-686 #1 SMP Thu May 10 03:24:35 UTC 2007 i686 GNU/Linux

A domain management service, Xen Daemon (Xend), performs system management functions related to virtual machines. It forms a central point of control for Domain 0 and can be controlled using an HTTP-based protocol. It must be running in order to start and manage virtual machines.

The first step for configuring xend is to setup bridge network for the guests. Edit /etc/xen/xend-config.sxp:

(network-script network-bridge)
#(network-script network-dummy)
Comment the original "(network-script network-dummy)" and enable "(network-script network-bridge)" line. Xend even allows you to define more comprehensive network setup, but network-bridge should be good enough for beginners.

Then restart Xen Daemon by:

# /etc/init.d/xend restart

2. Creating Debian Etch virtual guest system (domU)

Before we create virtual guest image, edit /etc/xen-tools/xen-tools.conf to define the default values.
#lvm = punknix-xen
dir = /home/xen
debootstrap = 1
size = 4Gb
memory = 128Mb
swap = 256Mb
fs = ext3
dist = etch
image = sparse
kernel = /boot/vmlinuz-2.6.18-4-xen-vserver-686
initrd = /boot/initrd.img-2.6.18-4-xen-vserver-686
mirror =
There are two options to store guest system in loop-back file image file or in LVM. In above xen-tools.conf, we will use the loop back image file (dir = /home/xen) to store the guest system. But you may want to consider LVM if you prefer a "standard" way in managing virtual server images. In this section, we will create guest system on a file image, but we will explore LVM in the later section:

Guest systems will be stored in /home/xen directory. Create it if it does not exist:

# mkdir /home/xen
Then, create the guest system images with the following commands:
# xen-create-image --hostname=punknix-xen1 --size=4Gb --swap=256Mb \
  --ip= --netmask=  --gateway= \
  --dir=/home/xen \
  --debootstrap --dist=etch --mirror= --passwd
xen-tools package provide handy and useful scripts to ease the creation of fully configured Xen guest domains (domU) which can be listed, updated, or copied easily. The above command is self-explanatory:
  1. creates the guest system with hostname punknix-xen1, with 4G root partition and 256MB swap
  2. assigns static IP, netmask and as default gateway.
  3. stores the guest system in /home/xen
  4. installs Debain Etch from through debootstrap and interactively asks for root password after installation
The following should be displayed while Debian Etch system image is being created:
General Information
Hostname       :  punknix-xen1
Distribution   :  etch
Fileystem Type :  ext3

Size Information
Image size     :  4Gb
Swap size      :  256Mb
Image type     :  sparse
Memory size    :  128Mb
Kernel path    :  /boot/vmlinuz-2.6.18-4-xen-vserver-686
Initrd path    :  /boot/initrd.img-2.6.18-4-xen-vserver-686

Networking Information
IP Address 1   :
Netmask        :
Gateway        :

Creating swap image: /home/xen/domains/punknix-xen1/swap.img
Creating disk image: /home/xen/domains/punknix-xen1/disk.img
Creating ext3 filesystem on /home/xen/domains/punknix-xen1/disk.img
Installing your system with debootstrap mirror
After a coffee break, root filesystem images and swap image should be created under /home/xen

3. Starting Xen virtual server

xen-create-image script automatically generates domain configuration files to ease the setup work for users. The domain configuration file of our newly created domain will be saved in /etc/xen/punknix-xen1.cfg. Before starting virtual server, you may want to review or edit the domain configuration.

Another domain management tool, xm, is the primary tool for managing Xen from the console. Use "xm create" to start the virtual server just created:

# xm create /etc/xen/punknix-xen1.cfg
Once the virtual server is started, you can connect a virtual console with command:
# xm console punknix-xen1
To verify how Xen manages the network interface, we run ifconfig on dom0:
vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:384 (384.0 b)  TX bytes:4031 (3.9 KiB)
Since we used bridge network, you can also see that Xen found a bridge interface and add the virtual interface to the bridge automatically:
# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.0016013e2bd6       no              eth1
And you can ping to domU punknix-xen1:
# ping -c 2
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=0.096 ms
64 bytes from icmp_seq=2 ttl=64 time=0.101 ms

--- ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.096/0.098/0.101/0.010 ms

4. Create Xen with LVM

This section demonstrates the use of LVM for virtual images instead of loop-back image.

Before creating guest image, make sure you have free space in the volume group.

root@punknix-xen:~# vgs
  VG          #PV #LV #SN Attr   VSize   VFree
  punknix-xen   1   8   0 wz--n- 233.52G 84.27G
Use the same xen-create-image script to create the image but this time supplied with parameter --lvm=<volume group>.
# xen-create-image --hostname=punknix-xen2 --size=4Gb --swap=256Mb \
  --ip= --netmask=  --gateway= \
  --lvm=punknix-xen \
  --debootstrap --dist=etch --mirror= --passwd
The above creates Debian Etch virtual guest image using volume group punknix-xen. After another coffee break, two logical volumes were created:
# lvscan
  ACTIVE            '/dev/punknix-xen/punknix-xen2-disk' [4.00 GB] inherit
  ACTIVE            '/dev/punknix-xen/punknix-xen2-swap' [256.00 MB] inherit
We now have two Xen virtual guest systems ready!

5. Performance

Again, we benchmark different virtualization technologies by measuring Voyage Linux kernel build time. Do you believe in near native performance in Xen? Here is the result:

Host OSAthlon 64 X2 4400+
Memory: 2GB Ram
UMLMemory: 128MB (use tmpfs)
Swap: 256MB
Disk Image: 4GB
KVMMemory: 128MB
Swap: 240MB
QEMU Disk Image: 4GB
Xen - File ImageMemory: 128MB
Swap: 256MB
File Image: 4GB
Xen - LVMMemory: 128MB
Swap: 256MB
LVM Root: 4GB

It is a bit surprise to see that running guest system on file image is faster than on LVM in a small fraction. Also, the paravirtualization approach and HVM hardware help the virtualization performance a lot.

It is known that Xen also supports full virtualization that allows running unmodified operating systems such as Windows on processors with Intel VT or AMD-V technology. We will investigate full virtualization in Xen in the coming future. Meanwhile, you may refer to this thread for setup example.


The below lists additional references for writing up this article:
[1] -
[2] -
[3] -
[4] -