XEN with live migration with Pacemaker and DRBD

On both members of the cluster

• /etc/modprobe.conf

options drbd disable_sendpage=1

• /etc/lvm/lvm.conf

filter = [ "r|/dev/cdrom|", "r|/dev/drbd.*|" ]

• create logical volume

lvcreate --size 320G --name vsvn vg1

• /etc/drbd.conf

Make sure device name and port numbers are unique

resource vsvn {
  net {
    allow-two-primaries;
  }
  on xen-11.dev.videonext.net {
    device    /dev/drbd2;
    disk      /dev/vg1/vsvn;
    address   10.0.0.1:7787;
    flexible-meta-disk internal;
  }
  on xen-12.dev.videonext.net {
    device    /dev/drbd2;
    disk      /dev/vg1/vsvn;
    address   10.0.0.2:7787;
    flexible-meta-disk internal;
  }
}

• Create drbd device

drbdadm create-md vsvn

• Bring it up

drbdadm up vsvn

• Make sure resource is in Connected state

# cat /proc/drbd 
 2: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:335534044

• initialize meta-data (run this command on one of the servers)

 drbdadm -- --clear-bitmap new-current-uuid vsvn

• Make sure resource is UpToDate

# cat /proc/drbd 
 2: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r----
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

• Make resource is primary on one of the host, where you intend to run installation process

drbdadm primary vsvn

• install VM, for example

virt-install --name vsvn --ram=512 --vcpus=1 -p --disk=path=/dev/drbd/by-res/vsvn \
--nographics --location=http://ftp.videonext.net/private/CentOS/5.4/x86_64/os \
--extra-args="noipv6 headless clocksource=jiffies dhcpclass=intserver ks=http://noc.videonext.net/cfg/vsvn.cfg"

• After installation complete

Make sure you have copies of you new initrd and vmlinuz in /xen/, they can be shared between different VM, but proper kernel version number needs to be installed in VM as well, if not just scp /boot/initrd* and /boot/vmlinuz* from VM to xen-11:/xen/

• Shutdown VM

xm shutdown vsvn

• Create configuration file xen-11, it will be propagated to xen-12 via rsync automatically

cp /etc/xen/vsvn /xen/vsvn.cfg
vi /xen/vsvn.cfg

replace bootloader with kernel,ramdisk,extra; modify disk definition to use drbd block device

cat /xen/vsvn.cfg

name = "vsvn"
uuid = "73ff1420-cc5e-ebfc-9046-b25e5e6b5b6f"
maxmem = 512
memory = 512
vcpus = 1
kernel = "/xen/vmlinuz-2.6.18-164.15.1.el5xen"
ramdisk = "/xen/initrd-2.6.18-164.15.1.el5xen.img"
extra = "ro root=/dev/xvda1 console=xvc0"
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "restart"
disk = [ "drbd:vsvn,xvda,w" ]
vif = [ "mac=00:16:36:4e:40:e1,bridge=xenbr0,script=vif-bridge" ]

• make sure xen relocation is enabled

grep ^\(xend-relocation /etc/xen/xend-config.sxp 
(xend-relocation-server yes)
(xend-relocation-port 8002)
(xend-relocation-address '')
(xend-relocation-hosts-allow '')

• create Pacemaker resource

crm configure primitive xen_vsvn ocf:heartbeat:Xen \
params xmfile="/xen/vsvn.cfg" \
op start interval="0" timeout="60" \
op stop interval="0" timeout="300" \
op monitor start-delay="120" interval="60" timeout="30" \
meta allow-migrate="true"

• Enjoy

# crm resource show xen_vsvn
resource xen_vsvn is running on: xen-12.dev.videonext.net