Ceci est une ancienne révision du document !
Contenue du paquet:
- script de sauvegarde virt-backup.pl
- script init virt-state pour hiberner/réveiller les VM lors de l'arrêt/démarrage de l'hôte
- script init virt-perm pour mettre les permissions qui vont bien (DAC et SELinux) sur les différentes ressources
- script init ksm pour lancer le scanner KSM (attendre correctif du module qui supprime le leak ?)
Script d'hibernation/réveil des VM lors de l'arrête/démarrage de l'hôte
#!/bin/sh # the following is chkconfig init header # # virt-helper: save and restore VM state # # chkconfig: 345 99 01 # description: Libvirt helper script to save and load VM # state when the host shutdown and start # ensure also correct permissions and SELinux # # Source function library. . /etc/rc.d/init.d/functions . /etc/sysconfig/virt-addons VIRSH=$(which virsh) VIRSH=$VIRSH" -c "$LIBVIRT_URI RETVAL=0 ERROR=0 hibernate(){ echo -n $"Saving Virtual Machines states: " # For each running VM (or paused ones), save the state in $STATE_DIR for VM in $(LANG=C $VIRSH list | egrep '(running|paused)' | awk {'print $2'}); do echo -n $"Saving $VM state: " $VIRSH save $VM $STATE_DIR/$VM.state 2>&1 > /dev/null if [ $? = 0 ]; then success; echo else ERROR=1 error; echo fi done } wakup(){ ERROR=0 # For each state file found in $STATE_DIR, try to restore it # and remove the file if we succed, else report an error for VM in $(ls $STATE_DIR/*.state 2>/dev/null); do echo -n $"Restoring $VM state: " $VIRSH restore $VM 2>&1 > /dev/null if [ $? = 0 ]; then sucess; echo rm -f $VM else error; echo ERROR=1 fi done } set_perms(){ for DIR in /var/lib/libvirt/qemu /var/run/libvirt/qemu /var/cache/libvirt/qemu; do [ -d $DIR ] || mkdir -p $DIR chown $KVMUSER:$KVMGROUP $DIR || ERROR=1 done } set_selinux(){ # Fix SELinux context for all storage support used by a VM echo -n $"Fixing SELinux contexts: " VMS=$(LANG=C virsh list --all | egrep '(running|shut off|paused)' | awk {'print $2'}) VOLS=$(for VM in $VMS;do virsh dumpxml $VM | xmlstarlet sel -t -m \ "/domain/devices/disk/source" -v @dev -v @file -n; done | grep '/') for VOL in $VOLS;do chcon -t virt_image_t $VOL || ERROR=1 done if [ $(grep -c virt-addons $(semodule -l)) = 0]; then semodule -i /usr/share/selinux/targeted/virt-addons.pp || ERROR=1 fi if [ $ERROR = 0 ]; then sucess; echo rm -f $VM else error; echo ERROR=1 fi } start_ksm(){ modprobe ksm 2>&1 > /dev/null chown :$KVMUSER /dev/ksm chmod 660 /dev/ksm ksmctl start $NBPAGES $SLEEP 2>&1 > /dev/null } stop_ksm(){ ksmctl stop 2&1 > /dev/null } start(){ [ $STATE ] && wakup [ $SET_PERM ] && set_perms [ $SET_SELINUX ] && set_selinux [ $KSM ] && start_ksm if [ $ERROR = 1 ]; then RETVAL=1 fi } stop(){ [ $STATE ] && hibernate [ $KSM ] && stop_ksm if [ $ERROR = 1 ]; then RETVAL=1 fi } restart(){ stop && start } case "$1" in start|stop|restart) $1 ;; *) echo $"Usage: $0 {start|stop|restart}" exit 1 ;; esac exit $RETVAL
/etc/sysconfig/virt-addons:
# Should VM states should be saved when host shutdown and restored when started # This acts as an auto hibernate function for VM, and make rebooting guests # independantly from the host STATE=true # Should permissions should be set on some directory SET_PERM=true # Should SELinux functions (fixing context on devices, and allowing save/restore on # systems using SELinux enforcing SET_SELINUX=true # Enable KSM KSM=true # Default URI for libvirt LIBVIRT_URI="qemu:///system" # User and group kvm runs as (as configured in /etc/libvirt/qemu.conf) KVMUSER="qemu" KVMGROUP="qemu"
Script de configuration des permissions qui vont bien
Un chcon tout les supports utilisés par une VM:
# Fix SELinux context for all storage support used by a VM VMS=$(LANG=C virsh list --all | egrep '(running|shut off|paused)' | awk {'print $2'}) VOLS=$(for VM in $VMS;do virsh dumpxml $VM | xmlstarlet sel -t -m \ "/domain/devices/disk/source" -v @dev -v @file -n; done | grep '/') for VOL in $VOLS;do chcon -t virt_image_t $VOL done # And fix permissions on some directories for DIR in /var/lib/libvirt/qemu /var/run/libvirt/qemu /var/cache/libvirt/qemu; do [ -d $DIR ] || mkdir -p $DIR chown $KVMUSER:$KVMGROUP $DIR done
:
- Et pour les volumes créés après le démarrage ? il faut lancer à la mains le script :/
- Ça serait probablement plus élégant d'utiliser les nouveaux Hooks (synchrones): on chcon uniquement les volumes utilisé par le domaine démarré, juste avant qu'il ne démarre. En espérant que ces hooks soient intégrés à libvirt 0.7.8. Les permissions sur les répertoires pourraient être vérifiés sur le hook du démon, alors que les permissions sur les volumes utiliseraient les hooks qemu
Lancement de KSM
modprobe ksm chown :$KVMUSER /dev/ksm ksmctl start $NBPAGES $SLEEP 2>&1 > /dev/null
:
- Dommage qu'on ait pas les stats de KSM disponibles dans /sys, ça permettrait d'utiliser les scripts ksm et ksmtuned de fedora (adaptation auto de l'agressivité de ksm en fonction de la charge du serveur)
Script de sauvegarde
#!/usr/bin/perl -w # AUTHOR # Daniel Berteaud <daniel@firewall-services.com> # # COPYRIGHT # Copyright (C) 2009 Daniel Berteaud # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # This script allows you to backup Virtual Machines managed by libvirt. # It has only be tested with KVM based VM # This script will dump: # * each block devices # * optionnally the memory (if --state flag is given) # * the XML description of the VM # These files are writen in a temporary backup dir. Everything is done # in order to minimize donwtime of the guest. For example, it takes # a snapshot of the block devices (if backed with LVM) so the guest is # just paused for a couple of seconds. Once this is done, the guest is # resumed, and the script starts to dump the snapshot. # Once a backup is finished, you'll have several files in the backup # directory. Let's take an example with a VM called my_vm which has # two virtual disks: hda and hdb. You have passed the --state flag: # * my_vm.lock: lock file to prevent another backup to run at the same time # * my_vm.xml: this file is the XML description of the VM (for libvirt configuraiton) # * my_vm_hda.img: this file is an image of the hda drive of the guest # * my_vm_hdb.img: this file is an image of the hdb drive of the guest # * my_vm.state: this is a dump of the memory (result of virsh save my_vm my_vm.state) # This script was made to be ran with BackupPC pre/post commands. # In the pre-backup phase, you dump everything then, backuppc backups, # compress, pools etc... the dumped file. Eventually, when the backup is finished # The script is called with the --cleanup flag, which cleanups everything. # Some examples: # # Backup the VM named mail01 and devsrv. Also dump the memory. # Exclude any virtual disk attached as vdb or hdb and on the fly # compress the dumped disks (uses gzip by default) # virt-backup.pl --dump --vm=mail01,devsrv --state --exclude=vdb,hdb --compress # Remove all the files related to mail01 VM in the backup directory # virt-backup.pl --cleanup --vm=mail01 # Backup devsrv, use 10G for LVM snapshots (if available), do not dump the memory # (the guest will just be paused while we take a snapshot) # Keep the lock file present after the dump # virt-backup.pl --dump --vm=devsrv --snapsize=10G --keep-lock # Backup devsrv, and disable LVM snapshots # virt-backup.pl --dump --vm=devsrv --no-snapshot # Backup mail01, and enable debug (verbose output) # virt-backup.pl --dump --vm=mail01 --debug ### TODO: # - Make it more robust (script crash sometime while trying to restore # Probably a bug somewhere between libvirt and Sys::Virt) # - Test with images as backend. Should just work, but without the snapshot function # - Add snapshot (LVM) support for image based disk ? (should we detect the mount moint, and block device # of the storage or let the user specify it with a --logical ?) # - Additionnal check that the vm is available after a restore (via $dom->get_info->{status}, ping ?) # - Check if compression utilies are available # - Support per vm excludes in one run ### CHANGES # * 26/03/2010 # - Initial packaged version use XML::Simple; use Sys::Virt; use Getopt::Long; # Some constant our %opts = (); our @vms = (); our @excludes = (); # Sets some defaults values $opts{backupdir} = '/var/lib/libvirt/backup'; $opts{snapsize} = '5G'; $opts{state} = 0; $opts{debug} = 0; $opts{keeplock} = 0; $opts{snapshot} = 1; $opts{connect} = "qemu:///system"; $opts{compress} = 'none'; $opts{lvcreate} = '/usr/sbin/lvcreate'; $opts{lvremove} = '/usr/sbin/lvremove'; $opts{nice} = 'nice -n 19'; $opts{ionice} = 'ionice -c 2 -n 7'; $opts{livebackup} = 1; $opts{wasrunning} = 1; $opts{bs} = "1M"; # get command line arguments GetOptions( "debug" => \$opts{debug}, "keep-lock" => \$opts{keeplock}, "state" => \$opts{state}, "snapsize=s" => \$opts{snapsize}, "backupdir=s" => \$opts{backupdir}, "vm=s" => \@vms, "cleanup" => \$opts{cleanup}, "dump" => \$opts{dump}, "connect=s" => \$opts{connect}, "snapshot!" => \$opts{snapshot}, "compress:s" => \$opts{compress}, "exclude=s" => \@excludes, "bs=s" => \$opts{bs}, "help" => \$opts{help} ); # Set compression settings if ($opts{compress} eq 'lzop'){ $opts{compext} = ".lzo"; $opts{compcmd} = "lzop -c"; } elsif ($opts{compress} eq 'bzip2'){ $opts{compext} = ".bz2"; $opts{compcmd} = "bzip2 -c"; } elsif ($opts{compress} eq 'pbzip2'){ $opts{compext} = ".bz2"; $opts{compcmd} = "pbzip2 -c"; } elsif ($opts{compress} eq 'xz'){ $opts{compext} = ".xz"; $opts{compcmd} = "xz -c"; } # Default is gzip elsif (($opts{compress} eq 'gzip') || ($opts{compress} eq '')) { $opts{compext} = ".gz"; $opts{compcmd} = "gzip -c"; } else{ $opts{compext} = ""; $opts{compcmd} = "cat"; } # Allow comma separated multi-argument @vms = split(/,/,join(',',@vms)); @excludes = split(/,/,join(',',@excludes)); # Stop here if we either have dump and cleanup, no dump and no cleanup, no vm # Or the help flag is present if ( ($opts{dump} && $opts{cleanup}) || (!$opts{dump} && !$opts{cleanup}) || (!@vms) || ($opts{help}) ){ usage(); exit 1; } if (! -d $opts{backupdir} ){ print "$opts{backupdir} is not a valid directory\n"; exit 1; } # Connect to libvirt print "\n\nConnecting to libvirt daemon using $opts{connect} as URI\n" if ($opts{debug}); our $libvirt = Sys::Virt->new( uri => $opts{connect} ) || die "Error connecting to libvirt on URI: $opts{connect}"; print "\n" if ($opts{debug}); foreach our $vm (@vms){ # Create a new object representing the VM print "Checking $vm status\n\n" if ($opts{debug}); our $dom = $libvirt->get_domain_by_name($vm) || die "Error opening $vm object"; if ($opts{dump}){ print "Running dump routine for $vm, as requested by the --dump flag\n\n" if ($opts{debug}); $opts{backupdir} .= '/'.$vm; mkdir $opts{backupdir} || die $!; run_dump(); } elsif ($opts{cleanup}){ print "Running cleanup routine for $vm, as requested by the --cleanup flag\n\n" if ($opts{debug}); run_cleanup(); } else { usage(); exit 1; } } ############################################################################ ############## FUNCTIONS #################### ############################################################################ sub run_dump{ # Create a new XML object my $xml = new XML::Simple (); my $data = $xml->XMLin( $dom->get_xml_description() ); # STop here if the lock file is present, another dump might be running die "Another backup is running\n" if ( -e "$opts{backupdir}/$vm.lock" ); # Lock VM: Create a lock file so only one dump process can run lock_vm(); # Save the XML description save_xml(); # Save the VM state if it's running and --state is present # (else, just suspend the VM) $opts{wasrunning} = 0 unless ($dom->is_active()); if ($opts{wasrunning}){ if ($opts{state}){ save_vm_state(); } else{ suspend_vm(); } } my @disks; # Create a list of disks used by the VM foreach $disk (@{$data->{devices}->{disk}}){ my $source; if ($disk->{type} eq 'block'){ $source = $disk->{source}->{dev}; } elsif ($disk->{type} eq 'file'){ $source = $disk->{source}->{file}; } else{ print "\nSkiping $source for vm $vm as it's type is $disk->{type}: " . " and only block and file are supported\n" if ($opts{debug}); next; } my $target = $disk->{target}->{dev}; # Check if the current disk is not excluded if (!!grep { $_ eq "$target" } @excludes){ print "\nSkiping $source for vm $vm as it's matching one of the excludes: " . join(",",@excludes)."\n\n" if ($opts{debug}); next; } # If the device is a disk (and not a cdrom) and the source dev exists if (($disk->{device} eq 'disk') && (-e $source) && (!$disk->{readonly})){ print "\nAnalysing disk $source connected on $vm as $target\n\n" if ($opts{debug}); # If it's a block device if ($disk->{type} eq 'block'){ my $time = "_".time(); # Try to snapshot the source if snapshot is enabled if ( ($opts{snapshot}) && (create_snapshot($source,$time)) ){ print "$source seems to be a valid logical volume (LVM), a snapshot has been taken as " . $source . $time ."\n" if ($opts{debug}); $source = $source.$time; push (@disks, {source => $source, target => $target, type => 'snapshot'}); } # Snapshot failed, or disabled: disabling live backups else{ if ($opts{snapshot}){ print "Snapshoting $source has failed (not managed by LVM, or already a snapshot ?)" . ", live backup will be disabled\n" if ($opts{debug}) ; } else{ print "Not using LVM snapshots, live backups will be disabled\n" if ($opts{debug}); } $opts{livebackup} = 0; push (@disks, {source => $source, target => $target, type => 'block'}); } } elsif ($disk->{type} eq 'file'){ $opts{livebackup} = 0; push (@disks, {source => $source, target => $target, type => 'file'}); } print "Adding $source to the list of disks to be backed up\n" if ($opts{debug}); } } # Summarize the list of disk to be dumped if ($opts{debug}){ print "\n\nThe following disks will be dumped:\n\n"; foreach $disk (@disks){ print "Source: $disk->{source}\tDest: $opts{backupdir}/$vm" . '_' . $disk->{target} . ".img$opts{compext}\n"; } print "\n"; } # If livebackup is possible (every block devices can be snapshoted) # We can restore the VM now, in order to minimize the downtime if ($opts{livebackup}){ print "\nWe can run a live backup\n" if ($opts{debug}); if ($opts{wasrunning}){ if ($opts{state}){ restore_vm(); } else{ resume_vm(); } } } # Now, it's time to actually dump the disks foreach $disk (@disks){ my $source = $disk->{source}; my $dest = "$opts{backupdir}/$vm" . '_' . $disk->{target} . ".img$opts{compext}"; print "\nStarting dump of $source to $dest\n" if ($opts{debug}); my $ddcmd = "$opts{ionice} dd bs=$opts{bs} if=$source 2>/dev/null | $opts{nice} $opts{compcmd} > $dest 2>/dev/null"; unless( system("$ddcmd") == 0 ){ die "Couldn't dump the block device/file $source to $dest with dd\n"; } # Remove the snapshot if the current dumped disk is a snapshot destroy_snapshot($source) if ($disk->{type} eq 'snapshot'); } # If the VM was running before the dump, restore (or resume) it if ($opts{wasrunning}){ if ($opts{state}){ restore_vm(); } else{ resume_vm(); } } # And remove the lock file, unless the --keep-lock flag is present unlock_vm() unless ($opts{keeplock}); } # Remove the dumps sub run_cleanup{ print "\nRemoving backup files\n" if ($opts{debug}); my $cnt = 0; $cnt= unlink <$opts{backupdir}/$vm/*>; rmdir "$opts{backupdir}/$vm"; print "$cnt file(s) removed\n" if $opts{debug}; } sub usage{ print "usage:\n$0 --dump|--cleanup --vm=name[,vm2,vm3] [--debug] [--exclude=hda,hdb] [--compress] ". "[--state] [--no-snapshot] [--snapsize=<size>] [--backupdir=/path/to/dir] [--connect=<URI>] ". "[--keep-lock] [--bs=<block size>]\n" . "\n\nMandatory options:\n" . "\t--dump: Run the dump routine (dump disk image to temp dir, pausing the VM if needed)\n\n" . "\t--cleanup: Run the cleanup routine, cleaning up the backup dir\n\n" . "\t\tNote: --dump and --cleanup are mutually exclusive\n\n" . "\t--vm=name: The VM you want to work on (as known by libvirt). You can backup several VMs in one shot " . "if you separate them with comma, or with multiple --vm argument. You have to use the name of the domain, ". "ID and UUID is not supported at the moment\n\n" . "\n\nOther options:\n\n" . "\t--state: Cleaner way to take backups. If this flag is present, the script will save the current state of " . "the VM (if running) instead of just suspending it. With this you should be able to restore the VM at " . "the exact state it was when the backup started. The reason this flag is optional is that some guests " . "crashes after the restoration, especially when using the kvm-clock. Test this functionnality with" . "your environnement before using this flag on production\n\n" . "\t--no-snapshot: Do not attempt to use LVM snapshots. If not present, the script will try to take a snapshot " . "of each disk of type 'block'. If all disk can be snapshoted, the VM is resumed, or restored (depending " . "on the --state flag) immediatly after the snapshots have been taken, resulting in almost no downtime. " . "This is called a \"live backup\" in this script" . "If at least one disk cannot be snapshoted, the VM is suspended (or stoped) for the time the disks are " . "dumped in the backup dir. That's why you should use a fast support for the backup dir (fast disks, RAID0 " . "or RAID10)\n\n" . "\t--snapsize=<snapsize>: The amount of space to use for snapshots. Use the same format as -L option of lvcreate. " . "eg: --snapsize=15G. Default is 5G\n\n" . "\t--compress[=[gzip|bzip2|pbzip2|lzop|xz]]: On the fly compress the disks images during the dump. If you " . "don't specify a compression algo, gzip will be used.\n\n" . "\t--exclude=hda,hdb: Prevent the disks listed from being dumped. The names are from the VM perspective, as " . "configured in livirt as the target element. It can be usefull for example if you want to dump the system " . "disk of a VM, but not the data one which can be backed up separatly, at the files level.\n\n" . "\t--backupdir=/path/to/backup: Use an alternate backup dir. The directory must exists and be writable. " . "The default is /var/lib/libvirt/backup\n\n" . "\t--connect=<URI>: URI to connect to libvirt daemon (to suspend, resume, save, restore VM etc...). " . "The default is qemu:///system.\n\n" . "\t--keep-lock: Let the lock file present. This prevent another " . "dump to run while an third party backup software (BackupPC for example) saves the dumped files.\n\n" . "\t--bs: Specify a custom block size for the dump (which uses dd). like --bs=10M. The format is the same as the " . "bs option of dd\n\n" } # Save a running VM, if it's running sub save_vm_state{ if ($dom->is_active()){ print "$vm is running, saving state....\n" if ($opts{debug}); $dom->save("$opts{backupdir}/$vm.state"); print "$vm state saved as $opts{backupdir}/$vm.state\n" if ($opts{debug}); } else{ print "$vm is not running, nothing to do\n" if ($opts{debug}); } } # Restore the state of a VM sub restore_vm{ if (! $dom->is_active()){ if (-e "$opts{backupdir}/$vm.state"){ print "\nTrying to restore $vm from $opts{backupdir}/$vm.state\n" if ($opts{debug}); $libvirt->restore_domain("$opts{backupdir}/$vm.state"); print "Waiting for restoration to complete\n" if ($opts{debug}); my $i = 0; while ((!$dom->is_active()) && ($i < 120)){ sleep(5); $i = $i+5; } print "Timeout while trying to restore $vm, aborting\n" if (($i > 120) && ($opts{debug})); } else{ print "\nRestoration impossible, $opts{backupdir}/$vm.state is missing\n" if ($opts{debug}); } } else{ print "\nCannot start domain restoration, $vm is running (maybe already restored after a live backup ?)\n" if ($opts{debug}); } } # Suspend a VM sub suspend_vm(){ if ($dom->is_active()){ print "$vm is running, suspending\n" if ($opts{debug}); $dom->suspend(); print "$vm now suspended\n" if ($opts{debug}); } else{ print "$vm is not running, nothing to do\n" if ($opts{debug}); } } # Resume a VM if it's paused sub resume_vm(){ if ($dom->get_info->{state} == Sys::Virt::Domain::STATE_PAUSED){ print "$vm is suspended, resuming\n" if ($opts{debug}); $dom->resume(); print "$vm now resumed\n" if ($opts{debug}); } else{ print "$vm is not suspended, nothing to do\n" if ($opts{debug}); } } # Dump the domain description as XML sub save_xml{ print "\nSaving XML description for $vm to $opts{backupdir}/$vm.xml\n" if ($opts{debug}); open(XML, ">$opts{backupdir}/$vm" . ".xml") || die $!; print XML $dom->get_xml_description(); close XML; } # Create an LVM snapshot # Pass the original logical volume and the suffix # to be added to the snapshot name as arguments sub create_snapshot{ my ($blk,$suffix) = @_; my $ret = 0; print "Running: $opts{lvcreate} -p r -s -n " . $blk . $suffix . " -L $opts{snapsize} $blk > /dev/null 2>&1\n" if $opts{debug}; if ( system("$opts{lvcreate} -s -n " . $blk . $suffix . " -L $opts{snapsize} $blk > /dev/null 2>&1") == 0 ) { $ret = 1; } return $ret; } # Remove an LVM snapshot sub destroy_snapshot{ my $ret = 0; my ($snap) = @_; print "Removing snapshot $snap\n" if $opts{debug}; if (system ("$opts{lvremove} -f $snap > /dev/null 2>&1") == 0 ){ $ret = 1; } return $ret; } # Lock a VM backup dir # Just creates an empty lock file sub lock_vm{ print "Locking $vm\n" if $opts{debug}; open ( LOCK, ">$opts{backupdir}/$vm.lock" ) || die $!; print LOCK ""; close LOCK; } # Unlock the VM backup dir # Just removes the lock file sub unlock_vm{ print "Removing lock file for $vm\n" if $opts{debug}; unlink <$opts{backupdir}/$vm.lock>; }
Dépendances
à mettre dans le rpm:
- perl-Sys-Virt
- perl-XML-Simple
- xmlstarlet
- lzop
- pbzip2
- xz