Script lvmstrap

Program which configures LVM on the Ganeti nodes.

This program wipes disks and creates a volume group on top of them. It can also show disk information to help you decide which disks you want to wipe.

The error handling is done by raising our own exceptions from most of the functions; these exceptions then handled globally in the main() function. The exceptions that each function can raise are not documented individually, since almost every error path ends in a raise.

Another two exceptions that are handled globally are IOError and OSError. The idea behind this is, since we run as root, we should usually not get these errors, but if we do it's most probably a system error, so they should be handled and the user instructed to report them.

Classes

[hide private]

Error
Generic exception

ProgrammingError
Exception denoting invalid assumptions in programming.

SysconfigError
Exception denoting invalid system configuration.

PrereqError
Exception denoting invalid prerequisites.

OperationalError
Exception denoting actual errors.

ParameterError
Exception denoting invalid input from user.

Functions

[hide private]

Usage()
Shows program usage information and exits the program.

tuple

ParseOptions()
Parses the command line options.

IsPartitioned(disk)
Returns whether a given disk should be used partitioned or as-is.

DeviceName(disk)
Returns the appropriate device name for a disk.

SysfsName(disk)
Returns the sysfs name for a disk or partition.

tuple

ExecCommand(command)
Executes a command.

CheckPrereq()
Check the prerequisites of this program.

CheckVGExists(vgname)
Checks to see if a volume group exists.

CheckSysDev(name, devnum)
Checks consistency between /sys and /dev trees.

ReadDev(syspath)
Reads the device number from a sysfs path.

int

ReadSize(syspath)
Reads the size from a sysfs path.

ReadPV(name)
Reads physical volume information.

GetDiskList(opts)
Computes the block device list for this system.

dict

GetMountInfo()
Reads /proc/mounts and computes the mountpoint-devnum mapping.

GetSwapInfo()
Reads /proc/swaps and returns the list of swap backing stores.

DevInfo(name, dev, mountinfo)
Computes miscellaneous information about a block device.

ShowDiskInfo(opts)
Shows a nicely formatted block device list for this system.

boolean

CheckSysfsHolders(name)
Check to see if a device is 'hold' at sysfs level.

boolean

CheckReread(name)
Check to see if a block device is in use.

CheckMounted(name)
Check to see if a block device is a mountpoint.

CheckSwap(name)
Check to see if a block device is being used as swap.

InUse(name)
Returns if a disk is in use or not.

WipeDisk(name)
Wipes a block device.

PartitionDisk(name, use_sfdisk)
Partitions a disk.

CreatePVOnDisk(name)
Creates a physical volume on a block device.

CreateVG(vgname, disks)
Creates the volume group.

_ComputeSysdFreeUsed(sysdisks)

ValidateDiskList(options)
Validates or computes the disk list for create.

BootStrap()
Actual main routine.

main()
Application entry point.

Variables

[hide private]

USAGE = "\tlvmstrap diskinfo\n" "\tlvmstrap [--vg-name=NAME] [...

verbose_flag = False

SUPPORTED_TYPES = ["hd", "sd", "md", "ubd",]
Supported disk types (as prefixes)

EXCLUDED_FS = compat.UniqueFrozenset(["nfs", "nfs4", "autofs",...
Excluded filesystem types

PART_RE = re.compile("^((?:h|s|m|ub)d[a-z]{1,2})[0-9]+$")
A regular expression that matches partitions (must be kept in sync

PART_MINSIZE = 1024* 1024* 1024
Minimum partition size to be considered (1 GB)

MBR_MAX_SIZE = 2*(10** 12)

Imports: os, sys, optparse, time, errno, re, RunCmd, ReadFile, constants, cli, compat

Function Details

[hide private]

ParseOptions()

Parses the command line options.

In case of command line errors, it will show the usage and exit the program.

Returns: tuple: a tuple of (options, args), as returned by OptionParser.parse_args

IsPartitioned(disk)

Returns whether a given disk should be used partitioned or as-is.

Currently only md devices are used as is.

DeviceName(disk)

Returns the appropriate device name for a disk.

For non-partitioned devices, it returns the name as is, otherwise it returns the first partition.

ExecCommand(command)

Executes a command.

This is just a wrapper around commands.getstatusoutput, with the difference that if the command line argument -v has been given, it will print the command line and the command output on stdout.

Parameters:

command - the command line to be executed

Returns: tuple

a tuple of (status, output) where status is the exit status and output the stdout and stderr of the command together

CheckPrereq()

Check the prerequisites of this program.

It check that it runs on Linux 2.6, and that /sys is mounted and the fact that /sys/block is a directory.

CheckVGExists(vgname)

Checks to see if a volume group exists.

Parameters:

vgname - the volume group name

Returns:

a four-tuple (exists, lv_count, vg_size, vg_free), where:

exists: True if the volume exists, otherwise False; if False, all other members of the tuple are None
lv_count: The number of logical volumes in the volume group
vg_size: The total size of the volume group (in gibibytes)
vg_free: The available space in the volume group

CheckSysDev(name, devnum)

Checks consistency between /sys and /dev trees.

In /sys/block/<name>/dev and /sys/block/<name>/<part>/dev are the kernel-known device numbers. The /dev/<name> block/char devices are created by userspace and thus could differ from the kernel view. This function checks the consistency between the device number read from /sys and the actual device number in /dev.

Note that since the system could be using udev which removes and recreates the device nodes on partition table rescan, we need to do some retries here. Since we only do a stat, we can afford to do many short retries.

Parameters:

name - the device name, e.g. 'sda'
devnum - the device number, e.g. 0x803 (2051 in decimal) for sda3

Raises:

SysconfigError - in case of failure of the check

ReadDev(syspath)

Reads the device number from a sysfs path.

The device number is given in sysfs under a block device directory in a file named 'dev' which contains major:minor (in ASCII). This function reads that file and converts the major:minor pair to a dev number.

Parameters:

syspath (string) - the path to a block device dir in sysfs, e.g. /sys/block/sda

Returns:

the device number

ReadSize(syspath)

Reads the size from a sysfs path.

The size is given in sysfs under a block device directory in a file named 'size' which contains the number of sectors (in ASCII). This function reads that file and converts the number in sectors to the size in bytes.

Parameters:

syspath (string) - the path to a block device dir in sysfs, e.g. /sys/block/sda

Returns: int

the device size in bytes

ReadPV(name)

Reads physical volume information.

This function tries to see if a block device is a physical volume.

Parameters:

name (string) - the device name (e.g. sda)

Returns:

the name of the volume group to which this PV belongs, or "" if this PV is not in use, or None if this is not a PV

GetDiskList(opts)

Computes the block device list for this system.

This function examines the /sys/block tree and using information therein, computes the status of the block device.

Returns:

a list like [(name, size, dev, partitions, inuse), ...], where:

name is the block device name (e.g. sda)
size the size in bytes
dev is the device number (e.g. 8704 for hdg)
partitions is [(name, size, dev), ...] mirroring the disk list data inuse is a boolean showing the in-use status of the disk, computed as the possibility of re-reading the partition table (the meaning of the operation varies with the kernel version, but is usually accurate; a mounted disk/partition or swap-area or PV with active LVs on it is busy)

GetMountInfo()

Reads /proc/mounts and computes the mountpoint-devnum mapping.

This function reads /proc/mounts, finds the mounted filesystems (excepting a hard-coded blacklist of network and virtual filesystems) and does a stat on these mountpoints. The st_dev number of the results is memorised for later matching against the /sys/block devices.

Returns: dict: a {mountpoint: device number} dictionary

DevInfo(name, dev, mountinfo)

Computes miscellaneous information about a block device.

Parameters:

name (string) - the device name, e.g. sda

Returns:

a tuple (mpath, whatvg, fileinfo), where:

mpath is the mount path where this device is mounted or None
whatvg is the result of the ReadPV function
fileinfo is the output of file -bs on the device

ShowDiskInfo(opts)

Shows a nicely formatted block device list for this system.

This function shows the user a table with the information gathered by the other functions defined, in order to help the user make a choice about which disks should be allocated to our volume group.

CheckSysfsHolders(name)

Check to see if a device is 'hold' at sysfs level.

This is usually the case for Physical Volumes under LVM.

Returns: boolean: true if the device is available according to sysfs

CheckReread(name)

Check to see if a block device is in use.

Uses blockdev to reread the partition table of a block device (or fuser if the device is not partitionable), and thus compute the in-use status. See the discussion in GetDiskList about the meaning of 'in use'.

Returns: boolean: the in-use status of the device

CheckMounted(name)

Check to see if a block device is a mountpoint.

In recent distros/kernels, this is reported directly via fuser, but on older ones not, so we do an additional check here (manually).

WipeDisk(name)

Wipes a block device.

This function wipes a block device, by clearing and re-reading the partition table. If not successful, it writes back the old partition data, and leaves the cleanup to the user.

Parameters:

name - the device name (e.g. sda)

PartitionDisk(name, use_sfdisk)

Partitions a disk.

This function creates a single partition spanning the entire disk, by means of fdisk.

Parameters:

name - the device name, e.g. sda

CreatePVOnDisk(name)

Creates a physical volume on a block device.

This function creates a physical volume on a block device, overriding all warnings. So it can wipe existing PVs and PVs which are in a VG.

Parameters:

name - the device name, e.g. sda

CreateVG(vgname, disks)

Creates the volume group.

This function creates a volume group named `vgname` on the disks given as parameters. The physical extent size is set to 64MB.

Parameters:

disks - a list of disk names, e.g. ['sda','sdb']

ValidateDiskList(options)

Validates or computes the disk list for create.

This function either computes the available disk list (if the user gave --alldisks option), or validates the user-given disk list (by using the --disks option) such that all given disks are present and not in use.

Parameters:

options - the options returned from OptParser.parse_options

Returns:

a list of disk names, e.g. ['sda', 'sdb']

main()

Application entry point.

This is just a wrapper over BootStrap, to handle our own exceptions.

Variables Details

[hide private]

USAGE

Value:

"\tlvmstrap diskinfo\n" "\tlvmstrap [--vg-name=NAME] [--allow-removabl
e]" " { --alldisks | --disks DISKLIST } [--use-sfdisk]" " create"

EXCLUDED_FS

Excluded filesystem types

Value:

compat.UniqueFrozenset(["nfs", "nfs4", "autofs", "tmpfs", "proc", "sys
fs", "usbfs", "devpts",])