ganeti - cluster-based virtualization management
# gnt-cluster init cluster1.example.com
# gnt-node add node2.example.com
# gnt-instance add -n node2.example.com \
> -o debootstrap --disk 0:size=30g \
> -t plain instance1.example.com
The Ganeti software manages physical nodes and virtual instances of a cluster based on a virtualization software. The current version (2.3) supports Xen 3.x and KVM (72 or above) as hypervisors, and LXC as an experimental hypervisor.
First you must install the software on all the cluster nodes, either from sources or (if available) from a package. The next step is to create the initial cluster configuration, using gnt-cluster init.
Then you can add other nodes, or start creating instances.
In Ganeti 2.0, the architecture of the cluster is a little more complicated than in 1.2. The cluster is coordinated by a master daemon (ganeti-masterd(8)), running on the master node. Each node runs (as before) a node daemon, and the master has the RAPI daemon running too.
Each node can be in one of the following states:
Nodes have two flags which govern which roles they can take:
The ndparams refer to node parameters. These can be set as defaults on cluster and node group levels, but they take effect for nodes only.
Currently we support the following node parameters:
Using --hypervisor-state you can set hypervisor specific states as pointed out in Ganeti Resource Model <design-resource-model.rst>.
The format is: hypervisor:option=value.
Currently we support the following hypervisor state values:
Note that currently this option is unused by Ganeti; values will be recorded but will not influence the Ganeti operation.
Using --disk-state you can set disk specific states as pointed out in Ganeti Resource Model <design-resource-model.rst>.
The format is: storage_type/identifier:option=value. Where we currently just support lvm as storage type. The identifier in this case is the LVM volume group. By default this is xenvg.
Currently we support the following hypervisor state values:
Note that currently this option is unused by Ganeti; values will be recorded but will not influence the Ganeti operation.
The master node keeps and is responsible for the cluster configuration. On the filesystem, this is stored under the /var/ganeti/lib directory, and if the master daemon is stopped it can be backed up normally.
The master daemon will replicate the configuration database called config.data and the job files to all the nodes in the master candidate role. It will also distribute a copy of some configuration values via the ssconf files, which are stored in the same directory and start with a ssconf_ prefix, to all nodes.
All cluster modification are done via jobs. A job consists of one or more opcodes, and the list of opcodes is processed serially. If an opcode fails, the entire job is failed and later opcodes are no longer processed. A job can be in one of the following states:
Many Ganeti commands provide the following options. The availability for a certain command can be checked by calling the command using the --help option.
The --dry-run option can be used to check whether an operation would succeed.
The option --priority sets the priority for opcodes submitted by the command.
The --submit option is used to send the job to the master daemon but not wait for its completion. The job ID will be shown so that it can be examined using gnt-job info.
The --print-job-id option makes the command print the job id as first line on stdout, so that it is easy to parse by other programs.
For certain commands you can use environment variables to provide default command line arguments. Just assign the arguments as a string to the corresponding environment variable. The format of that variable name is binary**_*command*. **binary is the name of the gnt-* script all upper case and dashes replaced by underscores, and command is the command invoked on that script.
Currently supported commands are gnt-node list, gnt-group list and gnt-instance list. So you can configure default command line flags by setting GNT_NODE_LIST, GNT_GROUP_LIST and GNT_INSTANCE_LIST.
If the variable FORCE_LUXI_SOCKET is set, it will override the socket used for LUXI connections by command-line tools (gnt-*). This is useful mostly for debugging, and some operations won’t work at all if, for example, you point this variable to the confd-supplied query socket and try to submit a job.
If the variable is set to the value master, it will connect to the correct path for the master daemon (even if, for example, split queries are enabled and this is a query operation). If set to query, it will always (try to) connect to the query socket, even if split queries are disabled. Otherwise, the value is taken to represent a filesystem path to the socket to use.
Multiple ganeti commands use the same framework for tabular listing of resources (e.g. gnt-instance list, gnt-node list, gnt-group list, gnt-debug locks, etc.). For these commands, special states are denoted via a special symbol (in terse mode) or a string (in verbose mode):
Multiple options take parameters that are of the form key=value,key=value,... or category:key=value,.... Examples are the hypervisor parameters, backend parameters, etc. For these, it’s possible to use values that contain commas by escaping with via a backslash (which needs two if not single-quoted, due to shell behaviour):
# gnt-instance modify -H kernel_path=an\\,example instance1
# gnt-instance modify -H kernel_path='an\,example' instance1
Additionally, the following non-string parameters can be passed. To pass the boolean value True, only mention the key (leaving out the equality sign and any value). To pass the boolean value False, again only mention the key, but prefix it with no_. To pass the special None value, again only mention the key, but prefix it with a single - sign.
Most commands listing resources (e.g. instances or nodes) support filtering. The filter language is similar to Python expressions with some elements from Perl. The language is not generic. Each condition must consist of a field name and a value (except for boolean checks), a field can not be compared to another field. Keywords are case-sensitive.
Examples (see below for syntax details):
List webservers:
gnt-instance list --filter 'name =* "web*.example.com"'
List instances with three or six virtual CPUs and whose primary nodes reside in groups starting with the string “rack”:
gnt-instance list --filter
'(be/vcpus == 3 or be/vcpus == 6) and pnode.group =~ m/^rack/'
Nodes hosting primary instances:
gnt-node list --filter 'pinst_cnt != 0'
Nodes which aren’t master candidates:
gnt-node list --filter 'not master_candidate'
Short version for globbing patterns:
gnt-instance list '*.site1' '*.site2'
Syntax in pseudo-BNF:
<quoted-string> ::= /* String quoted with single or double quotes,
backslash for escaping */
<integer> ::= /* Number in base-10 positional notation */
<re> ::= /* Regular expression */
/*
Modifier "i": Case-insensitive matching, see
http://docs.python.org/library/re#re.IGNORECASE
Modifier "s": Make the "." special character match any character,
including newline, see http://docs.python.org/library/re#re.DOTALL
*/
<re-modifiers> ::= /* empty */ | i | s
<value> ::= <quoted-string> | <integer>
<condition> ::=
{ /* Value comparison */
<field> { == | != | < | <= | >= | > } <value>
/* Collection membership */
| <value> [ not ] in <field>
/* Regular expressions (recognized delimiters
are "/", "#", "^", and "|"; backslash for escaping)
*/
| <field> { =~ | !~ } m/<re>/<re-modifiers>
/* Globbing */
| <field> { =* | !* } <quoted-string>
/* Boolean */
| <field>
}
<filter> ::=
{ [ not ] <condition> | ( <filter> ) }
[ { and | or } <filter> ]
Operators:
All Ganeti daemons re-open the log file(s) when sent a SIGHUP signal. logrotate(8) can be used to rotate Ganeti’s log files.