Node roles

Each node can be in one of the following states:

master

Only one node per cluster can be in this role, and this node is the one holding the authoritative copy of the cluster configuration and the one that can actually execute commands on the cluster and modify the cluster state. See more details under Cluster configuration.

master_candidate

The node receives the full cluster configuration (configuration file and jobs) and can become a master via the gnt-cluster master-failover command. Nodes that are not in this state cannot transition into the master role due to missing state.

regular

This the normal state of a node.

drained

Nodes in this state are functioning normally but cannot receive new instances, because the intention is to set them to offline or remove them from the cluster.

offline

These nodes are still recorded in the Ganeti configuration, but except for the master daemon startup voting procedure, they are not actually contacted by the master. This state was added in order to allow broken machines (that are being repaired) to remain in the cluster but without creating problems.

Node flags

Nodes have two flags which govern which roles they can take:

master_capable

The node can become a master candidate, and furthermore the master node. When this flag is disabled, the node cannot become a candidate; this can be useful for special networking cases, or less reliable hardware.

vm_capable

The node can host instances. When enabled (the default state), the node will participate in instance allocation, capacity calculation, etc. When disabled, the node will be skipped in many cluster checks and operations.

Node Parameters

These parameters are node specific and can be preseeded on node-group and cluster level.

Currently we support the following node parameters:

oob_program

Path to an executable used as the out-of-band helper as described in the Ganeti Node OOB Management Framework design document.

Cluster configuration

The master node keeps and is responsible for the cluster configuration. On the filesystem, this is stored under the /usr/local/var/ganeti/lib directory, and if the master daemon is stopped it can be backed up normally.

The master daemon will replicate the configuration database called config.data and the job files to all the nodes in the master candidate role. It will also distribute a copy of some configuration values via the ssconf files, which are stored in the same directory and start with a ssconf_ prefix, to all nodes.

Jobs

All cluster modification are done via jobs. A job consists of one or more opcodes, and the list of opcodes is processed serially. If an opcode fails, the entire job is failed and later opcodes are no longer processed. A job can be in one of the following states:

queued

The job has been submitted but not yet processed by the master daemon.

waiting

The job is waiting for for locks before the first of its opcodes.

canceling

The job is waiting for locks, but is has been marked for cancellation. It will not transition to running, but to canceled.

running

The job is currently being executed.

canceled

The job has been canceled before starting execution.

success

The job has finished successfully.

error

The job has failed during runtime, or the master daemon has been stopped during the job execution.

Options

Many Ganeti commands provide the following options. The availability for a certain command can be checked by calling the command using the --help option.

gnt-... command [--dry-run] [--priority {low | normal | high}]

The --dry-run option can be used to check whether an operation would succeed.

The option --priority sets the priority for opcodes submitted by the command.

Key-value parameters

Multiple options take parameters that are of the form key=value,key=value,... or category:key=value,.... Examples are the hypervisor parameters, backend parameters, etc. For these, it's possible to use values that contain commas by escaping with via a backslash (which needs two if not single-quoted, due to shell behaviour):

# gnt-instance modify -H kernel_path=an\\,example instance1
# gnt-instance modify -H kernel_path='an\,example' instance1

Query filters

Most commands listing resources (e.g. instances or nodes) support filtering. The filter language is similar to Python expressions with some elements from Perl. The language is not generic. Each condition must consist of a field name and a value (except for boolean checks), a field can not be compared to another field. Keywords are case-sensitive.

Syntax in pseudo-BNF:

<quoted-string> ::= /* String quoted with single or double quotes,
                       backslash for escaping */

<integer> ::= /* Number in base-10 positional notation */

<re> ::= /* Regular expression */

/*
  Modifier "i": Case-insensitive matching, see
  http://docs.python.org/library/re#re.IGNORECASE

  Modifier "s": Make the "." special character match any character,
  including newline, see http://docs.python.org/library/re#re.DOTALL
*/
<re-modifiers> ::= /* empty */ | i | s

<value> ::= <quoted-string> | <integer>

<condition> ::=
  { /* Value comparison */
    <field> { == | != } <value>

    /* Collection membership */
    | <value> [ not ] in <field>

    /* Regular expressions (recognized delimiters
       are "/", "#", "^", and "|"; backslash for escaping)
    */
    | <field> { =~ | !~ } m/<re>/<re-modifiers>

    /* Globbing */
    | <field> { =* | !* } <quoted-string>

    /* Boolean */
    | <field>
  }

<filter> ::=
  { [ not ] <condition> | ( <filter> ) }
  [ { and | or } <filter> ]

Operators:

==

Equality

!=

Inequality

=~

Pattern match using regular expression

!~

Logically negated from =~

=*

Globbing, see glob(7), though only * and ? are supported

!*

Logically negated from =*

in, not in

Collection membership and negation

As a shortcut globbing patterns can be specified as names, e.g. gnt-instance list '*.site1' '*.site2'.