ganeti
Safe HaskellNone

Ganeti.HTools.Cluster

Description

Implementation of cluster-wide logic.

This module holds all pure cluster-logic; I/O related functionality goes into the Main module for the individual binaries.

Synopsis

Types

data AllocDetails Source #

Allocation details for an instance, specifying required number of nodes, and an optional group (name) to allocate to

Constructors

AllocDetails Int (Maybe String) 

Instances

Instances details
Show AllocDetails # 
Instance details

Defined in Ganeti.HTools.Cluster

Methods

showsPrec :: Int -> AllocDetails -> ShowS

show :: AllocDetails -> String

showList :: [AllocDetails] -> ShowS

type AllocResult = (FailStats, List, List, [Instance], [CStats]) Source #

Allocation results, as used in iterateAlloc and tieredAlloc.

type GenericAllocSolutionList a = [(Instance, GenericAllocSolution a)] Source #

Type alias for easier handling.

type AllocNodes = Either [Ndx] [(Ndx, [Ndx])] Source #

A type denoting the valid allocation mode/pairs.

For a one-node allocation, this will be a Left [Ndx], whereas for a two-node allocation, this will be a Right [(Ndx, [Ndx])]. In the latter case, the list is basically an association list, grouped by primary node and holding the potential secondary nodes in the sub-list.

data Table Source #

The complete state for the balancing solution.

Constructors

Table List List Score [Placement] 

Instances

Instances details
Show Table # 
Instance details

Defined in Ganeti.HTools.Cluster

Methods

showsPrec :: Int -> Table -> ShowS

show :: Table -> String

showList :: [Table] -> ShowS

data CStats Source #

Cluster statistics data type.

Constructors

CStats 

Fields

  • csFmem :: Integer

    Cluster free mem

  • csFdsk :: Integer

    Cluster free disk

  • csFspn :: Integer

    Cluster free spindles

  • csAmem :: Integer

    Cluster allocatable mem

  • csAdsk :: Integer

    Cluster allocatable disk

  • csAcpu :: Integer

    Cluster allocatable cpus

  • csMmem :: Integer

    Max node allocatable mem

  • csMdsk :: Integer

    Max node allocatable disk

  • csMcpu :: Integer

    Max node allocatable cpu

  • csImem :: Integer

    Instance used mem

  • csIdsk :: Integer

    Instance used disk

  • csIspn :: Integer

    Instance used spindles

  • csIcpu :: Integer

    Instance used cpu

  • csTmem :: Double

    Cluster total mem

  • csTdsk :: Double

    Cluster total disk

  • csTspn :: Double

    Cluster total spindles

  • csTcpu :: Double

    Cluster total cpus

  • csVcpu :: Integer

    Cluster total virtual cpus

  • csNcpu :: Double

    Equivalent to csIcpu but in terms of physical CPUs, i.e. normalised used phys CPUs

  • csXmem :: Integer

    Unnacounted for mem

  • csNmem :: Integer

    Node own memory

  • csScore :: Score

    The cluster score

  • csNinst :: Int

    The total number of instances

Instances

Instances details
Show CStats # 
Instance details

Defined in Ganeti.HTools.Cluster

Methods

showsPrec :: Int -> CStats -> ShowS

show :: CStats -> String

showList :: [CStats] -> ShowS

type AllocMethod Source #

Arguments

 = List

Node list

-> List

Instance list

-> Maybe Int

Optional allocation limit

-> Instance

Instance spec for allocation

-> AllocNodes

Which nodes we should allocate on

-> [Instance]

Allocated instances

-> [CStats]

Running cluster stats

-> Result AllocResult

Allocation result

A simple type for allocation functions.

Utility functions

verifyN1 :: [Node] -> [Node] Source #

Verifies the N+1 status and return the affected nodes.

computeBadItems :: List -> List -> ([Node], [Instance]) Source #

Computes the pair of bad nodes and instances.

The bad node list is computed via a simple verifyN1 check, and the bad instance list is the list of primary and secondary instances of those nodes.

emptyCStats :: CStats Source #

Zero-initializer for the CStats type.

updateCStats :: CStats -> Node -> CStats Source #

Update stats with data from a new node.

totalResources :: List -> CStats Source #

Compute the total free disk and memory in the cluster.

computeAllocationDelta :: CStats -> CStats -> AllocStats Source #

Compute the delta between two cluster state.

This is used when doing allocations, to understand better the available cluster resources. The return value is a triple of the current used values, the delta that was still allocated, and what was left unallocated.

getOnline :: List -> [Node] Source #

Compute online nodes from a List.

Balancing functions

compareTables :: Table -> Table -> Table Source #

Compute best table. Note that the ordering of the arguments is important.

checkSingleStep Source #

Arguments

:: Bool

Whether to unconditionally ignore soft errors

-> Table

The original table

-> Instance

The instance to move

-> Table

The current best table

-> IMove

The move to apply

-> Table

The final best table

Tries to perform an instance move and returns the best table between the original one and the new one.

possibleMoves Source #

Arguments

:: MirrorType

The mirroring type of the instance

-> Bool

Whether the secondary node is a valid new node

-> Bool

Whether we can change the primary node

-> Bool

Whether we alowed to move disks

-> (Bool, Bool)

Whether migration is restricted and whether the instance primary is offline

-> Ndx

Target node candidate

-> [IMove]

List of valid result moves

Given the status of the current secondary as a valid new node and the current candidate target node, generate the possible moves for a instance.

checkInstanceMove Source #

Arguments

:: AlgorithmOptions

Algorithmic options for balancing

-> [Ndx]

Allowed target node indices

-> Table

Original table

-> Instance

Instance to move

-> Table

Best new table for this instance

Compute the best move for a given instance.

checkMove Source #

Arguments

:: AlgorithmOptions

Algorithmic options for balancing

-> [Ndx]

Allowed target node indices

-> Table

The current solution

-> [Instance]

List of instances still to move

-> Table

The new solution

Compute the best next move.

doNextBalance Source #

Arguments

:: Table

The starting table

-> Int

Remaining length

-> Score

Score at which to stop

-> Bool

The resulting table and commands

Check if we are allowed to go deeper in the balancing.

tryBalance Source #

Arguments

:: AlgorithmOptions

Algorithmic options for balancing

-> Table

The starting table

-> Maybe Table

The resulting table and commands

Run a balance move.

Allocation functions

genAllocNodes Source #

Arguments

:: AlgorithmOptions

algorithmic options to honor

-> List

Group list

-> List

The node map

-> Int

The number of nodes required

-> Bool

Whether to drop or not unallocable nodes

-> Result AllocNodes

The (monadic) result

Generate the valid node allocation singles or pairs for a new instance.

tryAlloc Source #

Arguments

:: MonadFail m 
=> AlgorithmOptions 
-> List

The node list

-> List

The instance list

-> Instance

The instance to allocate

-> AllocNodes

The allocation targets

-> m AllocSolution

Possible solution list

Try to allocate an instance on the cluster.

filterMGResults :: [(Group, Result (GenericAllocSolution a))] -> [(Group, GenericAllocSolution a)] Source #

From a list of possibly bad and possibly empty solutions, filter only the groups with a valid result. Note that the result will be reversed compared to the original list.

sortMGResults :: Ord a => [(Group, GenericAllocSolution a)] -> [(Group, GenericAllocSolution a)] Source #

Sort multigroup results based on policy and score.

hasRequiredNetworks :: Group -> Instance -> Bool Source #

Determines if a group is connected to the networks required by the | instance.

filterValidGroups :: [(Group, (List, List))] -> Instance -> ([(Group, (List, List))], [String]) Source #

Removes node groups which can't accommodate the instance

findAllocation Source #

Arguments

:: AlgorithmOptions 
-> List

The group list

-> List

The node list

-> List

The instance list

-> Gdx

The group to allocate to

-> Instance

The instance to allocate

-> Int

Required number of nodes

-> Result (AllocSolution, [String]) 

Finds an allocation solution for an instance on a group

findBestAllocGroup Source #

Arguments

:: AlgorithmOptions 
-> List

The group list

-> List

The node list

-> List

The instance list

-> Maybe [Gdx]

The allowed groups

-> Instance

The instance to allocate

-> Int

Required number of nodes

-> Result (Group, AllocSolution, [String]) 

Finds the best group for an instance on a multi-group cluster.

Only solutions in preferred and last_resort groups will be accepted as valid, and additionally if the allowed groups parameter is not null then allocation will only be run for those group indices.

tryMGAlloc Source #

Arguments

:: AlgorithmOptions 
-> List

The group list

-> List

The node list

-> List

The instance list

-> Instance

The instance to allocate

-> Int

Required number of nodes

-> Result AllocSolution

Possible solution list

Try to allocate an instance on a multi-group cluster.

tryGroupAlloc Source #

Arguments

:: AlgorithmOptions 
-> List

The group list

-> List

The node list

-> List

The instance list

-> String

The allocation group (name)

-> Instance

The instance to allocate

-> Int

Required number of nodes

-> Result AllocSolution

Solution

Try to allocate an instance to a group.

allocList Source #

Arguments

:: AlgorithmOptions 
-> List

The group list

-> List

The node list

-> List

The instance list

-> [(Instance, AllocDetails)]

The instance to allocate

-> AllocSolutionList

Possible solution list

-> Result (List, List, AllocSolutionList)

The final solution list

Try to allocate a list of instances on a multi-group cluster.

tryChangeGroup Source #

Arguments

:: AlgorithmOptions 
-> List

The cluster groups

-> List

The node list (cluster-wide)

-> List

Instance list (cluster-wide)

-> [Gdx]

Target groups; if empty, any groups not being evacuated

-> [Idx]

List of instance (indices) to be evacuated

-> Result (List, List, EvacSolution) 

Change-group IAllocator mode main function.

This is very similar to tryNodeEvac, the only difference is that we don't choose as target group the current instance group, but instead:

  1. at the start of the function, we compute which are the target groups; either no groups were passed in, in which case we choose all groups out of which we don't evacuate instance, or there were some groups passed, in which case we use those
  2. for each instance, we use findBestAllocGroup to choose the best group to hold the instance, and then we do what tryNodeEvac does, except for this group instead of the current instance group.

Note that the correct behaviour of this function relies on the function nodeEvacInstance to be able to do correctly both intra-group and inter-group moves when passed the ChangeAll mode.

iterateAllocSmallStep :: AlgorithmOptions -> AllocMethod Source #

Standard-sized allocation method.

This places instances of the same size on the cluster until we're out of space. The result will be a list of identically-sized instances.

guessBigstepSize :: List -> Instance -> Int Source #

Guess a number of machines worth trying to put on the cluster in one step. The goal is to guess a number close to the actual capacity of the cluster but preferrably not bigger, unless it is quite small (as we don't want to do big steps smaller than 20).

iterateAlloc' :: Bool -> AlgorithmOptions -> AllocMethod Source #

A speed-up version of iterateAllocSmallStep.

This function returns precisely the same result as iterateAllocSmallStep. However the computation is speed up by the following heuristic: allocate a group of instances iteratively without considering global N+1 redundancy; if the result of this is globally N+1 redundant, then everything was OK inbetween and we can continue from there. Only if that fails, do a step-by-step iterative allocation. In order to further speed up the computation while keeping it robust, we first try (if the first argument is True) a number of steps guessed from the node capacity, then, if that failed, a fixed step size and only as last restort step-by-step iterative allocation.

sufficesShrinking :: (Instance -> AllocSolution) -> Instance -> FailMode -> Maybe Instance Source #

Predicate whether shrinking a single resource can lead to a valid allocation.

tieredAlloc :: AlgorithmOptions -> AllocMethod Source #

Tiered allocation method.

This places instances on the cluster, and decreases the spec until we can allocate again. The result will be a list of decreasing instance specs.

Formatting functions

computeMoves Source #

Arguments

:: Instance

The instance to be moved

-> String

The instance name

-> IMove

The move being performed

-> String

New primary

-> String

New secondary

-> (String, [String])

Tuple of moves and commands list; moves is containing either f for failover or r:name for replace secondary, while the command list holds gnt-instance commands (without that prefix), e.g "failover instance1"

Given the original and final nodes, computes the relocation description.

printSolutionLine Source #

Arguments

:: List

The node list

-> List

The instance list

-> Int

Maximum node name length

-> Int

Maximum instance name length

-> Placement

The current placement

-> Int

The index of the placement in the solution

-> (String, [String]) 

Converts a placement to string format.

involvedNodes Source #

Arguments

:: List

Instance list, used for retrieving the instance from its index; note that this must be the original instance list, so that we can retrieve the old nodes

-> Placement

The placement we're investigating, containing the new nodes and instance index

-> [Ndx]

Resulting list of node indices

Return the instance and involved nodes in an instance move.

Note that the output list length can vary, and is not required nor guaranteed to be of any specific length.

getMoves :: (Table, Table) -> [MoveJob] Source #

From two adjacent cluster tables get the list of moves that transitions from to the other

mergeJobs :: ([JobSet], [Ndx]) -> MoveJob -> ([JobSet], [Ndx]) Source #

Inner function for splitJobs, that either appends the next job to the current jobset, or starts a new jobset.

splitJobs :: [MoveJob] -> [JobSet] Source #

Break a list of moves into independent groups. Note that this will reverse the order of jobs.

formatJob :: Int -> Int -> (Int, MoveJob) -> [String] Source #

Given a list of commands, prefix them with gnt-instance and also beautify the display a little.

formatCmds :: [JobSet] -> String Source #

Given a list of commands, prefix them with gnt-instance and also beautify the display a little.

printNodes :: List -> [String] -> String Source #

Print the node list.

printInsts :: List -> List -> String Source #

Print the instance list.

Node group functions

instanceGroup :: List -> Instance -> Result Gdx Source #

Computes the group of an instance.

findSplitInstances :: List -> List -> [Instance] Source #

Compute the list of badly allocated instances (split across node groups).