Package ganeti :: Package utils :: Module text

[frames] | no frames]

Module text

source code

Utility functions for manipulating or working with text.

Classes

[hide private]

ShellWriter
Helper class to write scripts with indentation.

LineSplitter
Splits data chunks into lines separated by newline.

Functions

[hide private]

None or str

MatchNameComponent(key, name_list, case_sensitive=True)
Try to match a name against a list.

source code

_DnsNameGlobHelper(match)
Helper function for DnsNameGlobPattern.

source code

string

DnsNameGlobPattern(pattern)
Generates regular expression from DNS name globbing pattern.

source code

str

FormatUnit(value, units)
Formats an incoming number of MiB with the appropriate unit.

source code

ParseUnit(input_string)
Tries to extract number and scale from the given string.

source code

str

ShellQuote(value)
Quotes shell argument according to POSIX.

source code

str

ShellQuoteArgs(args)
Quotes a list of shell arguments.

source code

ShellCombineCommands(cmdlist)
Out of a list of shell comands construct a single one.

source code

str

GenerateSecret(numbytes=20)
Generates a random secret.

source code

_MakeMacAddrRegexp(octets)
Builds a regular expression for verifying MAC addresses.

source code

_MacAddressCheck(check_re, mac, msg)
Checks a MAC address using a regular expression.

source code

string

NormalizeAndValidateMac(mac)
Normalizes and check if a MAC address is valid and contains six octets.

source code

string

NormalizeAndValidateThreeOctetMacPrefix(mac)
Normalizes a potential MAC address prefix (three octets).

source code

str

SafeEncode(text)
Return a 'safe' version of a source string.

source code

string

UnescapeAndSplit(text, sep=",")
Split and unescape a string based on a given separator.

source code

string

EscapeAndJoin(slist, sep=",")
Encode a list in a way parsable by UnescapeAndSplit.

source code

CommaJoin(names)
Nicely join a set of identifiers.

source code

FormatTime(val, usecs=None)
Formats a time value.

source code

string

FormatSeconds(secs)
Formats seconds for easier reading.

source code

boolean

IsValidShellParam(word)
Verifies is the given word is safe from the shell's p.o.v.

source code

str

BuildShellCmd(template, *args)
Build a safe shell command line from the given arguments.

source code

string

FormatOrdinal(value)
Formats a number as an ordinal in the English language.

source code

string

Truncate(text, length)
Truncate string and add ellipsis if needed.

source code

list

FilterEmptyLinesAndComments(text)
Filters empty lines and comments from a line-based string.

source code

list of string

FormatKeyValue(data)
Formats a dictionary as "key=value" parameters.

source code

Variables

[hide private]

_PARSEUNIT_REGEX = re.compile(r"^([.\d]+)\s*([a-zA-Z]+)?$")
Unit checker regexp

_SHELL_UNQUOTED_RE = re.compile("^[-.,=:/_+@A-Za-z0-9]+$")
Characters which don't need to be quoted for shell commands

_SHELLPARAM_REGEX = re.compile(r"^[-a-zA-Z0-9._+/:%@]+$")
Shell param checker regexp

_ASCII_ELLIPSIS = "..."
ASCII equivalent of unicode character 'HORIZONTAL ELLIPSIS' (U+2026)

_MAC_ADDR_OCTET_RE = r"[0-9a-f]{2}"
MAC address octet

_MAC_CHECK_RE = _MakeMacAddrRegexp(6)
Regular expression for full MAC address

_MAC_PREFIX_CHECK_RE = _MakeMacAddrRegexp(3)
Regular expression for half a MAC address

Imports: re, os, time, collections, errors

Function Details

[hide private]

MatchNameComponent(key, name_list, case_sensitive=True)

source code

Try to match a name against a list.

This function will try to match a name like test1 against a list like ['test1.example.com', 'test2.example.com', ...]. Against this list, 'test1' as well as 'test1.example' will match, but not 'test1.ex'. A multiple match will be considered as no match at all (e.g. 'test1' against ['test1.example.com', 'test1.example.org']), except when the key fully matches an entry (e.g. 'test1' against ['test1', 'test1.example.com']).

Parameters:

key (str) - the name to be searched
name_list (list) - the list of strings against which to search the key
case_sensitive (boolean) - whether to provide a case-sensitive match

Returns: None or str

None if there is no match or if there are multiple matches, otherwise the element from the list which matches

_DnsNameGlobHelper(match)

source code

Helper function for DnsNameGlobPattern.

Returns regular expression pattern for parts of the pattern.

DnsNameGlobPattern(pattern)

source code

Generates regular expression from DNS name globbing pattern.

A DNS name globbing pattern (e.g. *.site) is converted to a regular expression. Escape sequences or ranges (e.g. [a-z]) are not supported.

Matching always starts at the leftmost part. An asterisk (*) matches all characters except the dot (.) separating DNS name parts. A question mark (?) matches a single character except the dot (.).

Parameters:

pattern (string) - DNS name globbing pattern

Returns: string

Regular expression

FormatUnit(value, units)

source code

Formats an incoming number of MiB with the appropriate unit.

Parameters:

value (int) - integer representing the value in MiB (1048576)
units (char) - the type of formatting we should do:
- 'h' for automatic scaling
- 'm' for MiBs
- 'g' for GiBs
- 't' for TiBs

Returns: str

the formatted value (with suffix)

ParseUnit(input_string)

source code

Tries to extract number and scale from the given string.

Input must be in the format NUMBER+ [DOT NUMBER+] SPACE* [UNIT]. If no unit is specified, it defaults to MiB. Return value is always an int in MiB.

ShellQuote(value)

source code

Quotes shell argument according to POSIX.

Parameters:

value (str) - the argument to be quoted

Returns: str

the quoted value

ShellQuoteArgs(args)

source code

Quotes a list of shell arguments.

Parameters:

args (list) - list of arguments to be quoted

Returns: str

the quoted arguments concatenated with spaces

GenerateSecret(numbytes=20)

source code

Generates a random secret.

This will generate a pseudo-random secret returning an hex string (so that it can be used where an ASCII string is needed).

Parameters:

numbytes - the number of bytes which will be represented by the returned string (defaulting to 20, the length of a SHA1 hash)

Returns: str

an hex representation of the pseudo-random sequence

_MakeMacAddrRegexp(octets)

source code

Builds a regular expression for verifying MAC addresses.

Parameters:

octets (integer) - How many octets to expect (1-6)

Returns:

Compiled regular expression

_MacAddressCheck(check_re, mac, msg)

source code

Checks a MAC address using a regular expression.

Parameters:

check_re - Compiled regular expression as returned by re.compile
mac (string) - MAC address to be validated
msg (string) - Error message (%s will be replaced with MAC address)

NormalizeAndValidateMac(mac)

source code

Normalizes and check if a MAC address is valid and contains six octets.

Checks whether the supplied MAC address is formally correct. Accepts colon-separated format only. Normalize it to all lower case.

Parameters:

mac (string) - MAC address to be validated

Returns: string

Normalized and validated MAC address

Raises:

errors.OpPrereqError - If the MAC address isn't valid

NormalizeAndValidateThreeOctetMacPrefix(mac)

source code

Normalizes a potential MAC address prefix (three octets).

Checks whether the supplied string is a valid MAC address prefix consisting of three colon-separated octets. The result is normalized to all lower case.

Parameters:

mac (string) - Prefix to be validated

Returns: string

Normalized and validated prefix

Raises:

errors.OpPrereqError - If the MAC address prefix isn't valid

SafeEncode(text)

source code

Return a 'safe' version of a source string.

This function mangles the input string and returns a version that should be safe to display/encode as ASCII. To this end, we first convert it to ASCII using the 'backslashreplace' encoding which should get rid of any non-ASCII chars, and then we process it through a loop copied from the string repr sources in the python; we don't use string_escape anymore since that escape single quotes and backslashes too, and that is too much; and that escaping is not stable, i.e. string_escape(string_escape(x)) != string_escape(x).

Parameters:

text (str or unicode) - input data

Returns: str

a safe version of text

UnescapeAndSplit(text, sep=",")

source code

Split and unescape a string based on a given separator.

This function splits a string based on a separator where the separator itself can be escape in order to be an element of the elements. The escaping rules are (assuming coma being the separator):

a plain , separates the elements
a sequence \\\\, (double backslash plus comma) is handled as a backslash plus a separator comma
a sequence \, (backslash plus comma) is handled as a non-separator comma

Parameters:

text (string) - the string to split
text (string) - the separator
sep (string)

Returns: string

a list of strings

EscapeAndJoin(slist, sep=",")

source code

Encode a list in a way parsable by UnescapeAndSplit.

Parameters:

slist (list of strings) - the strings to be encoded

Returns: string

the encoding of the list oas a string

CommaJoin(names)

source code

Nicely join a set of identifiers.

Parameters:

names - set, list or tuple

Returns:

a string with the formatted results

FormatTime(val, usecs=None)

source code

Formats a time value.

Parameters:

val (float or None) - Timestamp as returned by time.time() (seconds since Epoch, 1970-01-01 00:00:00 UTC)

Returns:

a string value or N/A if we don't have a valid timestamp

FormatSeconds(secs)

source code

Formats seconds for easier reading.

Parameters:

secs (number) - Number of seconds

Returns: string

Formatted seconds (e.g. "2d 9h 19m 49s")

IsValidShellParam(word)

source code

Verifies is the given word is safe from the shell's p.o.v.

This means that we can pass this to a command via the shell and be sure that it doesn't alter the command line and is passed as such to the actual command.

Note that we are overly restrictive here, in order to be on the safe side.

Parameters:

word (str) - the word to check

Returns: boolean

True if the word is 'safe'

BuildShellCmd(template, *args)

source code

Build a safe shell command line from the given arguments.

This function will check all arguments in the args list so that they are valid shell parameters (i.e. they don't contain shell metacharacters). If everything is ok, it will return the result of template % args.

Parameters:

template (str) - the string holding the template for the string formatting

Returns: str

the expanded command line

FormatOrdinal(value)

source code

Formats a number as an ordinal in the English language.

E.g. the number 1 becomes "1st", 22 becomes "22nd".

Parameters:

value (integer) - Number

Returns: string

Truncate(text, length)

source code

Truncate string and add ellipsis if needed.

Parameters:

text (string) - Text
length (integer) - Desired length

Returns: string

Truncated text

FilterEmptyLinesAndComments(text)

source code

Filters empty lines and comments from a line-based string.

Whitespace is also removed from the beginning and end of all lines.

Parameters:

text (string) - Input string

Returns: list

FormatKeyValue(data)

source code

Formats a dictionary as "key=value" parameters.

The keys are sorted to have a stable order.

Parameters:

data (dict)

Returns: list of string