Package ganeti :: Package utils :: Module text
[hide private]
[frames] | no frames]

Module text

source code

Utility functions for manipulating or working with text.

Classes [hide private]
  ShellWriter
Helper class to write scripts with indentation.
  LineSplitter
Splits data chunks into lines separated by newline.
Functions [hide private]
None or str
MatchNameComponent(key, name_list, case_sensitive=True)
Try to match a name against a list.
source code
 
_DnsNameGlobHelper(match)
Helper function for DnsNameGlobPattern.
source code
string
DnsNameGlobPattern(pattern)
Generates regular expression from DNS name globbing pattern.
source code
str
FormatUnit(value, units, roman=False)
Formats an incoming number of MiB with the appropriate unit.
source code
 
ParseUnit(input_string)
Tries to extract number and scale from the given string.
source code
str
ShellQuote(value)
Quotes shell argument according to POSIX.
source code
str
ShellQuoteArgs(args)
Quotes a list of shell arguments.
source code
 
ShellCombineCommands(cmdlist)
Out of a list of shell comands construct a single one.
source code
str
GenerateSecret(numbytes=20)
Generates a random secret.
source code
 
_MakeMacAddrRegexp(octets)
Builds a regular expression for verifying MAC addresses.
source code
 
_MacAddressCheck(check_re, mac, msg)
Checks a MAC address using a regular expression.
source code
string
NormalizeAndValidateMac(mac)
Normalizes and check if a MAC address is valid and contains six octets.
source code
string
NormalizeAndValidateThreeOctetMacPrefix(mac)
Normalizes a potential MAC address prefix (three octets).
source code
str
SafeEncode(text)
Return a 'safe' version of a source string.
source code
string
UnescapeAndSplit(text, sep=",")
Split and unescape a string based on a given separator.
source code
string
EscapeAndJoin(slist, sep=",")
Encode a list in a way parsable by UnescapeAndSplit.
source code
 
CommaJoin(names)
Nicely join a set of identifiers.
source code
 
FormatTime(val, usecs=None)
Formats a time value.
source code
string
FormatSeconds(secs)
Formats seconds for easier reading.
source code
boolean
IsValidShellParam(word)
Verifies is the given word is safe from the shell's p.o.v.
source code
str
BuildShellCmd(template, *args)
Build a safe shell command line from the given arguments.
source code
string
FormatOrdinal(value)
Formats a number as an ordinal in the English language.
source code
string
Truncate(text, length)
Truncate string and add ellipsis if needed.
source code
list
FilterEmptyLinesAndComments(text)
Filters empty lines and comments from a line-based string.
source code
list of string
FormatKeyValue(data)
Formats a dictionary as "key=value" parameters.
source code
Variables [hide private]
  _PARSEUNIT_REGEX = re.compile(r"^([.\d]+)\s*([a-zA-Z]+)?$")
Unit checker regexp
  _SHELL_UNQUOTED_RE = re.compile("^[-.,=:/_+@A-Za-z0-9]+$")
Characters which don't need to be quoted for shell commands
  _SHELLPARAM_REGEX = re.compile(r"^[-a-zA-Z0-9._+/:%@]+$")
Shell param checker regexp
  _ASCII_ELLIPSIS = "..."
ASCII equivalent of unicode character 'HORIZONTAL ELLIPSIS' (U+2026)
  _MAC_ADDR_OCTET_RE = r"[0-9a-f]{2}"
MAC address octet
  _MAC_CHECK_RE = _MakeMacAddrRegexp(6)
Regular expression for full MAC address
  _MAC_PREFIX_CHECK_RE = _MakeMacAddrRegexp(3)
Regular expression for half a MAC address

Imports: re, os, time, collections, errors, compat


Function Details [hide private]

MatchNameComponent(key, name_list, case_sensitive=True)

source code 

Try to match a name against a list.

This function will try to match a name like test1 against a list like ['test1.example.com', 'test2.example.com', ...]. Against this list, 'test1' as well as 'test1.example' will match, but not 'test1.ex'. A multiple match will be considered as no match at all (e.g. 'test1' against ['test1.example.com', 'test1.example.org']), except when the key fully matches an entry (e.g. 'test1' against ['test1', 'test1.example.com']).

Parameters:
  • key (str) - the name to be searched
  • name_list (list) - the list of strings against which to search the key
  • case_sensitive (boolean) - whether to provide a case-sensitive match
Returns: None or str
None if there is no match or if there are multiple matches, otherwise the element from the list which matches

_DnsNameGlobHelper(match)

source code 

Helper function for DnsNameGlobPattern.

Returns regular expression pattern for parts of the pattern.

DnsNameGlobPattern(pattern)

source code 

Generates regular expression from DNS name globbing pattern.

A DNS name globbing pattern (e.g. *.site) is converted to a regular expression. Escape sequences or ranges (e.g. [a-z]) are not supported.

Matching always starts at the leftmost part. An asterisk (*) matches all characters except the dot (.) separating DNS name parts. A question mark (?) matches a single character except the dot (.).

Parameters:
  • pattern (string) - DNS name globbing pattern
Returns: string
Regular expression

FormatUnit(value, units, roman=False)

source code 

Formats an incoming number of MiB with the appropriate unit.

Parameters:
  • value (int) - integer representing the value in MiB (1048576)
  • units (char) - the type of formatting we should do:
    • 'h' for automatic scaling
    • 'm' for MiBs
    • 'g' for GiBs
    • 't' for TiBs
Returns: str
the formatted value (with suffix)

ParseUnit(input_string)

source code 

Tries to extract number and scale from the given string.

Input must be in the format NUMBER+ [DOT NUMBER+] SPACE* [UNIT]. If no unit is specified, it defaults to MiB. Return value is always an int in MiB.

ShellQuote(value)

source code 

Quotes shell argument according to POSIX.

Parameters:
  • value (str) - the argument to be quoted
Returns: str
the quoted value

ShellQuoteArgs(args)

source code 

Quotes a list of shell arguments.

Parameters:
  • args (list) - list of arguments to be quoted
Returns: str
the quoted arguments concatenated with spaces

GenerateSecret(numbytes=20)

source code 

Generates a random secret.

This will generate a pseudo-random secret returning an hex string (so that it can be used where an ASCII string is needed).

Parameters:
  • numbytes - the number of bytes which will be represented by the returned string (defaulting to 20, the length of a SHA1 hash)
Returns: str
an hex representation of the pseudo-random sequence

_MakeMacAddrRegexp(octets)

source code 

Builds a regular expression for verifying MAC addresses.

Parameters:
  • octets (integer) - How many octets to expect (1-6)
Returns:
Compiled regular expression

_MacAddressCheck(check_re, mac, msg)

source code 

Checks a MAC address using a regular expression.

Parameters:
  • check_re - Compiled regular expression as returned by re.compile
  • mac (string) - MAC address to be validated
  • msg (string) - Error message (%s will be replaced with MAC address)

NormalizeAndValidateMac(mac)

source code 

Normalizes and check if a MAC address is valid and contains six octets.

Checks whether the supplied MAC address is formally correct. Accepts colon-separated format only. Normalize it to all lower case.

Parameters:
  • mac (string) - MAC address to be validated
Returns: string
Normalized and validated MAC address
Raises:

NormalizeAndValidateThreeOctetMacPrefix(mac)

source code 

Normalizes a potential MAC address prefix (three octets).

Checks whether the supplied string is a valid MAC address prefix consisting of three colon-separated octets. The result is normalized to all lower case.

Parameters:
  • mac (string) - Prefix to be validated
Returns: string
Normalized and validated prefix
Raises:

SafeEncode(text)

source code 

Return a 'safe' version of a source string.

This function mangles the input string and returns a version that should be safe to display/encode as ASCII. To this end, we first convert it to ASCII using the 'backslashreplace' encoding which should get rid of any non-ASCII chars, and then we process it through a loop copied from the string repr sources in the python; we don't use string_escape anymore since that escape single quotes and backslashes too, and that is too much; and that escaping is not stable, i.e. string_escape(string_escape(x)) != string_escape(x).

Parameters:
  • text (str or unicode) - input data
Returns: str
a safe version of text

UnescapeAndSplit(text, sep=",")

source code 

Split and unescape a string based on a given separator.

This function splits a string based on a separator where the separator itself can be escape in order to be an element of the elements. The escaping rules are (assuming coma being the separator):

  • a plain , separates the elements
  • a sequence \\\\, (double backslash plus comma) is handled as a backslash plus a separator comma
  • a sequence \, (backslash plus comma) is handled as a non-separator comma
Parameters:
  • text (string) - the string to split
  • text (string) - the separator
  • sep (string)
Returns: string
a list of strings

EscapeAndJoin(slist, sep=",")

source code 

Encode a list in a way parsable by UnescapeAndSplit.

Parameters:
  • slist (list of strings) - the strings to be encoded
Returns: string
the encoding of the list oas a string

CommaJoin(names)

source code 

Nicely join a set of identifiers.

Parameters:
  • names - set, list or tuple
Returns:
a string with the formatted results

FormatTime(val, usecs=None)

source code 

Formats a time value.

Parameters:
  • val (float or None) - Timestamp as returned by time.time() (seconds since Epoch, 1970-01-01 00:00:00 UTC)
Returns:
a string value or N/A if we don't have a valid timestamp

FormatSeconds(secs)

source code 

Formats seconds for easier reading.

Parameters:
  • secs (number) - Number of seconds
Returns: string
Formatted seconds (e.g. "2d 9h 19m 49s")

IsValidShellParam(word)

source code 

Verifies is the given word is safe from the shell's p.o.v.

This means that we can pass this to a command via the shell and be sure that it doesn't alter the command line and is passed as such to the actual command.

Note that we are overly restrictive here, in order to be on the safe side.

Parameters:
  • word (str) - the word to check
Returns: boolean
True if the word is 'safe'

BuildShellCmd(template, *args)

source code 

Build a safe shell command line from the given arguments.

This function will check all arguments in the args list so that they are valid shell parameters (i.e. they don't contain shell metacharacters). If everything is ok, it will return the result of template % args.

Parameters:
  • template (str) - the string holding the template for the string formatting
Returns: str
the expanded command line

FormatOrdinal(value)

source code 

Formats a number as an ordinal in the English language.

E.g. the number 1 becomes "1st", 22 becomes "22nd".

Parameters:
  • value (integer) - Number
Returns: string

Truncate(text, length)

source code 

Truncate string and add ellipsis if needed.

Parameters:
  • text (string) - Text
  • length (integer) - Desired length
Returns: string
Truncated text

FilterEmptyLinesAndComments(text)

source code 

Filters empty lines and comments from a line-based string.

Whitespace is also removed from the beginning and end of all lines.

Parameters:
  • text (string) - Input string
Returns: list

FormatKeyValue(data)

source code 

Formats a dictionary as "key=value" parameters.

The keys are sorted to have a stable order.

Parameters:
  • data (dict)
Returns: list of string