module documentation

Utility functions for manipulating or working with text.

Class LineSplitter Splits byte data chunks into lines of text separated by newline.
Class ShellWriter Helper class to write scripts with indentation.
Function BuildShellCmd Build a safe shell command line from the given arguments.
Function CommaJoin Nicely join a set of identifiers.
Function DnsNameGlobPattern Generates regular expression from DNS name globbing pattern.
Function EscapeAndJoin Encode a list in a way parsable by UnescapeAndSplit.
Function FilterEmptyLinesAndComments Filters empty lines and comments from a line-based string.
Function FormatKeyValue Formats a dictionary as "key=value" parameters.
Function FormatOrdinal Formats a number as an ordinal in the English language.
Function FormatSeconds Formats seconds for easier reading.
Function FormatTime Formats a time value.
Function FormatUnit Formats an incoming number of MiB with the appropriate unit.
Function GenerateSecret Generates a random secret.
Function IsValidShellParam Verifies is the given word is safe from the shell's p.o.v.
Function MatchNameComponent Try to match a name against a list.
Function NormalizeAndValidateMac Normalizes and check if a MAC address is valid and contains six octets.
Function NormalizeAndValidateThreeOctetMacPrefix Normalizes a potential MAC address prefix (three octets).
Function ParseUnit Tries to extract number and scale from the given string.
Function SafeEncode Return a 'safe' version of a source string.
Function ShellCombineCommands Out of a list of shell comands construct a single one.
Function ShellQuote Quotes shell argument according to POSIX.
Function ShellQuoteArgs Quotes a list of shell arguments.
Function Truncate Truncate string and add ellipsis if needed.
Function UnescapeAndSplit Split and unescape a string based on a given separator.
Function _DnsNameGlobHelper Helper function for DnsNameGlobPattern.
Function _MacAddressCheck Checks a MAC address using a regular expression.
Function _MakeMacAddrRegexp Builds a regular expression for verifying MAC addresses.
Constant _ASCII_ELLIPSIS Undocumented
Constant _MAC_ADDR_OCTET_RE Undocumented
Constant _MAC_CHECK_RE Undocumented
Constant _MAC_PREFIX_CHECK_RE Undocumented
Constant _PARSEUNIT_REGEX Undocumented
Constant _SHELL_UNQUOTED_RE Undocumented
Constant _SHELLPARAM_REGEX Undocumented
def BuildShellCmd(template, *args):

Build a safe shell command line from the given arguments.

This function will check all arguments in the args list so that they are valid shell parameters (i.e. they don't contain shell metacharacters). If everything is ok, it will return the result of template % args.

Parameters
template:strthe string holding the template for the string formatting
*argsUndocumented
Returns
strthe expanded command line
def CommaJoin(names):

Nicely join a set of identifiers.

Parameters
namesset, list or tuple
Returns
a string with the formatted results
def DnsNameGlobPattern(pattern):

Generates regular expression from DNS name globbing pattern.

A DNS name globbing pattern (e.g. *.site) is converted to a regular expression. Escape sequences or ranges (e.g. [a-z]) are not supported.

Matching always starts at the leftmost part. An asterisk (*) matches all characters except the dot (.) separating DNS name parts. A question mark (?) matches a single character except the dot (.).

Parameters
pattern:stringDNS name globbing pattern
Returns
stringRegular expression
def EscapeAndJoin(slist, sep=','):

Encode a list in a way parsable by UnescapeAndSplit.

Parameters
slist:list of stringsthe strings to be encoded
sepUndocumented
Returns
stringthe encoding of the list oas a string
def FilterEmptyLinesAndComments(text):

Filters empty lines and comments from a line-based string.

Whitespace is also removed from the beginning and end of all lines.

Parameters
text:stringInput string
Returns
listUndocumented
def FormatKeyValue(data):

Formats a dictionary as "key=value" parameters.

The keys are sorted to have a stable order.

Parameters
data:dictUndocumented
Returns
list of stringUndocumented
def FormatOrdinal(value):

Formats a number as an ordinal in the English language.

E.g. the number 1 becomes "1st", 22 becomes "22nd".

Parameters
value:integerNumber
Returns
stringUndocumented
def FormatSeconds(secs):

Formats seconds for easier reading.

Parameters
secs:numberNumber of seconds
Returns
stringFormatted seconds (e.g. "2d 9h 19m 49s")
def FormatTime(val, usecs=None):

Formats a time value.

Parameters
val:float or NoneTimestamp as returned by time.time() (seconds since Epoch, 1970-01-01 00:00:00 UTC)
usecsUndocumented
Returns
a string value or N/A if we don't have a valid timestamp
def FormatUnit(value, units, roman=False):

Formats an incoming number of MiB with the appropriate unit.

Parameters
value:intinteger representing the value in MiB (1048576)
units:char

the type of formatting we should do:

  • 'h' for automatic scaling
  • 'm' for MiBs
  • 'g' for GiBs
  • 't' for TiBs
romanUndocumented
Returns
strthe formatted value (with suffix)
def GenerateSecret(numbytes=20):

Generates a random secret.

This will generate a pseudo-random secret returning an hex string (so that it can be used where an ASCII string is needed).

Parameters
numbytesthe number of bytes which will be represented by the returned string (defaulting to 20, the length of a SHA1 hash)
Returns
stran hex representation of the pseudo-random sequence
def IsValidShellParam(word):

Verifies is the given word is safe from the shell's p.o.v.

This means that we can pass this to a command via the shell and be sure that it doesn't alter the command line and is passed as such to the actual command.

Note that we are overly restrictive here, in order to be on the safe side.

Parameters
word:strthe word to check
Returns
booleanTrue if the word is 'safe'
def MatchNameComponent(key, name_list, case_sensitive=True):

Try to match a name against a list.

This function will try to match a name like test1 against a list like ['test1.example.com', 'test2.example.com', ...]. Against this list, 'test1' as well as 'test1.example' will match, but not 'test1.ex'. A multiple match will be considered as no match at all (e.g. 'test1' against ['test1.example.com', 'test1.example.org']), except when the key fully matches an entry (e.g. 'test1' against ['test1', 'test1.example.com']).

Parameters
key:strthe name to be searched
name_list:listthe list of strings against which to search the key
case_sensitive:booleanwhether to provide a case-sensitive match
Returns
None or strNone if there is no match or if there are multiple matches, otherwise the element from the list which matches
def NormalizeAndValidateMac(mac):

Normalizes and check if a MAC address is valid and contains six octets.

Checks whether the supplied MAC address is formally correct. Accepts colon-separated format only. Normalize it to all lower case.

Parameters
mac:stringMAC address to be validated
Returns
stringNormalized and validated MAC address
Raises
errors.OpPrereqErrorIf the MAC address isn't valid
def NormalizeAndValidateThreeOctetMacPrefix(mac):

Normalizes a potential MAC address prefix (three octets).

Checks whether the supplied string is a valid MAC address prefix consisting of three colon-separated octets. The result is normalized to all lower case.

Parameters
mac:stringPrefix to be validated
Returns
stringNormalized and validated prefix
Raises
errors.OpPrereqErrorIf the MAC address prefix isn't valid
def ParseUnit(input_string):

Tries to extract number and scale from the given string.

Input must be in the format NUMBER+ [DOT NUMBER+] SPACE* [UNIT]. If no unit is specified, it defaults to MiB. Return value is always an int in MiB.

def SafeEncode(text):

Return a 'safe' version of a source string.

This function mangles the input string and returns a version that should be safe to display/encode as ASCII. To this end, we first convert it to ASCII using the 'backslashreplace' encoding which should get rid of any non-ASCII chars, and then we process it through a loop copied from the string repr sources in the python; we don't use string_escape anymore since that escape single quotes and backslashes too, and that is too much; and that escaping is not stable, i.e. string_escape(string_escape(x)) != string_escape(x).

Parameters
text:str or unicodeinput data
Returns
stra safe version of text
def ShellCombineCommands(cmdlist):

Out of a list of shell comands construct a single one.

def ShellQuote(value):

Quotes shell argument according to POSIX.

Parameters
value:strthe argument to be quoted
Returns
strthe quoted value
def ShellQuoteArgs(args):

Quotes a list of shell arguments.

Parameters
args:listlist of arguments to be quoted
Returns
strthe quoted arguments concatenated with spaces
def Truncate(text, length):

Truncate string and add ellipsis if needed.

Parameters
text:stringText
length:integerDesired length
Returns
stringTruncated text
def UnescapeAndSplit(text, sep=','):

Split and unescape a string based on a given separator.

This function splits a string based on a separator where the separator itself can be escape in order to be an element of the elements. The escaping rules are (assuming coma being the separator):

  • a plain , separates the elements
  • a sequence \\\\, (double backslash plus comma) is handled as a backslash plus a separator comma
  • a sequence \, (backslash plus comma) is handled as a non-separator comma
Parameters
text:stringthe separator
sep:stringUndocumented
Returns
stringa list of strings
def _DnsNameGlobHelper(match):

Helper function for DnsNameGlobPattern.

Returns regular expression pattern for parts of the pattern.

def _MacAddressCheck(check_re, mac, msg):

Checks a MAC address using a regular expression.

Parameters
check_reCompiled regular expression as returned by re.compile
mac:stringMAC address to be validated
msg:stringError message (%s will be replaced with MAC address)
def _MakeMacAddrRegexp(octets):

Builds a regular expression for verifying MAC addresses.

Parameters
octets:integerHow many octets to expect (1-6)
Returns
Compiled regular expression
_ASCII_ELLIPSIS: str =

Undocumented

Value
'...'
_MAC_ADDR_OCTET_RE: str =

Undocumented

Value
'[0-9a-f]{2}'
_MAC_CHECK_RE =

Undocumented

Value
_MakeMacAddrRegexp(6)
_MAC_PREFIX_CHECK_RE =

Undocumented

Value
_MakeMacAddrRegexp(3)
_PARSEUNIT_REGEX =

Undocumented

Value
re.compile(r'^([\.\d]+)\s*([a-zA-Z]+)?$')
_SHELL_UNQUOTED_RE =

Undocumented

Value
re.compile(r'^[-\.,=:/_\+@A-Za-z0-9]+$')
_SHELLPARAM_REGEX =

Undocumented

Value
re.compile(r'^[-a-zA-Z0-9\._\+/:%@]+$')