Trees       Indices       Help   
Ganeti
Package ganeti :: Package utils :: Module io
[hide private]
[frames] | no frames]

Module io

source code

Utility functions for I/O.

Classes [hide private]
  FileStatHelper
Helper to store file handle's fstat.
  TemporaryFileManager
Stores the list of files to be deleted and removes them on demand.
Functions [hide private]
 
ErrnoOrStr(err)
Format an EnvironmentError exception.		 source code
str
ReadFile(file_name, size=-1, preread=None)
Reads a file.		 source code
None or int
WriteFile(file_name, fn=None, data=None, mode=None, uid=-1, gid=-1, atime=None, mtime=None, close=True, dry_run=False, backup=False, prewrite=None, postwrite=None, keep_perms=KP_NEVER)
(Over)write a file atomically.		 source code
 
GetFileID(path=None, fd=None)
Returns the file 'id', i.e.		 source code
boolean
VerifyFileID(fi_disk, fi_ours)
Verifies that two file IDs are matching.		 source code
 
SafeWriteFile(file_name, file_id, **kwargs)
Wraper over WriteFile that locks the target file.		 source code
 
ReadOneLineFile(file_name, strict=False)
Return the first non-empty line from a file.		 source code
 
RemoveFile(filename)
Remove a file ignoring some errors.		 source code
 
RemoveDir(dirname)
Remove an empty directory.		 source code
 
RenameFile(old, new, mkdir=False, mkdir_mode=0750, dir_uid=None, dir_gid=None)
Renames a file.		 source code
 
EnforcePermission(path, mode, uid=None, gid=None, must_exist=True, _chmod_fn=os.chmod, _chown_fn=os.chown, _stat_fn=os.stat)
Enforces that given path has given permissions.		 source code
 
MakeDirWithPerm(path, mode, uid, gid, _lstat_fn=os.lstat, _mkdir_fn=os.mkdir, _perm_fn=EnforcePermission)
Enforces that given path is a dir and has given mode, uid and gid set.		 source code
 
Makedirs(path, mode=0750)
Super-mkdir; create a leaf directory and all intermediate ones.		 source code
 
TimestampForFilename()
Returns the current time formatted for filenames.		 source code
str
CreateBackup(file_name)
Creates a backup of a file.		 source code
list
ListVisibleFiles(path, _is_mountpoint=os.path.ismount)
Returns a list of visible files in a directory.		 source code
 
EnsureDirs(dirs)
Make required directories, if they don't exist.		 source code
str or None
FindFile(name, search_path, test=os.path.exists)
Look for a filesystem object in a given path.		 source code
 
IsNormAbsPath(path)
Check whether a path is absolute and also normalized		 source code
 
IsBelowDir(root, other_path)
Check whether a path is below a root dir.		 source code
 
IsUrl(path)
Check whether a path is a HTTP URL.		 source code
 
PathJoin(*args)
Safe-join a list of path components.		 source code
 
TailFile(fname, lines=20)
Return the last lines from a file.		 source code
int
BytesToMebibyte(value)
Converts bytes to mebibytes.		 source code
int
CalculateDirectorySize(path)
Calculates the size of a directory recursively.		 source code
int
GetFilesystemStats(path)
Returns the total and free space on a filesystem.		 source code
int
ReadPidFile(pidfile)
Read a pid from a file.		 source code
int
_ParsePidFileContents(data)
Tries to extract a process ID from a PID file's content.		 source code
 
ReadLockedPidFile(path)
Reads a locked PID file.		 source code
str
DaemonPidFileName(name)
Compute a ganeti pid file absolute path		 source code
int
WritePidFile(pidfile)
Write the current process pidfile.		 source code
 
ReadWatcherPauseFile(filename, now=None, remove_after=3600)
Reads the watcher pause file.		 source code
str
NewUUID()
Returns a random UUID.		 source code
bool
IsUserInGroup(uid, gid)
Returns True if the user belongs to the group.		 source code
bool
CanRead(username, filename)
Returns True if the user can access (read) the file.		 source code
Variables [hide private]
  _LOST_AND_FOUND = "lost+found"
Directory used by fsck(8) to store recovered data, usually at a file system's root directory
  KP_NEVER = 0
  KP_ALWAYS = 1
  KP_IF_EXISTS = 2
  KEEP_PERMS_VALUES = [KP_NEVER, KP_ALWAYS, KP_IF_EXISTS,]
  URL_RE = re.compile(r'(https?|ftps?)://')

Imports: os, re, logging, shutil, tempfile, errno, time, stat, grp, pwd, errors, constants, pathutils, filelock


Function Details [hide private]

ErrnoOrStr(err)

source code 

Format an EnvironmentError exception.

If the err argument has an errno attribute, it will be looked up and converted into a textual E... description. Otherwise the string representation of the error will be returned.

Parameters:
  • err (EnvironmentError) - the exception to format

ReadFile(file_name, size=-1, preread=None)

source code 

Reads a file.

Parameters:
  • size (int) - Read at most size bytes (if negative, entire file)
  • preread (callable receiving file handle as single parameter) - Function called before file is read
Returns: str
the (possibly partial) content of the file
Raises:
  • IOError - if the file cannot be opened

WriteFile(file_name, fn=None, data=None, mode=None, uid=-1, gid=-1, atime=None, mtime=None, close=True, dry_run=False, backup=False, prewrite=None, postwrite=None, keep_perms=KP_NEVER)

source code 

(Over)write a file atomically.

The file_name and either fn (a function taking one argument, the file descriptor, and which should write the data to it) or data (the contents of the file) must be passed. The other arguments are optional and allow setting the file mode, owner and group, and the mtime/atime of the file.

If the function doesn't raise an exception, it has succeeded and the target file has the new contents. If the function has raised an exception, an existing target file should be unmodified and the temporary file should be removed.

Parameters:
  • file_name (str) - the target filename
  • fn (callable) - content writing function, called with file descriptor as parameter
  • data (str) - contents of the file
  • mode (int) - file mode
  • uid (int) - the owner of the file
  • gid (int) - the group of the file
  • atime (int) - a custom access time to be set on the file
  • mtime (int) - a custom modification time to be set on the file
  • close (boolean) - whether to close file after writing it
  • prewrite (callable) - function to be called before writing content
  • postwrite (callable) - function to be called after writing content
  • keep_perms (members of KEEP_PERMS_VALUES) - if KP_NEVER (default), owner, group, and mode are taken from the other parameters; if KP_ALWAYS, owner, group, and mode are copied from the existing file; if KP_IF_EXISTS, owner, group, and mode are taken from the file, and if the file doesn't exist, they are taken from the other parameters. It is an error to pass KP_ALWAYS when the file doesn't exist or when uid, gid, or mode are set to non-default values.
Returns: None or int
None if the 'close' parameter evaluates to True, otherwise the file descriptor
Raises:

GetFileID(path=None, fd=None)

source code 

Returns the file 'id', i.e. the dev/inode and mtime information.

Either the path to the file or the fd must be given.

Parameters:
  • path - the file path
  • fd - a file descriptor
Returns:
a tuple of (device number, inode number, mtime)

VerifyFileID(fi_disk, fi_ours)

source code 

Verifies that two file IDs are matching.

Differences in the inode/device are not accepted, but and older timestamp for fi_disk is accepted.

Parameters:
  • fi_disk - tuple (dev, inode, mtime) representing the actual file data
  • fi_ours - tuple (dev, inode, mtime) representing the last written file data
Returns: boolean

SafeWriteFile(file_name, file_id, **kwargs)

source code 

Wraper over WriteFile that locks the target file.

By keeping the target file locked during WriteFile, we ensure that cooperating writers will safely serialise access to the file.

Parameters:
  • file_name (str) - the target filename
  • file_id (tuple) - a result from GetFileID

ReadOneLineFile(file_name, strict=False)

source code 

Return the first non-empty line from a file.

Parameters:
  • strict (boolean) - if True, abort if the file has more than one non-empty line

RemoveFile(filename)

source code 

Remove a file ignoring some errors.

Remove a file, ignoring non-existing ones or directories. Other errors are passed.

Parameters:
  • filename (str) - the file to be removed

RemoveDir(dirname)

source code 

Remove an empty directory.

Remove a directory, ignoring non-existing ones. Other errors are passed. This includes the case, where the directory is not empty, so it can't be removed.

Parameters:
  • dirname (str) - the empty directory to be removed

RenameFile(old, new, mkdir=False, mkdir_mode=0750, dir_uid=None, dir_gid=None)

source code 

Renames a file.

This just creates the very least directory if it does not exist and mkdir is set to true.

Parameters:
  • old (string) - Original path
  • new (string) - New path
  • mkdir (bool) - Whether to create target directory if it doesn't exist
  • mkdir_mode (int) - Mode for newly created directories
  • dir_uid (int) - The uid for the (if fresh created) dir
  • dir_gid (int) - The gid for the (if fresh created) dir

EnforcePermission(path, mode, uid=None, gid=None, must_exist=True, _chmod_fn=os.chmod, _chown_fn=os.chown, _stat_fn=os.stat)

source code 

Enforces that given path has given permissions.

Parameters:
  • path - The path to the file
  • mode - The mode of the file
  • uid - The uid of the owner of this file
  • gid - The gid of the owner of this file
  • must_exist - Specifies if non-existance of path will be an error
  • _chmod_fn - chmod function to use (unittest only)
  • _chown_fn - chown function to use (unittest only)

MakeDirWithPerm(path, mode, uid, gid, _lstat_fn=os.lstat, _mkdir_fn=os.mkdir, _perm_fn=EnforcePermission)

source code 

Enforces that given path is a dir and has given mode, uid and gid set.

Parameters:
  • path - The path to the file
  • mode - The mode of the file
  • uid - The uid of the owner of this file
  • gid - The gid of the owner of this file
  • _lstat_fn - Stat function to use (unittest only)
  • _mkdir_fn - mkdir function to use (unittest only)
  • _perm_fn - permission setter function to use (unittest only)

Makedirs(path, mode=0750)

source code 

Super-mkdir; create a leaf directory and all intermediate ones.

This is a wrapper around os.makedirs adding error handling not implemented before Python 2.5.

TimestampForFilename()

source code 

Returns the current time formatted for filenames.

The format doesn't contain colons as some shells and applications treat them as separators. Uses the local timezone.

CreateBackup(file_name)

source code 

Creates a backup of a file.

Parameters:
  • file_name (str) - file to be backed up
Returns: str
the path to the newly created backup
Raises:

ListVisibleFiles(path, _is_mountpoint=os.path.ismount)

source code 

Returns a list of visible files in a directory.

Parameters:
  • path (str) - the directory to enumerate
Returns: list
the list of all files not starting with a dot
Raises:

EnsureDirs(dirs)

source code 

Make required directories, if they don't exist.

Parameters:
  • dirs (list of (string, integer)) - list of tuples (dir_name, dir_mode)

FindFile(name, search_path, test=os.path.exists)

source code 

Look for a filesystem object in a given path.

This is an abstract method to search for filesystem object (files, dirs) under a given search path.

Parameters:
  • name (str) - the name to look for
  • search_path (iterable of string) - locations to start at
  • test (callable) - a function taking one argument that should return True if the a given object is valid; the default value is os.path.exists, causing only existing files to be returned
Returns: str or None
full path to the object if found, None otherwise

IsNormAbsPath(path)

source code 

Check whether a path is absolute and also normalized

This avoids things like /dir/../../other/path to be valid.

IsBelowDir(root, other_path)

source code 

Check whether a path is below a root dir.

This works around the nasty byte-byte comparison of commonprefix.

PathJoin(*args)

source code 

Safe-join a list of path components.

Requirements:

  • the first argument must be an absolute path
  • no component in the path must have backtracking (e.g. /../), since we check for normalization at the end
Parameters:
  • args - the path components to be joined
Raises:
  • ValueError - for invalid paths

TailFile(fname, lines=20)

source code 

Return the last lines from a file.

Parameters:
  • fname - the file name
  • lines (int) - the (maximum) number of lines to return

Note: this function will only read and parse the last 4KB of the file; if the lines are very long, it could be that less than the requested number of lines are returned

BytesToMebibyte(value)

source code 

Converts bytes to mebibytes.

Parameters:
  • value (int) - Value in bytes
Returns: int
Value in mebibytes

CalculateDirectorySize(path)

source code 

Calculates the size of a directory recursively.

Parameters:
  • path (string) - Path to directory
Returns: int
Size in mebibytes

GetFilesystemStats(path)

source code 

Returns the total and free space on a filesystem.

Parameters:
  • path (string) - Path on filesystem to be examined
Returns: int
tuple of (Total space, Free space) in mebibytes

ReadPidFile(pidfile)

source code 

Read a pid from a file.

Parameters:
  • pidfile (string) - path to the file containing the pid
Returns: int
The process id, if the file exists and contains a valid PID, otherwise 0

_ParsePidFileContents(data)

source code 

Tries to extract a process ID from a PID file's content.

Parameters:
  • data (string)
Returns: int
Zero if nothing could be read, PID otherwise

ReadLockedPidFile(path)

source code 

Reads a locked PID file.

This can be used together with utils.process.StartDaemon.

Parameters:
  • path (string) - Path to PID file
Returns:
PID as integer or, if file was unlocked or couldn't be opened, None

DaemonPidFileName(name)

source code 

Compute a ganeti pid file absolute path

Parameters:
  • name (str) - the daemon name
Returns: str
the full path to the pidfile corresponding to the given daemon name

WritePidFile(pidfile)

source code 

Write the current process pidfile.

Parameters:
  • pidfile (string) - the path to the file to be written
Returns: int
the file descriptor of the lock file; do not close this unless you want to unlock the pid file
Raises:
  • errors.LockError - if the pid file already exists and points to a live process

ReadWatcherPauseFile(filename, now=None, remove_after=3600)

source code 

Reads the watcher pause file.

Parameters:
  • filename (string) - Path to watcher pause file
  • now (None, float or int) - Current time as Unix timestamp
  • remove_after (int) - Remove watcher pause file after specified amount of seconds past the pause end time

NewUUID()

source code 

Returns a random UUID.

Returns: str

Note: This is a Linux-specific method as it uses the /proc filesystem.

IsUserInGroup(uid, gid)

source code 

Returns True if the user belongs to the group.

Parameters:
  • uid (int) - the user id
  • gid (int) - the group id
Returns: bool

CanRead(username, filename)

source code 

Returns True if the user can access (read) the file.

Parameters:
  • username (string) - the name of the user
  • filename (string) - the name of the file
Returns: bool

   Trees       Indices       Help   
Ganeti
Generated by Epydoc 3.0.1 on Mon Aug 22 11:06:08 2016 http://epydoc.sourceforge.net