ganeti.workerpool.WorkerPool

class documentation

class WorkerPool(object):

Worker pool with a queue.

This class is thread-safe.

Tasks are guaranteed to be started in the order in which they're added to the pool. Due to the nature of threading, they're not guaranteed to finish in the same order.

Method	`__init__`	Constructor for worker pool.
Method	`AddManyTasks`	Add a list of tasks to the queue.
Method	`AddTask`	Adds a task to the queue.
Method	`ChangeTaskPriority`	Changes a task's priority.
Method	`HasRunningTasks`	Checks whether there's at least one task running.
Method	`Quiesce`	Waits until the task queue is empty.
Method	`Resize`	Changes the number of workers in the pool.
Method	`TerminateWorkers`	Terminate all worker threads.
Method	`_AddTaskUnlocked`	Adds a task to the internal queue.
Method	`_HasRunningTasksUnlocked`	Checks whether there's a task running in a worker.
Method	`_NewWorkerIdUnlocked`	Return an identifier for a new worker.
Method	`_ResizeUnlocked`	Changes the number of workers.
Method	`_ShouldWorkerTerminateUnlocked`	Returns whether a worker should terminate.
Method	`_WaitForTaskUnlocked`	Waits for a task for a worker.
Method	`_WaitWhileQuiescingUnlocked`	Wait until the worker pool has finished quiescing.
Instance Variable	`_counter`	Undocumented
Instance Variable	`_last_worker_id`	Undocumented
Instance Variable	`_lock`	Undocumented
Instance Variable	`_name`	Undocumented
Instance Variable	`_pool_to_pool`	Undocumented
Instance Variable	`_pool_to_worker`	Undocumented
Instance Variable	`_quiescing`	Undocumented
Instance Variable	`_taskdata`	Mapping from task IDs to entries in `_tasks`
Instance Variable	`_tasks`	Each tuple has the format (priority, order ID, task ID, arguments). Priority and order ID are numeric and essentially control the sort order. The order ID is an increasing number denoting the order in which tasks are added to the queue...
Instance Variable	`_termworkers`	Undocumented
Instance Variable	`_worker_class`	Undocumented
Instance Variable	`_worker_to_pool`	Undocumented
Instance Variable	`_workers`	Undocumented

def __init__(self, name, num_workers, worker_class): ¶

Constructor for worker pool.

Parameters
name	Undocumented
num_workers	number of workers to be started (dynamic resizing is not yet implemented)
worker_class	the class to be instantiated for workers; should derive from `BaseWorker`

def AddManyTasks(self, tasks, priority=_DEFAULT_PRIORITY, task_id=None): ¶

Add a list of tasks to the queue.

Parameters
tasks:list of tuples	list of args passed to `BaseWorker.RunTask`
priority:number or list of numbers	Priority for all added tasks or a list with the priority for each task
task_id:list	List with the ID for each task
Note
See `AddTask` for a note on task IDs.

def AddTask(self, args, priority=_DEFAULT_PRIORITY, task_id=None): ¶

Adds a task to the queue.

Parameters
args:sequence	arguments passed to `BaseWorker.RunTask`
priority:number	Task priority
task_id	Task ID
Note
The task ID can be essentially anything that can be used as a dictionary key. Callers, however, must ensure a task ID is unique while a task is in the pool or while it might return to the pool due to deferring using `DeferTask`.

def ChangeTaskPriority(self, task_id, priority): ¶

Changes a task's priority.

Parameters
task_id	Task ID
priority:number	New task priority
Raises
`NoSuchTask`	When the task referred by `task_id` can not be found (it may never have existed, may have already been processed, or is currently running)

def HasRunningTasks(self): ¶

Checks whether there's at least one task running.

def Quiesce(self): ¶

Waits until the task queue is empty.

def Resize(self, num_workers): ¶

Changes the number of workers in the pool.

Parameters
num_workers	the new number of workers

def TerminateWorkers(self): ¶

Terminate all worker threads.

Unstarted tasks will be ignored.

def _AddTaskUnlocked(self, args, priority, task_id): ¶

Adds a task to the internal queue.

Parameters
args:sequence	Arguments passed to `BaseWorker.RunTask`
priority:number	Task priority
task_id	Task ID

def _HasRunningTasksUnlocked(self): ¶

Checks whether there's a task running in a worker.

def _NewWorkerIdUnlocked(self): ¶

Return an identifier for a new worker.

def _ResizeUnlocked(self, num_workers): ¶

Changes the number of workers.

def _ShouldWorkerTerminateUnlocked(self, worker): ¶

Returns whether a worker should terminate.

def _WaitForTaskUnlocked(self, worker): ¶

Waits for a task for a worker.

Parameters
worker:`BaseWorker`	Worker thread

def _WaitWhileQuiescingUnlocked(self): ¶

Wait until the worker pool has finished quiescing.

_counter = ¶

Undocumented

_last_worker_id: int = ¶

Undocumented

_lock = ¶

Undocumented

_name = ¶

Undocumented

_pool_to_pool = ¶

Undocumented

_pool_to_worker = ¶

Undocumented

_quiescing: bool = ¶

Undocumented

_taskdata: dict; (task IDs as keys, tuples as values) = ¶

Mapping from task IDs to entries in _tasks

_tasks: list of tuples = ¶

Each tuple has the format (priority, order ID, task ID, arguments). Priority and order ID are numeric and essentially control the sort order. The order ID is an increasing number denoting the order in which tasks are added to the queue. The task ID is controlled by user of workerpool, see AddTask for details. The task arguments are None for abandoned tasks, otherwise a sequence of arguments to be passed to BaseWorker.RunTask). The list must fulfill the heap property (for use by the heapq module).

_termworkers: list = ¶

Undocumented

_worker_class = ¶

Undocumented

_worker_to_pool = ¶

Undocumented

_workers: list = ¶

Undocumented