Documentation

Representation of the job queue

We keep two lists of jobs (together with information about the last fstat result observed): the jobs that are enqueued, but not yet handed over for execution, and the jobs already handed over for execution. They are kept together in a single IORef, so that we can atomically update both, in particular when scheduling jobs to be handed over for execution.

Apply a function on the running jobs.

Apply a function on the queued jobs.

Obtain a JobWithStat from a QueuedJob.

Reload interval for polling the running jobs for updates in microseconds.

Read a cluster parameter from the configuration, using a default if the configuration is not available.

Get the maximual number of jobs to be run simultaneously from the configuration. If the configuration is not available, be conservative and use the smallest possible value, i.e., 1.

Get the maximual number of jobs to be tracked simultaneously from the configuration. If the configuration is not available, be conservative and use the smallest possible value, i.e., 1.

Get the number of jobs currently running.

Wrapper function to atomically update the jobs in the queue status.

Reread a job from disk, if the file has changed.

Update a job in the job queue, if it is still there. This is the pure function for inserting a previously read change into the queue. as the change contains its time stamp, we don't have to worry about a later read change overwriting a newer read state. If this happens, the fstat value will be outdated, so the next poller run will fix this.

Update a single job by reading it from disk, if necessary.

Move a job from one part of the queue to another. Return the job that was moved, or Nothing if it wasn't found in the queue.

Atomically move a job from one part of the queue to another. Return the job that was moved, or Nothing if it wasn't found in the queue.

Manipulate a running job by atomically moving it from qRunning into qManipulated, running a given IO action and then atomically returning it back.

Returns the result of the IO action, or Nothing, if the job wasn't found in the queue.

Sort out the finished jobs from the monitored part of the queue. This is the pure part, splitting the queue into a remaining queue and the jobs that were removed.

Actually clean up the finished jobs. This is the IO wrapper around the pure sortoutFinishedJobs.

Watcher task for a job, to update it on file changes. It also reinstantiates itself upon receiving an Ignored event.

Attach the job watcher to a running job.

For a queued job, determine whether it is eligible to run, i.e., if no jobs it depends on are either enqueued or running.

Decide on which jobs to schedule next for execution. This is the pure function doing the scheduling.

Logs errors of failed jobs and returns the set of job IDs.

Fail jobs that were previously selected for execution but couldn't be started.

Checks if any jobs match a REJECT filter rule, and cancels them.

Schedule jobs to be run. This is the IO wrapper around the pure selectJobsToRun.

Format the job queue status in a compact, human readable way.

Check if a job died, and clean up if so. Return True, if the job was found dead.

Trigger job detection for the job with the given job id. Return True, if the job is dead.

Force the queue to check the state of all jobs.

Time-based watcher for updating the job queue.

Read a single, non-archived, job, specified by its id, from disk.

Read all non-finalized jobs from disk.

Set up the job scheduler. This will also start the monitoring of changes to the running jobs.

Enqueue new jobs. This will guarantee that the jobs will be executed eventually.

Pure function for removing a queued job from the job queue by atomicModifyIORef. The answer is Just the job if the job could be removed before being handed over to execution, Nothing if it already was started and a Bad result if the job is not found in the queue.

Try to remove a queued job from the job queue. Return True, if the job could be removed from the queue before being handed over to execution, False if the job already started, and a Bad result if the job is unknown.

Change the priority of a queued job (once the job is handed over to execution, the job itself needs to be informed). To avoid the job being started unmodified, it is temporarily unqueued during the change. Return the modified job, if the job's priority was sucessfully modified, Nothing, if the job already started, and a Bad value, if the job is unkown.

Given old and new configs, determines if the changes between them should trigger the scheduler to run.