Worker daemon fairness

In order to improve the user experience of our applications, each job that is sent to the worker daemon is assigned a priority.

To determine the priority we have the following criteria:

Decision criteria

Decision

Chained jobs

If another job belonging to the same workfow is already in the queue it’s priority is increased.

User priority

Each workflow is configured with a priority based on how quickly a user expects it to be processed.

The following levels exist, from highest to lowest:

HIGH: a user interaction triggered this process (UI change is triggered by this process) ex. series_evaluation, media_evaluation
NORMAL: propagation
LOW: ingest and processes triggered by external systems
BACKGROUND: migrations and timers

Organization fairness INTRODUCED IN 222

Organizations are prioritized in a round-robin based fashion.

For example, if a job was processed for organization A, the priority for jobs belonging to org B are increased.

The upside for this is that organizations with higher loads don’t suppress jobs of other organisations