Run coordinators
In production Dagster deployments, there are often many runs being launched at once. The run coordinator lets you control the policy that Dagster uses to manage the set of runs in your deployment.
When you submit a run from the Dagster UI or the Dagster command line, it’s first sent to the run coordinator, which applies any limits or prioritization policies before eventually sending it to the run launcher to be launched.
Run coordinator types
The following run coordinators can be configured on your Dagster instance:
| Term | Definition |
|---|---|
DefaultRunCoordinator | The DefaultRunCoordinator calls launch_run on the instance’s run launcher immediately in the same process, without applying any limits or prioritization rules.When this coordinator is set, clicking Launch Run in the Dagster UI will immediately launch the run from the Dagster daemon process. Similarly, scheduled runs will immediately launch from the scheduler process. |
QueuedRunCoordinator | The QueuedRunCoordinator sends runs to the Dagster daemon via a run queue. The daemon pulls runs from the queue and calls launch_run on submitted runs.Using this run coordinator enables instance-level limits on run concurrency, as well as custom run prioritization rules. |
Configuring run coordinators
If you use the DefaultRunCoordinator, no configuration is required on your part.
However, if using the QueuedRunCoordinator or building a custom implementation, you can define custom run prioritization rules and instance-level concurrency limits.
Concurrency pools and dequeue ordering
When runs are queued because a concurrency pool is at capacity, Dagster+ does not guarantee strict FIFO ordering across all runs. The dequeue order is determined by internal prioritization logic rather than strictly by queue arrival time, so jobs may not execute in the exact order they were submitted. See Managing concurrency.
If your workload depends on a specific execution order, define explicit run prioritization rules with the QueuedRunCoordinator rather than relying on queue arrival time. See Customizing run queue priority.