Sticky Worker Assignment (Beta)

This feature is currently in beta and may be subject to change.

Sticky assignment is a task property that allows you to specify that all child tasks should be assigned to the same worker for the duration of its execution. This can be useful in situations like when you need to maintain expensive local memory state across multiple tasks in a workflow or ensure that certain tasks are processed by the same worker for consistency.

⚠️

This feature is only compatible with long lived workers, and not webhook workers.

Setting Sticky Assignment

Sticky assignment is set on the task level by adding the sticky property to the task definition. When a task is marked as sticky, all steps within that task will be assigned to the same worker for the duration of the task execution.

⚠️

While sticky assignment can be useful in certain scenarios, it can also introduce potential bottlenecks if the assigned worker becomes unavailable, or if local state is not maintained when the job is picked up. Be sure to consider the implications of sticky assignment when designing your tasks and have a plan in place to handle local state issues.

There are two strategies for setting sticky assignment for DAG workflows:

SOFT: All tasks in the workflow will attempt to be assigned to the same worker, but if that worker is unavailable, it will be assigned to another worker.
HARD: All taks in the workflow will only be assigned to the same worker. If that worker is unavailable, the workflow run will not be assigned to another worker and will remain in a pending state until the original worker becomes available or timeout is reached. (See Scheduling Timeouts)

examples/sticky_workers/worker.py



sticky_workflow = hatchet.workflow(
    name="StickyWorkflow",
    sticky=StickyStrategy.SOFT,
)


@sticky_workflow.task()
def step1a(input: EmptyModel, ctx: Context) -> dict[str, str | None]:
    return {"worker": ctx.worker.id()}


@sticky_workflow.task()
def step1b(input: EmptyModel, ctx: Context) -> dict[str, str | None]:
    return {"worker": ctx.worker.id()}

src/v1/examples/sticky/workflow.ts

export const sticky = hatchet.task({
  name: 'sticky',
  retries: 3,
  sticky: StickyStrategy.SOFT,
  fn: async (_, ctx) => {
    // specify a child workflow to run on the same worker
    const result = await ctx.runChild(
      child,
      {
        N: 1,
      },
      { sticky: true }
    );

    return {
      result,
    };
  },
});

examples/assignment-sticky/run.go


err = w.RegisterWorkflow(
	&worker.WorkflowJob{
		On:          worker.Events("user:create:sticky"),
		Name:        "sticky",
		Description: "sticky",
		StickyStrategy: types.StickyStrategyPtr(types.StickyStrategy_HARD),
		Steps: []*worker.WorkflowStep{
			worker.Fn(func(ctx worker.HatchetContext) (result *stepOneOutput, err error) {

				sticky := true

				_, err = ctx.SpawnWorkflow("sticky-child", nil, &worker.SpawnWorkflowOpts{
					Sticky: &sticky,
				})

				if err != nil {
					return nil, fmt.Errorf("error spawning workflow: %w", err)
				}

				return &stepOneOutput{
					Message: ctx.Worker().ID(),
				}, nil
			}).SetName("step-one"),
			worker.Fn(func(ctx worker.HatchetContext) (result *stepOneOutput, err error) {
				return &stepOneOutput{
					Message: ctx.Worker().ID(),
				}, nil
			}).SetName("step-two").AddParents("step-one"),
			worker.Fn(func(ctx worker.HatchetContext) (result *stepOneOutput, err error) {
				return &stepOneOutput{
					Message: ctx.Worker().ID(),
				}, nil
			}).SetName("step-three").AddParents("step-two"),
		},
	},
)

In this example, the sticky property is set to SOFT, which means that the task will attempt to be assigned to the same worker for the duration of its execution. If the original worker is unavailable, the task will be assigned to another worker.

Sticky Child Tasks

It is possible to spawn child tasks on the same worker as the parent task by setting the sticky property to true in the run method options. This can be useful when you need to maintain local state across multiple tasks or ensure that child tasks are processed by the same worker for consistency.

However, the child task must:

Specify a sticky strategy in the child task’s definition
Be registered with the same worker as the parent task

If either condition is not met, an error will be thrown when the child task is spawned.