Manual Slot Release

The Hatchet execution model sets a number of available slots for running steps in a workflow. When a step is running, it occupies a slot, and if a worker has no available slots, it will not be able to run any more steps concurrently.

In some cases, you may have a step in your workflow that is resource-intensive and requires exclusive access to a shared resource, such as a database connection or a GPU compute instance. To ensure that other steps in the workflow can run concurrently, you can manually release the slot after the resource-intensive step has completed, but the step still has non-resource-intensive work to do (i.e. upload or cleanup).

Using Manual Slot Release

You can manually release a slot in from within a running step in your Workflow using the Hatchet context param:

@hatchet.workflow(on_events=["user:create"])
    @hatchet.step()
    def step1(self, context: Context):
        print('RESOURCE INTENSIVE PROCESS')
        time.sleep(10)
        # Release the slot after the resource-intensive process, so that other steps can run
        context.release_slot()
        print("NON RESOURCE INTENSIVE PROCESS")
        return {"status": "success"}

In the above examples, the release_slot() method is called after the resource-intensive process has completed. This allows other steps in the workflow to start executing while the current step continues with non-resource-intensive tasks.

Use Cases

Some common use cases for Manual Slot Release include:

• Performing data processing or analysis that requires significant CPU, GPU, or memory resources
• Acquiring locks or semaphores to access shared resources
• Executing long-running tasks that don't need to block other steps after some initial work is done

By utilizing Manual Slot Release, you can optimize the concurrency and resource utilization of your workflows, allowing multiple steps to run in parallel when possible.