OpenTelemetry
OpenTelemetry support is currently only available for the Python SDK.
Hatchet supports exporting traces from your workflows to an OpenTelemetry Collector (opens in a new tab) to improve visibility into your Hatchet tasks.
Usage
Setup
Hatchet's SDK provides an instrumentor that auto-instruments Hatchet code, if you opt in. Setup is straightforward:
First, install the otel
extra with (e.g.) pip install hatchet-sdk[otel]
. Then, import the instrumentor:
from path.to.your.trace.provider import trace_provider
from hatchet_sdk.opentelemetry.instrumentor import HatchetInstrumentor
HatchetInstrumentor(tracer_provider=trace_provider).instrument()
You bring your own trace provider and plug it into the HatchetInstrumentor
, call instrument
, and that's it!
Check out the OpenTelemetry documentation (opens in a new tab) for more information on how to set up a trace provider.
Providing a traceparent
In some cases, you might also want to provide a traceparent
so any spans created in Hatchet are children of a parent that was created elsewhere in your application. You can do that by providing a traceparent
key in the additional_metadata
field of the following methods:
hatchet.event.push
hatchet.event.bulk_push
hatchet.admin.run_workflow
hatchet.admin.run_workflows
hatchet.admin.run_workflow_async
hatchet.admin.run_workflows_async
For example:
hatchet.event.push(
"user:create",
{'userId': '1234'},
options={
"additional_metadata": {
"traceparent":"00-f1aff5c5ea45185eff2a06fd5c0ed6c5-6f4116aff54d54d1-01" ## example traceparent
}
})
The HatchetInstrumentor
also has some methods for generating traceparents that might be helpful:
create_traceparent
creates atraceparent
inject_traceparent_into_metadata
injects a traceparent into theadditional_metadata
field
Spans
By default, Hatchet creates spans at the following points in the lifecycle of a workflow run:
- When a trigger is run on the client side, e.g.
run_workflow()
orpush()
is called. - When a worker handles a task event, like starting running the task or cancelling the task
In addition, you'll get a handful of attributes set (prefixed by hatchet.
) on the step run events, such as the workflow name and the worker id, as well as success / failure states, and so on.