Skip to content

OTEL Metrics

Queuert emits OpenTelemetry metrics through the @queuert/otel adapter. Users must configure their OTEL SDK with desired exporters (Prometheus, OTLP, etc.) before using the adapter. See the ObservabilityAdapter TSDoc for the adapter interface.

All metrics follow the naming pattern:

queuert.{component}.{operation}[.{suboperation}]

The ObservabilityAdapter interface accepts milliseconds; the @queuert/otel adapter converts duration values to seconds per OTEL Messaging Semantic Conventions.

MetricAttributesDescription
queuert.worker.startedworkerIdWorker started processing
queuert.worker.errorworkerIdWorker encountered an error
queuert.worker.stoppingworkerIdWorker received stop signal
queuert.worker.stoppedworkerIdWorker fully stopped
MetricAttributesDescription
queuert.job.createdtypeName, chainTypeNameJob created
queuert.job.completedtypeName, chainTypeName, workerId, continuedJob completed. workerId is "null" for workerless completion. continued is "true"/"false"
queuert.job.reapedtypeName, chainTypeName, workerIdStale job reclaimed by reaper
queuert.job.blockedtypeName, chainTypeNameJob blocked by pending blocker chains
queuert.job.triggeredtypeName, chainTypeNamePending job triggered to run immediately
queuert.job.unblockedtypeName, chainTypeNameJob unblocked after blocker chain completed
MetricAttributesDescription
queuert.job.attempt.startedtypeName, chainTypeName, workerIdWorker began processing an attempt
queuert.job.attempt.completedtypeName, chainTypeName, workerIdAttempt completed successfully
queuert.job.attempt.failedtypeName, chainTypeName, workerIdAttempt failed (may retry)
queuert.job.attempt.taken_by_another_workertypeName, chainTypeName, workerIdJob already leased by another worker
queuert.job.attempt.already_completedtypeName, chainTypeName, workerIdJob already completed when worker tried to process it
queuert.job.attempt.lease_expiredtypeName, chainTypeName, workerIdLease expired before attempt finished
queuert.job.attempt.lease_renewedtypeName, chainTypeName, workerIdLease successfully renewed during processing
MetricAttributesDescription
queuert.job_chain.createdchainTypeNameJob chain created
queuert.job_chain.completedchainTypeNameJob chain completed
queuert.job_chain.deletedchainTypeNameJob chain deleted
MetricAttributesDescription
queuert.state_adapter.erroroperationState adapter operation failed
queuert.notify_adapter.erroroperationNotify adapter operation failed

Histograms track duration distributions at three levels. Unit is seconds.

MetricAttributesDescription
queuert.job_chain.durationchainTypeNameDuration from chain creation to completion
queuert.job.durationtypeName, chainTypeNameDuration from job creation to completion
queuert.job.attempt.durationtypeName, chainTypeName, workerIdDuration of a single attempt

These form a hierarchy — chain duration encompasses job durations (plus wait time between continuations), and job duration encompasses attempt durations (plus wait time between retries):

queuert.job_chain.duration
├── queuert.job.duration (first job)
│ ├── queuert.job.attempt.duration (attempt 1)
│ └── queuert.job.attempt.duration (attempt 2, retry)
├── queuert.job.duration (continuation)
│ └── queuert.job.attempt.duration
└── (wait time between jobs)

Two gauges track real-time worker state. They are incremented/decremented via delta values.

MetricAttributesDescription
queuert.job_type.idletypeName, workerIdWorkers currently idle for this job type
queuert.job_type.processingtypeName, workerIdJobs of this type currently being processed