Job Processing

Overview

This document describes how Queuert processes jobs: transactional design, prepare/complete pattern, and timeout philosophy.

Transactional Design

Queuert’s core design principle is that jobs are created inside the same database transaction as your application state changes. This follows the transactional outbox pattern:

await withTransactionHooks(async (transactionHooks) =>
  db.transaction(async (tx) => {
    // Application state change
    const image = await tx.images.create({ ... });

    // Job creation in the same transaction
    // The transaction context property name matches your StateProvider
    await client.createChain({
      tx,
      transactionHooks,
      typeName: "process-image",
      input: { imageId: image.id },
    });
  }),
);

Why This Matters

Atomicity: If the transaction rolls back, the job is never created. No orphaned jobs.
Consistency: The job always references valid application state.
No dual-write problem: You don’t need to coordinate between your database and a separate job queue.

Extending to Job Processing

The same transactional principle extends to job processing through the prepare/complete pattern:

Prepare phase: Read application state within a transaction
Processing phase: Perform side-effects (API calls, file operations) outside the transaction
Complete phase: Write results back within a transaction

This ensures that job outputs and continuations are also created atomically with any state changes they produce.

Observability events (metrics, span ends, logs) emitted during the prepare and complete phases are transactional — they are buffered and only flushed after the transaction commits. If the transaction rolls back, no observability events leak out.

Prepare/Complete Pattern

Attempt handlers split processing into phases: prepare reads the data the attempt needs, the handler does its work in between, and complete writes exactly one outcome — an output that ends the chain, a continuation that extends it with a successor job, or a reschedule that returns the job to pending (finish({ reschedule })). The split is what lets an attempt run atomically in a single transaction or stage long-running work between two.

Auto-Setup (Default)

Most jobs don’t need prepare. Call complete directly and auto-setup infers the mode:

Synchronous complete (called immediately, no prior await): atomic mode — single transaction wraps everything
Async work before complete: staged mode – heartbeat active between async work and complete
Accessing prepare after auto-setup throws: “prepare cannot be accessed after auto-setup”

See Processing Modes for examples and a decision flowchart.

Explicit Modes

For more control, call prepare explicitly:

Atomic mode: Prepare and complete run in the same transaction. Rarely needed since calling complete directly achieves the same result with less ceremony.
Staged mode: Prepare runs in one transaction, long-running work happens outside, then complete runs in another transaction. The worker automatically extends the attempt between phases. Implement the processing phase idempotently as it may retry if the worker crashes.

Intermediate Transactions with `step`

Handlers can call step to perform intermediate transactional work — each call opens a fresh guarded transaction with attempt verification. This is useful for batched operations like bulk deletes where holding a single long-lived transaction would be impractical. If prepare hasn’t been called, step automatically enters staged mode. step is not reachable once complete has started — work that must observe the terminal transition belongs inside the complete transaction. See Processing Modes — Step.

Error Recovery and Savepoints

Both the prepare and complete callbacks run inside database savepoints. This is the mechanism that keeps jobs safe when user code throws.

Why Savepoints

A naive approach would run user callbacks directly inside the job’s transaction. The problem: if user code throws after executing partial SQL, the transaction is poisoned — most databases reject further statements on a transaction that has seen an error. The engine couldn’t even reschedule the job because the reschedule SQL would fail on the same broken transaction.

Savepoints solve this. A savepoint is a checkpoint within a transaction. If code inside the savepoint throws, the database rolls back to that checkpoint — undoing the partial work — while the outer transaction remains healthy. The engine can then reschedule the job and commit normally.

How It Works

transaction · acquires the job

savepoint · prepare callback

your SQLthrows? roll back to the savepoint

async work · staged mode onlythe attempt auto-extends between transactions

transaction · completes the job

savepoint · complete callback

your SQLcommit writes the outcome throws? roll back to the savepoint

success

commit

error

rescheduleexponential backoff

On any unhandled error the job is rescheduled with exponential backoff (default: 10 s → 20 s → 40 s → … capped at 300 s). There is no maximum retry count — jobs retry indefinitely. Use discriminated unions or compensation patterns to handle permanently failing jobs.

See Job Processing Reliability for per-phase error scenarios with code examples.

Abort Signal and Reasons

Every attempt handler receives a signal (TypedAbortSignal<JobAbortReason>) that is aborted when the job should stop. The signal.reason distinguishes the cause:

Reason	Meaning
`"taken_by_another_worker"`	Another worker acquired the job (attempt expired and was released)
`"not_found"`	The job no longer exists in the database
`"already_completed"`	The job was already completed
`"error"`	Renewing the attempt (heartbeat) failed unexpectedly — ownership is uncertain, not confirmed lost
`"worker_stopping"`	The worker is shutting down — the job is still valid and the attempt is still held

The first three reasons indicate the job is no longer owned by this worker, and "error" means ownership can no longer be confirmed — in all four cases handlers should stop immediately. "worker_stopping" is different: the job remains valid, but the worker is draining, so handlers should check signal.reason and wrap up quickly. Every attempt must still end in an outcome: call complete (which may finish({ reschedule })), or throw so the attempt is rescheduled with backoff. A handler that simply returns without completing fails the attempt.

attemptHandler: async ({ signal, complete }) => {
  for (const item of items) {
    if (signal.aborted && signal.reason === "worker_stopping") {
      break; // finish early, complete with partial results
    }
    await processItem(item);
  }
  return complete(async ({ finish }) => finish({ output: { processed: items.length } }));
},