Redis Notify Adapter

Wake workers with Redis pub/sub on standalone Redis or Redis Cluster: node-redis, ioredis and Bun.

createRedisNotifyAdapter uses Redis pub/sub for broadcasting job notifications to workers, publishing each batch from one Lua script. The notify adapter stores no job state; job storage is handled by a separate state adapter, which can also be the experimental Redis state adapter.

Install

Terminal

npm install @queuert/redis

Requires Redis 6 or later, standalone or Redis Cluster.

Setup

Each example's notify provider and adapter:

src/provider.ts

import type { RedisNotifyProvider } from "@queuert/redis";
import type { RedisClientType } from "redis";

export const createNodeRedisNotifyProvider = ({
  client,
  subscribeClient,
}: {
  client: RedisClientType;
  subscribeClient: RedisClientType;
}): RedisNotifyProvider => ({
  subscribe: async (channel, onMessage) => {
    await subscribeClient.subscribe(channel, onMessage);
    return async () => {
      await subscribeClient.unsubscribe(channel);
    };
  },
  eval: async (script, keys, args) => {
    return client.eval(script, { keys, arguments: args });
  },
});

src/index.ts

await using drivers = new AsyncDisposableStack();

// Create Redis connections
const redis = drivers.adopt(createRedisClient({ url: redisUrl }), async (redis) => {
  await redis.quit();
});
await redis.connect();

const redisSubscription = drivers.adopt(
  createRedisClient({ url: redisUrl }),
  async (redisSubscription) => {
    await redisSubscription.quit();
  },
);
await redisSubscription.connect();

// Create the notify provider using node-redis
const notifyProvider = createNodeRedisNotifyProvider({
  client: redis,
  subscribeClient: redisSubscription,
});

// Create the notify adapter
await using notifyAdapter = createRedisNotifyAdapter({ notifyProvider });

The notify adapter and your clients are held with await using. A client that is not disposable itself is adopted by a drivers AsyncDisposableStack. Disposal runs in reverse order, so when the scope ends the notify adapter closes before your clients, in the shutdown order.

Configuration

createRedisNotifyAdapter options

Notify provider

A RedisNotifyProvider bridges your Redis client with the adapter. Each Setup tab is a complete provider for its client; to use another one, implement the same members:

RedisNotifyProvider

  • Runs a Lua script; the adapter publishes each batch of notifications from one script.
  • Listens on a channel and invokes a callback when messages arrive. Returns a dispose function to unsubscribe.
  • close optional
    Only needed when the provider owns resources. notifyAdapter.close() invokes it.

Testing your provider

redisNotifyProviderConformanceCases from @queuert/redis/conformance check a provider you write: they are every case a notify adapter must pass, run against the context createRedisNotifyProviderConformanceContext({ createNotifyProvider }) returns: each case builds its own notify adapter over a new provider from your factory and closes both when it ends. The suite drives no test runner: your test file owns the connections in its own hooks and registers one test per case. Every example's src/provider.spec.ts is a working template:

src/provider.spec.ts

describe("notify-redis-redis provider conformance", () => {
  let redis: AcquiredRedis;
  let client: RedisClientType;
  let subscribeClient: RedisClientType;
  let context: NotifyAdapterConformanceContext;

  beforeAll(async () => {
    redis = await acquireRedis("redis:8");
    client = createClient({ url: redis.connectionUrl });
    subscribeClient = createClient({ url: redis.connectionUrl });
    await client.connect();
    await subscribeClient.connect();
    context = createRedisNotifyProviderConformanceContext({
      createNotifyProvider: async () => createNodeRedisNotifyProvider({ client, subscribeClient }),
    });
  }, 60_000);

  afterAll(async () => {
    await client.close();
    await subscribeClient.close();
    await redis[Symbol.asyncDispose]();
  });

  for (const conformanceCase of redisNotifyProviderConformanceCases) {
    test(conformanceCase.title, async () => {
      await conformanceCase.run(context);
    });
  }
});

The full API is in the conformance reference.

Channels

Three channels carry notifications between processes (configurable prefix, default queuert). Channels use Redis Pub/Sub — messages are fire-and-forget with no persistence. If no subscriber is listening when a message is published, it is lost. This is acceptable because workers fall back to polling when notifications are missed.

  • {prefix}:sched job scheduled
    payload · job type name
    Published when jobs become pending. Wakes idle workers.
  • {prefix}:chainc chain completed
    payload · chain id
    Published when a chain completes. Wakes clients awaiting results.
  • {prefix}:atls attempt lost
    payload · job id
    Published when a running job's attempt is lost. Notifies attempt loss.

Connection model

Redis Pub/Sub requires two separate connections. The RedisNotifyProvider interface abstracts this — you manage the two connections in your provider implementation.

  • Command connection
    Runs EVAL for the publish script, which issues the PUBLISH calls. It cannot be in subscribe mode.
  • Subscription connection
    Runs SUBSCRIBE and UNSUBSCRIBE. In subscribe mode it cannot execute regular commands.

Examples

@queuert/redis

esc

Search guides, integrations and every exported symbol.

↑ ↓ to navigate↵ to open