IPC Notify Adapter

Wake workers in other processes on the same host over Unix sockets or named pipes, with no broker to run.

createIpcNotifyAdapter delivers job notifications between processes on one host, such as a web process and its worker processes sharing one SQLite database file. There is no broker to run: the processes find each other through registry files in a shared directory. Job state stays in your state adapter.

Experimental

@queuert/ipc is experimental: its API may change in a minor release.

Install

Terminal

npm install @queuert/ipc

Requires Node.js 24 or later.

Setup

The adapter is a NotifyAdapter in the "sync" mode: every method returns directly. Pass it to createClient over the synchronous SQLite adapter the processes share. Give every process the same directory:

src/index.ts

// Create the state adapter
const db = drivers.adopt(new Database(databasePath), (database) => {
  database.close();
});
db.pragma("journal_mode = WAL");
db.pragma("busy_timeout = 5000");
using stateAdapter = createSqliteStateAdapter({
  stateProvider: createBetterSqlite3StateProvider({ db }),
});
stateAdapter.migrateToLatest();

// Create the notify adapter: every process passing the same directory reaches the others
using notifyAdapter = createIpcNotifyAdapter({
  directory: ipcDirectory,
  onError: (error) => {
    console.error("IPC notify adapter error", error);
  },
});

const client = createClient({ stateAdapter, notifyAdapter, jobTypes });

The adapter is held with using, so it closes when the scope ends, before the state adapter created above it, in the shutdown order.

Configuration

createIpcNotifyAdapter options

  • directory required
    string
    Directory for the registry files. Every process that passes the same directory receives every other one's notifications. Created on first use if it does not exist. The sockets themselves live in the operating system's temporary directory (os.tmpdir()), so the processes must also share that: not with systemd's PrivateTmp, and in containers that share only this directory, mount a shared temporary directory as well. On Windows they use named pipes instead.
  • (error: unknown) => void
    Called when the server started by a listen call cannot listen or its registry file cannot be written. The next listen call tries again.

Conformance

The example's src/adapter.spec.ts registers one test per case of notifyAdapterConformanceCases and is a working template. To run it under another test runner, see Custom Notify Adapter.

src/adapter.spec.ts

let directory: string;
beforeAll(async () => {
  directory = await mkdtemp(join(tmpdir(), "queuert-notify-ipc-spec-"));
});
afterAll(async () => {
  await rm(directory, { recursive: true, force: true });
});

const context: NotifyAdapterConformanceContext = {
  createNotifyAdapter: async () => createIpcNotifyAdapter({ directory }),
};

for (const conformanceCase of notifyAdapterConformanceCases) {
  test(conformanceCase.title, async () => {
    await conformanceCase.run(context);
  });
}

Delivery

The first listen* call starts a server on a Unix socket in the operating system's temporary directory (a named pipe on Windows) in the background and writes a registry file into the directory once it listens. The first notify* call reads the registry files, rereads them every second, and writes each notification to every other registered process, which filters it locally. Listeners in the same adapter receive notifications directly, during the notify* call.

Notifications are hints. One sent while a process starts or restarts, or before the sender has reread the registry after a new process registered, is lost, and the worker's next poll picks the job up. So is one sent while 64 KiB of earlier notifications to the same process are still unsent, which keeps a process that reads slowly from growing the sender's memory. Socket errors are never thrown: the adapter drops the connection and reconnects on the next send.

Examples

@queuert/ipc

esc

Search guides, integrations and every exported symbol.

↑ ↓ to navigate↵ to open