catqueue

catqueue

catqueue is a PostgreSQL-native job queue designed for Node.js that operates without the need for an external Redis broker. If PostgreSQL is already part of the stack, catqueue provides a streamlined solution utilizing just one table, a single migration, and three primary methods to manage jobs effectively.

Key Features

• Database-Only Solution: Catqueue requires only PostgreSQL, eliminating additional dependencies.
• Built-In Idempotency: Automatically prevent duplicate jobs with built-in idempotency key constraints at the database level.
• Comprehensive Error Handling: Effortlessly track job failures with detailed error logging stored in a JSON array format within PostgreSQL.
• Batch Concurrent Processing: Supports batch processing, allowing multiple jobs to run concurrently, optimizing job execution in scenarios with I/O-bound tasks.
• Automatic Cleanup: A weekly cron job automatically purges completed jobs older than 7 days, keeping the database manageable without user intervention.
• TypeScript Compatibility: Offers full TypeScript support with generic types for typed payloads.

Quick Start Example

Here's how to quickly get started with catqueue in your application:

import { CatQueue } from "catqueue";

const queue = new CatQueue({
    connectionString: process.env.DATABASE_URL!,
});

// Register handlers
queue.register("send-email", async (payload) => {
    await mailer.send({ to: payload.to, subject: payload.subject });
});

// Start worker
queue.start();

// Enqueue a job
const jobId = await queue.enqueue("send-email", {
    to: "user@example.com",
    subject: "Welcome!",
});

Job Lifecycle

The lifecycle of a job within catqueue follows a structured path:
PENDING → PROCESSING → COMPLETED
In the event of a failure, jobs can be retried with exponential backoff until they reach a dead status after exceeding the maximum retry attempts.

When to Choose catqueue

Opt for catqueue when:

• PostgreSQL is already in use and eliminating another dependency is preferred.
• Durable and queryable job history is required for compliance and auditing purposes.
• Idempotent jobs are a necessity, with enforced constraints at the database level.
• The expected job volume is under 500 jobs per minute with a simple stack in mind.

When to Choose BullMQ

Consider BullMQ if the use case demands:

• A higher throughput of over 10,000 jobs per minute.
• Redis is part of the existing stack and low-latency job pickup is crucial.
• Advanced features such as rate limiting and job flows are required.

For further information, including benchmark comparisons and a complete API reference, consult the extensive documentation within the repository.
Catqueue streamlines job management and enhances your Node.js applications with a simple yet powerful job queue solution.