Skip to Content
BlockQueue
Overview

bq-queue

We built bq-queue because our apps run as multiple Docker replicas, and in-process cron is a trap in that world. Three replicas mean three “nightly” jobs. Sleeping until publish time inside a request is worse. Billing reminders, digests, delayed publishing, and webhook retries kept showing up as the same problem in different codebases.

So scheduling and delivery live in one place. Your app declares queues in YAML (one file per project). The queue service stores the job in PostgreSQL via pg-boss , waits until the right time, then POSTs your HTTP endpoint with a signed payload. Retries, concurrency, and retention are config, not another library in every service.

Why it exists

  • Multi-replica Docker: cron inside the app fires on every instance.
  • The same timing needs appear across products: notifications, billing, digests, delayed publish.
  • One reusable queue service plus per-project YAML beats reimplementing schedules everywhere.

How you use it

  1. Add a YAML file for your project: queue names, endpoint URLs, signature_secret, optional static cron_jobs.
  2. Deploy queue-service with that config. Your app exposes the endpoint(s).
  3. Static recurring work (daily digest, nightly cleanup): put cron_jobs in YAML. No API call each day.
  4. Dynamic / one-shot work (publish this post at 3pm, send one reminder): your app POSTs /api/jobs/one-off (optional runAt) or /api/jobs/schedule. At the right time, queue-service calls you back.

Example: a CMS schedules “publish post 42 at 15:00 UTC”. The app does not sleep and does not trust a replica’s clock. It enqueues a one-off with runAt. At 15:00 the queue hits your POST_PUBLISHING endpoint; your handler publishes and returns 200.

How it works (mechanics)

  1. Your app signs a JSON body with REQUEST_SIGNING_SECRET and calls /api/jobs/*.
  2. queue-service stores the job in pg-boss.
  3. A worker POSTs to the queue’s endpoint with a body signed by that queue’s signature_secret.
  4. Your handler verifies the queue signature, does the work, and returns 2xx. Non-2xx triggers retry.

You can run it on a private network or behind a public edge. TLS belongs at the reverse proxy. Auth for job APIs and worker deliveries is HMAC signing.

Guides

Source

github.com/blockqueue/bq-queue 

Last updated on
Getting started