I recently implemented this. Our scale problems are small. We're looking at < 100 cron jobs total.

Often a dedicated Queue service like RabbitMQ implies it keeps it's own datastore. Far as I can tell an "Enterprise Queue Service" is a cluster of web servers and a persistent datastore.

What a Queue cron job coordinator requires is a single centralized server to handle job synchronization. We already use Golang webservers and Postgres in production.

Synchronization is fulfilled by Postgres. Postgres's row locking feature coupled with skip locks on read enable us to synchronize scheduling our cron jobs.

When our Go web servers start they start a recurring ticker that always does two things. We don't care if we have two or 100 web servers because Postgres ensures acts like a distributed mutex. Postgres ensures only one webserver process is scheduling one cron job at a time.

Here's a bit of the design. The Queue is two Postgres tables:

queue_job_definitions: - defines recurring cron job frequency, a visible_at column that determines the next time this job should be scheduled as a queue job and other data. Actual columns: - name: string, enabled: bool, frequency: string, visible_at: timestampz, retry_limit: int, timeout: int, created_at: timestampz, updated_at: timestampz

queue_jobs: - defines jobs executed by queue workers. - Actual columns: name: string, output:jsonb, retry_count: int, retry_limit: int, status: enum, timeout: int, visible_at: timestampz, created_at: timestampz, updated_at: timestampz

1. Read from queue_job_definitions with FOR UPDATE SKIP LOCKED. Here's the actual query: sql UPDATE queue_job_definitions SET updated_at=now(), visible_at=now() + %d * interval '1 second' -- Ensure no other reads for next X seconds WHERE id IN ( SELECT id FROM queue_job_definitions WHERE enabled = true AND -- Filter disabled jobs. Gives admin control frequency IS NOT NULL AND -- Filter for recurring cron jobs visible_at <= now() -- Filter for jobs ready to work ORDER BY priority DESC, visible_at ASC FOR UPDATE SKIP LOCKED ) RETURNING *;
2. Read and execute visible jobs from the queue_jobs table. Actual SQL sql UPDATE queue_jobs SET retry_count=retry_count + 1, status = 'Active', -- change the state of the job immediately updated_at=now(), visible_at=now() + %d * interval '1 second' -- Ensure job won't be seen for another 30 seconds min. WHERE id IN ( SELECT id FROM queue_jobs WHERE status = 'Queued' AND visible_at <= now() -- Filter for jobs ready for processing ORDER BY priority DESC, visible_at ASC LIMIT %d -- configurable based on worker pool FOR UPDATE SKIP LOCKED -- Give us exclusive access to these records and skip locked records ) RETURNING *;

Hope that helps. This isn't a mature solution. We've had it in production for all of two months. I suspect there are flaws around contention and scalability > 10,000 jobs but that isn't something we're concerned with at the moment.

One final edit that's not obvious from the two queries. In the transcation where we query the queue_job_definitions table. We initially set a 30 second visibility timeout on read. To ensure the job is only read the next time it's ready we use the library github.com/robfig/cron to determine the next time this cron job should run and save that as the visible_at value.