RedBeat: A Celery Beat Scheduler for Redis

RedBeat is a Celery Beat Scheduler that stores scheduled tasks and their runtime metadata in Redis. This provides a centralized, dynamic, and resilient scheduling system.

Why RedBeat?

RedBeat addresses several common challenges with the default Celery Beat scheduler:

• Dynamic Task Management: Create, modify, and delete scheduled tasks at runtime without restarting the Beat service. This is ideal for multi-tenant applications or systems where schedules change frequently.

• External Management: Because the schedule is stored in Redis, you can manage tasks from any application or language with Redis bindings, not just your Celery workers.

• Centralized & Resilient Storage: The schedule is not tied to a single machine's filesystem. You can run Beat on any server with access to the Redis instance, making your scheduling system more robust.

• Fast Startup: RedBeat loads its schedule efficiently from Redis, ensuring quick startup times even with a very large number of scheduled tasks.

• High Availability: A built-in distributed lock prevents multiple Beat instances from running simultaneously, allowing you to run redundant Beat servers for high availability. If one instance fails, another can safely take over.

For more background on the genesis of RedBeat, see this blog post.