opcotech/elemo

# Background Task Scheduling

| author       | created at | updated at | status   |
|:-------------|:-----------|------------|:---------|
| @gabor-boros | 2023-03-19 | -          | accepted |

## Abstract

Many tasks and actions need to be scheduled in the background for later
processing. The project will schedule numerous tasks in the background. A few
examples are:

- Sending emails
- Generating reports
- Running backups
- Running maintenance tasks

Since the task processing should be decoupled from the request/response cycle,
the project will use a background task scheduler to schedule and execute tasks.

The designed solution must be able to process tasks separately from the main
application. The solution must be able to handle tasks that take a long time
to complete and preferably be able to handle tasks that fail.

## Decision

To implement background tasks, we are going to use [Asynq], a simple, reliable,
and scalable background task processing system.

The scheduling and task processing is implemented in two parts: the client and
workers.The scheduler needs a message broker to communicate with the workers,
therefore we are going to use [Redis] as the message broker.

Since Redis is a performant in-memory database, it is a good fit for the
scheduler. Redis is also a good fit for the project, because it could also be
used as a cache for the application, so it fits well into the overall project
architecture.

The "client" will be implemented in the application's service and repository
layers. The client will be responsible for scheduling tasks and sending them to
the message broker. The client will be implemented as a thin wrapper around
Asynq's client.

The "worker" will be implemented as a separate service, which will be started
independently of the main application. Hence, it will be possible to scale up
or down the number of workers as needed. The workers will have a single
responsibility: to process tasks.

[Asynq]: https://github.com/hibiken/asynq

[Redis]: https://redis.io/

## Consequences

One of the many advantages of this solution is having a simple and reliable
workflow for scheduling and processing tasks. The solution is also scalable,
because it is possible to scale up or down the number of workers as needed.

Furthermore, the solution is easy to implement and maintain, and builds on
top of existing technologies. We don't need to reinvent the wheel, and we can
use Redis which is already part of the long-term architecture of the project.

The only disadvantage is that we have no other options than Redis as the
message broker, however, Redis is a good fit for the project.

## References

None.