Comprehensive Celery Integration Enhancement in FastAPI Boilerplate

[marcosmamorim]

Comprehensive Celery Integration Enhancement in FastAPI Boilerplate

Overview

This PR introduces a significant enhancement to the Celery integration mechanism within the FastAPI Boilerplate project. The changes aim to streamline task management and improve the efficiency of background task processing by sharing database session configurations between FastAPI and Celery components.

Key Changes

• CeleryConfigurator Class: A new class that centralizes all Celery configurations, facilitating easier maintenance and updates.
• Dynamic Queue Configuration: Automatically configures Celery queues based on the project's module structure. This dynamic setup scans for tasks.py files within each module's app/*/application/celery directory, creating a dedicated queue for each module found. This approach eliminates the need for manual queue setup and enhances scalability.
• Shared Session Management: Integrates database session management between FastAPI and Celery, ensuring that both components handle sessions in a unified manner. This reduces the overhead associated with session management and minimizes potential for errors.

Benefits

• Improved Scalability: As the project grows, the new setup automatically adapts, adding necessary task queues without manual configuration.
• Enhanced Maintainability: Centralizing configuration reduces complexity and makes the system easier to understand and modify.
• Consistency Across Components: By sharing session management logic, the system behaves consistently whether tasks are executed synchronously or asynchronously.

Usage Example

With the new setup, adding a new module with background tasks only requires adding a tasks.py file in the correct directory. The system automatically handles the rest, setting up queues and ensuring the tasks are correctly routed and executed.