celery with django

Celery is a distributed task queue system that can be used with Django to perform asynchronous tasks such as sending emails, processing background jobs, and more. In this guide, I’ll walk you through the steps to integrate and use Celery with a Django project.

1. Install Celery:

You need to install Celery and a message broker (such as Redis or RabbitMQ) to use it for background task processing. For this example, I’ll use Redis as the message broker.

pip install celery[redis]

2. Create a Django Project:

If you haven’t already, create a Django project or use an existing one.

3. Configure Celery in Django:

In your Django project, create a file named celery.py in the same directory as your settings.py file (usually your project's main directory). This file will hold the Celery configuration.

# celery.py

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from django.conf import settings

# Set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'your_project.settings')

app = Celery('your_project')  # Replace 'your_project' with your project's name.

# Configure Celery using settings from Django settings.py.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load tasks from all registered Django app configs.
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

4. Configure Celery Broker (Redis):

In your Django project’s settings.py, configure Celery to use Redis as the message broker. Add the following lines to your settings.py:

# settings.py

# Celery settings
CELERY_BROKER_URL = 'redis://localhost:6379/0'

Make sure to replace the CELERY_BROKER_URL with the appropriate Redis server URL if it's running on a different host or port.

5. Create Celery Tasks:

Now, you can define your Celery tasks as regular Python functions in your Django app. Here’s an example task:

# tasks.py

from celery import shared_task

@shared_task
def my_task(arg1, arg2):
    # Task logic here
    result = arg1 + arg2
    return result

6. Use Celery Tasks:

You can now use Celery tasks in your Django views, models, or any other parts of your application. To call a task, import it and use the delay method to enqueue it for execution asynchronously:

from .tasks import my_task

result = my_task.delay(3, 5)

7. Start Celery Worker:

To run Celery and process tasks, you need to start a Celery worker process. In your project directory, run the following command:

celery -A your_project worker --loglevel=info

Replace 'your_project' with your project's name.

That’s it! You’ve successfully integrated Celery with your Django project. Now you can use it to perform background tasks asynchronously, improving the performance and responsiveness of your application.