Using Django Tasks in production

The Djangonaut Space website has been using the Django Tasks framework and django-tasks-db in production successfully for about six months now. The integration has been straightforward, and there’s clearly been some lessons learned from the community’s efforts in other background task processors.

Installation

We use the database backend because the application is small and we’re using tasks to send emails and run some one-off, long-running events.

Currently, the Django documentation doesn’t explain how to install everything you need to use tasks. This is because a third-party package is required to actually run the tasks.

Note: If you’re using a version of Django earlier than 6.0, you’ll need to use the django-tasks backport.

As per the django-tasks-db installation docs, we must install the dependency:

python3 -m pip install django-tasks-db

Then make a few changes to our settings:

INSTALLED_APPS = [
    # ...
    "django_tasks_db",
]

TASKS = {
    "default": {
        "BACKEND": "django_tasks_db.DatabaseBackend",
    }
}

At this point we should be able to run the worker and see tasks run.

python3 -m manage db_worker

Using tasks

Now let’s look at how to use tasks. A common use case in web applications is sending emails in the background without blocking the request cycle. Let’s look at a concrete example of how Djangonaut Space does this.

We collect testimonials from the folks who participate in a session and post them on the site. This process involves a period of review to moderate content to make sure it abides by our code of conduct.

The requirement is simple. When a testimonial is created, notify the admins via email.

This post includes the simplified version of the code running in production. You can view Djangonaut Space’s source code here.

# views.py

from django.contrib import messages

from .forms import TestimonialForm
from .models import Testimonial
from .tasks import send_testimonial_notification

class TestimonialCreateView(LoginRequiredMixin, CreateView):
    """Create a new testimonial."""
    model = Testimonial
    form_class = TestimonialForm

    def form_valid(self, form: TestimonialForm) -> HttpResponse:
        """Set author and trigger notification."""
        form.instance.author = self.request.user
        response = super().form_valid(form)

        send_testimonial_notification.enqueue(
            testimonial_id=self.object.pk,
            is_new=True,
        )

        messages.success(
            self.request,
            _(
                "Your testimonial has been submitted and is pending review. "
                "Thank you for sharing your experience!"
			      ),
        )
        return response

The task being scheduled is as followed:

# tasks.py

from django.contrib.auth import get_user_model
from django.tasks import task
from django.utils.translation import gettext_lazy as _

from . import email
from .models import Testimonial

User = get_user_model()


@task()
def send_testimonial_notification(
    testimonial_id: int,
    is_new: bool,
    old_values: dict | None = None,
) -> None:
    """
    Send a notification email to superusers about a new or updated testimonial.

	Args:
		testimonial_id: The ID of the Testimonial
		is_new: True if this is a new testimonial, False if it's an update
		old_values: Dictionary of old values for comparison (for updates only)
		            Contains keys: title, text, session_id
	"""
    superuser_emails = list(
        User.objects.filter(is_superuser=True, is_active=True).values_list(
            "email", flat=True
        )
    )
    if not superuser_emails:
        return

    testimonial = Testimonial.objects.select_related("author", "session").get(
        pk=testimonial_id
    )
    # Generate context
    context = {
        # ...
    }

    email.send(
        email_template="testimonial_notification",
        recipient_list=superuser_emails,
        context=context,
    )

One thing to note: with the database backend, you can monitor the progress of your tasks in the admin. You can see which tasks are scheduled, completed, and have errored. You don’t need a separate monitoring application like you would with Celery.

What I’ve enjoyed using Django Tasks

It feels like we’ve learned lessons from other background task processors and have incorporated them here. The interface that Django Tasks landed on is sound. It works well, and that’s what the community needs.

I’m also enjoying using the database as a task backend. The djangonaut.space site is hosted on a small VPS. Being able to eliminate having to run another process to share resources is beneficial. The user requirements of the site are limited and are unlikely to scale unexpectedly, meaning this approach is an excellent solution for us. I appreciate Jake Howard for providing this out of the box, so thank you, Jake!

Things I’d like to see, but I’m too lazy to implement myself

We know that Django Tasks is just a beginning; Jake pointed this out in his Django Chat episode. So I’ve got a few ideas of tools that would be beneficial for our community to implement.

1. A new tutorial section in the Django documentation that shows off tasks
2. A Django Debug Toolbar panel to show execution and debugging information of tasks
3. A test/mock backend that can programmatically control the flow of tasks in tests

The Django docs currently lack an example of start to finish for using tasks, which is fine. It’s a new feature. If someone is looking for some low-hanging fruit, this is your sign.

The Django Debug Toolbar has plans to build this out but has stalled in implementation. The benefit of a consistent API for tasks allows us to leverage that for other purposes. Like observability!

When testing with tasks, you are left with either a DummyBackend that only stores the task queueing, an ImmediateBackend which calls your logic immediately, or your actual production backend. It’s possible to write multiple tests, changing the backend between dummy and immediate, to verify the task is scheduled and the logic change is appropriate. However, there’s room for a testing backend that would do both. Collect all queued tasks and allow them to be executed, similar to ImmediateBackend on demand. This would allow a single test to confirm that the task is wired up properly while also doing the integration-level logic verification.