Nicolas Le Manchet

Django database connections in background workers

Executing of a task in a background worker is similar to handling a request in a WSGI server. At the beginning the task or request must fetch a database connection and at the end it must clean it up.

Django uses signals sent at the beginning and end of a request to manage database connections. This means that task queues can do the same and be confident that background tasks get the same isolation as HTTP requests.

In Spinach this translates to:

from django.db import reset_queries, close_old_connections
from spinach import signals

@signals.job_started.connect
def job_started(*args, job=None, **kwargs):
    reset_queries()
    close_old_connections()

@signals.job_finished.connect
def job_finished(*args, job=None, **kwargs):
    close_old_connections()