Getting logging to work with Django + Celery + Sentry

How properly to set up Django + Celery + Sentry logging

Celery + Sentry logging is a mess

One of the most frustrating steps in setting up a Django + Celery + Sentry logging in a way that

  • doesn’t break your existing Django + Sentry configuration and
  • sends log messages from Celery, with tracebacks, to Sentry and
  • doesn’t send messages from Celery twice.

If you look at the Sentry, Raven (a Sentry client), and Celery bugtrackers, there are hundreds of issues on the topic. Information is scattered over dozens of threads, with solutions ranging from changing some configuration, over creating a custom logging handler, to catching some event signal.

All of this while having special support for Celery in Sentry/Raven.

The issues seem to stem from the fact that Django, Raven, and Celery all make complex changes to the Python runtime: They mess with the logging configuration, and modify tracebacks so they’re nicer-to-read. While these changes are of great convenience it looks like they don’t work together very well.

How to get it working

What has worked for us is to disable Celery’s logging modifications altogether as described in this GitHub issue.

Add this to our your Django project’s celery.py (or some other piece of code that gets executed early in the Django boot sequence):

from celery import Celery

# See https://github.com/getsentry/raven-python/issues/922
@signals.setup_logging.connect
def setup_logging(**kwargs):
    """Skip Celery logging modifications."""

This seems to be the recommended solution at the time of writing. We’ll keep this tutorial up-to-date when things change in either of the projects.