Django Advanced #4: Caching — per-view / template fragment / low-level

If #3 Query optimization was about cutting queries, caching is about storing computed results so you don’t redo them. The two are complementary. A high cache hit rate makes ORM optimizations even more valuable, and a fast miss path is what makes a cache worth having.

Django provides caching at multiple levels. From fine-grained to coarse:

Cache backends #

Defined in settings.CACHES.

locmem — default (development) #

CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.locmem.LocMemCache",
        "LOCATION": "unique-per-process",
    }
}

In-process memory. With multiple workers, each has its own cache — unsuitable for production. Dev/test territory.

Redis — standard production backend #

Django 5.x has a built-in backend.

CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
    }
}

If you need richer features (distributed locks, etc.), django-redis is the de facto standard.

pip install django-redis

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "PARSER_CLASS": "redis.connection._HiredisParser",
            "CONNECTION_POOL_KWARGS": {"max_connections": 50},
            "SOCKET_CONNECT_TIMEOUT": 5,
            "SOCKET_TIMEOUT": 5,
            "IGNORE_EXCEPTIONS": True,   # On Redis down, return None instead of raising
        },
        "KEY_PREFIX": "myproject",
    }
}
DJANGO_REDIS_IGNORE_EXCEPTIONS = True

IGNORE_EXCEPTIONS=True is critical in production — when Redis goes down briefly, the entire site doesn’t return 500; it falls back to a cache miss.

Memcached #

CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.memcached.PyMemcacheCache",
        "LOCATION": "127.0.0.1:11211",
    }
}

Lightest and fastest. But it’s simple TTL-based KV only — for distributed locks, sorted sets, pub/sub, choose Redis.

DB / file #

CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.db.DatabaseCache",
        "LOCATION": "my_cache_table",
    }
}

After installing, run python manage.py createcachetable once. Useful when you need caching without extra infrastructure. Adds DB load, so not recommended for production.

filebased.FileBasedCache lives in the same neighborhood.

Per-view cache — @cache_page #

The simplest cache. The whole response, end to end.

from django.views.decorators.cache import cache_page

@cache_page(60 * 15)   # 15 minutes
def article_list(request):
    articles = Article.objects.published().order_by("-created_at")[:50]
    return render(request, "articles/list.html", {"articles": articles})

How it works:

• Cache key: hash of (request method, URL, query string, Accept-Language, ...)
• Miss: run view → cache response → return
• Hit: return cached response without entering the view (DB untouched)

Only GET / HEAD are cached. POST/PUT/DELETE are ignored.

Class-based views #

from django.utils.decorators import method_decorator
from django.views.generic import ListView

@method_decorator(cache_page(60 * 15), name="dispatch")
class ArticleListView(ListView):
    model = Article

Or at URL registration:

from django.views.decorators.cache import cache_page

urlpatterns = [
    path("articles/", cache_page(60 * 15)(ArticleListView.as_view())),
]

Different cache per user #

@cache_page alone returns the same response to all users. For logged-in pages this is dangerous — another user’s page could appear.

from django.views.decorators.vary import vary_on_cookie
from django.views.decorators.cache import cache_page

@cache_page(60 * 5)
@vary_on_cookie
def my_dashboard(request):
    ...

vary_on_cookie makes a different cache key per cookie. With many users, the hit rate drops, so efficiency suffers. For per-user pages, fragment caching is usually a better fit.

Per-site middleware cache #

When you want to cache the whole site.

MIDDLEWARE = [
    "django.middleware.cache.UpdateCacheMiddleware",   # ← top
    "django.middleware.common.CommonMiddleware",
    ...
    "django.middleware.cache.FetchFromCacheMiddleware",  # ← bottom
]

CACHE_MIDDLEWARE_ALIAS = "default"
CACHE_MIDDLEWARE_SECONDS = 600
CACHE_MIDDLEWARE_KEY_PREFIX = "myproject"

Order matters — UpdateCacheMiddleware on top (response time), FetchFromCacheMiddleware on the bottom (request time).

Fits mostly-static marketing sites, blogs, etc. For dynamic sites, view/fragment caches are more precise.

Template fragment cache #

Cache only part of a view. Sidebar, header, menu, popular posts, that kind of case.

{% load cache %}

{% cache 500 sidebar request.user.username %}
    <aside>
        <h3>Popular posts</h3>
        {% for post in popular_posts %}
            <a href="{{ post.url }}">{{ post.title }}</a>
        {% endfor %}
    </aside>
{% endcache %}

Syntax: {% cache <seconds> <key name> [extra key args...] %}

The extra key args are cache branching keys. Including request.user.username gives a different cache per user. Including LANGUAGE_CODE gives one per language.

{% cache 600 article_card article.id article.updated_at user.is_authenticated %}
   ...
{% endcache %}

Putting article.updated_at in the key is a common trick to dodge manual invalidation — when the post changes, the key changes automatically and a new cache is built. You don’t need explicit invalidation.

Low-level cache API #

Three-line summary:

from django.core.cache import cache

cache.set("key", value, timeout=300)   # 5 minutes
value = cache.get("key", default=None)
cache.delete("key")

Common methods #

cache.set("a", 1, timeout=60)
cache.add("a", 2)                  # No-op if exists (set-if-not-exists)
cache.get_or_set("a", lambda: compute(), timeout=60)   # Build & set if missing

cache.set_many({"x": 1, "y": 2}, timeout=60)
cache.get_many(["x", "y"])         # {"x": 1, "y": 2}
cache.delete_many(["x", "y"])

cache.incr("counter", 1)            # Atomic increment
cache.decr("counter", 1)

cache.has_key("a")                  # bool
cache.clear()                       # All (dangerous)

cache.set("a", 1, timeout=None)     # No expiry
cache.set("a", 1, timeout=0)        # Expire immediately (= not cached)

get_or_set is the most common pattern.

def get_dashboard_stats(user_id):
    return cache.get_or_set(
        f"dashboard_stats:{user_id}",
        lambda: compute_dashboard_stats(user_id),
        timeout=300,
    )

Multiple cache backends side by side #

CACHES = {
    "default": {"BACKEND": "...redis...", "LOCATION": "redis://r1:6379/1"},
    "sessions": {"BACKEND": "...redis...", "LOCATION": "redis://r1:6379/2"},
    "long": {"BACKEND": "...redis...", "LOCATION": "redis://r2:6379/0"},
}

from django.core.cache import caches

caches["long"].set("annual_report", data, timeout=86400 * 7)

Cache key design — invalidation strategies #

There are two hard things in computer science, they say: naming and cache invalidation. And off-by-one errors.

1) TTL only — simplest #

cache.set("popular_posts", qs, timeout=60)

For data where staleness is acceptable. Simplicity is a big virtue. If you can accept 60 seconds of stale, 95% of cache problems go away.

2) Put a version in the key — auto invalidation #

key = f"article:{article.id}:{int(article.updated_at.timestamp())}"
cache.set(key, rendered, timeout=86400 * 7)

When updated_at changes, the key changes, so a new cache is built. Old caches expire naturally via TTL.

3) Explicit delete — most accurate, hardest #

from django.db.models.signals import post_save
from django.dispatch import receiver

@receiver(post_save, sender=Article)
def invalidate_article_cache(sender, instance, **kwargs):
    cache.delete(f"article:{instance.id}")
    cache.delete("article_list:popular")   # affected keys

The problem is you must know all affected keys. In a large project you’ll soon miss some. Solve with #1 or #2 if you can; keep explicit delete to a minimum.

4) Pattern delete — django-redis’s strength #

from django_redis import get_redis_connection

cache.delete_pattern("article:*")   # django-redis only

The built-in Redis backend has no delete_pattern. If pattern invalidation is needed, use django-redis.

Cache-Control / ETag — client-side cache #

After the server has built a result, it’s even faster if browsers/CDNs don’t ask again.

cache_control — Cache-Control header #

from django.views.decorators.cache import cache_control

@cache_control(public=True, max_age=3600)
def static_ish_view(request):
    ...

The response gets Cache-Control: public, max-age=3600, and the browser/CDN doesn’t ask again for an hour.

@cache_control(private=True, max_age=300)
def user_view(request):
    ...

private means shared caches (CDN) must not cache; only the browser may.

condition — ETag / Last-Modified #

from django.views.decorators.http import condition

def article_etag(request, pk):
    return Article.objects.filter(pk=pk).values_list("updated_at", flat=True).first()

def article_last_modified(request, pk):
    return Article.objects.filter(pk=pk).values_list("updated_at", flat=True).first()

@condition(etag_func=article_etag, last_modified_func=article_last_modified)
def article_detail(request, pk):
    article = Article.objects.get(pk=pk)
    return render(request, "article_detail.html", {"article": article})

Flow:

1. Client sends If-None-Match: <etag> or If-Modified-Since: <date>
2. Django calls etag_func / last_modified_func and compares
3. If match, respond with 304 Not Modified only (no body)
4. If different, run the view

etag_func should be cheap to be worthwhile. A single updated_at lookup is fine.

Cache stampede #

The cache expires all at once, every request misses simultaneously, and the same expensive computation runs N times in parallel. That’s a stampede.

Simple lock #

from django_redis import get_redis_connection

def get_expensive(key):
    val = cache.get(key)
    if val is not None:
        return val

    redis = get_redis_connection("default")
    lock = redis.lock(f"lock:{key}", timeout=10)
    if lock.acquire(blocking=True, blocking_timeout=5):
        try:
            val = cache.get(key)   # double-check
            if val is not None:
                return val
            val = compute_expensive()
            cache.set(key, val, timeout=300)
            return val
        finally:
            lock.release()

    # Couldn't get the lock — return stale or briefly retry
    return cache.get(key) or compute_expensive()

Only the first request runs the expensive compute; others wait for it to finish and read from the cache.

Pre-compute / spread refresh times #

High-traffic sites refresh in the background just before TTL expires — using tools like Celery beat. Or add random jitter to TTL to spread out simultaneous expirations.

import random

cache.set(key, val, timeout=300 + random.randint(0, 60))

Cache vs memoization vs CDN — pick the right one #

Use all three together. Static responses go on CDN, per-user dynamic on Django cache, small pure functions get lru_cache. There’s also overlap with DRF #3’s pagination.

Cases not to cache #

• POST/PUT/DELETE
• Auth token validation itself
• SaaS dashboards where every page is fully different per user — hit rate too low
• Payments, balances — data where stale is dangerous
• Data that changes faster than TTL expiry

Common pitfalls #

1) @cache_page only on user pages #

The one we just saw — another user can see someone else’s page. Use vary_on_cookie or fragment caching.

2) Caching huge objects #

Don’t cache the QuerySet itself — cache list(qs) or the serialized result. Anything too large might be better left uncached if serialize/deserialize cost exceeds the savings.

3) Redis dies → site dies #

Use the IGNORE_EXCEPTIONS=True setting and the cache.get(..., default=None) pattern.

4) Sensitive data in cache #

Tokens, password hashes, etc. — never cache. The cache is for recomputable data.

5) Key collisions #

Set KEY_PREFIX differently per environment (dev, prod) — required if you share the same Redis. Pairs with the settings split covered in #7.

Wrap-up #

What you covered this time:

• Backends: locmem (dev), Redis (prod standard), Memcached, DB
• django-redis + IGNORE_EXCEPTIONS=True recommended for production
• Per-view: @cache_page; per-user use vary_on_cookie or fragment
• Per-site: UpdateCacheMiddleware + FetchFromCacheMiddleware
• Template fragment: {% cache <seconds> <key name> <branching args> %}
• Low-level: cache.set/get/delete, get_or_set, incr, set_many, alias
• Invalidation: TTL > key version (updated_at) > explicit delete > pattern (django-redis)
• Client cache: cache_control, condition (ETag/Last-Modified)
• Stampede: lock + double-check, jitter, pre-compute
• Cache vs memoization vs CDN — pick the right one
• Pitfalls: cache_page on per-user response, huge objects, Redis down, sensitive data, key collisions

In the next post (#5 Signals in depth and post-transaction work) we layer transaction.on_commit, savepoints, custom signals, and Celery integration on top of Intermediate #3’s signals. The answer to “what’s scary when calling external systems inside a signal” lives there.