Caching

Cache-aside. TTL vs event invalidation. Stampedes, poisoning, HTTP caching.

Why Cache?

A database query might take 50ms. Reading from Redis takes <1ms. For data that doesn't change every second, why fetch from the database every time?

Caching is the practice of storing the result of expensive operations so future requests get the result faster.

Types of caching in a backend system:
1. In-memory cache (within the process) — fastest, not shared across instances
2. Distributed cache (Redis, Memcached) — shared, slightly slower, persistent
3. HTTP caching (CDN/browser) — for static or semi-static responses
4. Database query cache — database-level (often unreliable, avoid)
5. Computed result cache — cache expensive computation results

Cache-Aside Pattern

The most common caching pattern. Application manages the cache:

async function getUser(id) {
  // 1. Check cache
  const cached = await redis.get(`user:${id}`);
  if (cached) return JSON.parse(cached);

  // 2. Cache miss — fetch from DB
  const user = await db.users.findById(id);
  if (!user) throw new NotFoundError();

  // 3. Store in cache with TTL
  await redis.set(`user:${id}`, JSON.stringify(user), "EX", 300); // 5 min

  return user;
}

On update: invalidate the cache entry

await db.users.update(id, data);
await redis.del(`user:${id}`);

On delete: also invalidate

await db.users.delete(id);
await redis.del(`user:${id}`);

Cache Invalidation

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

Strategies:
TTL (Time-To-Live) — Cache expires automatically. Simple, but data may be stale until TTL expires.

Event-based invalidation — When data changes, explicitly delete cache. Consistent but complex.

Write-through — Always write to both cache and DB simultaneously. Consistent, but writes are slower.

Write-behind — Write to cache first, DB async later. Fast writes, risk of data loss.

Tag-based invalidation — Tag cache entries with related keys. Invalidate all entries with a tag when data changes. (Redis 7.4+ supports this natively.)

The hard problem: knowing WHEN to invalidate, and which cache keys to delete when data changes.

Common Caching Pitfalls

Cache stampede (thundering herd) — Cache expires. 1000 requests all miss at once. All go to DB simultaneously. DB dies.
Fix: Cache lock (only one request rebuilds), stale-while-revalidate, probabilistic early expiration.

Cache poisoning — Malicious data gets into the cache. Always validate data before caching.

Stale data — Users see outdated information. Balance TTL with freshness requirements.

Memory pressure — Cache too many large objects. Set max-memory policies. Use LRU eviction.

Missing cache keys — Caching "null" results prevents repeated DB hits for non-existent records. Cache negative results too.

Over-caching — Caching data that changes every second wastes memory and adds complexity. Cache selectively.

HTTP Caching

The browser and CDN can cache responses without your app doing anything — if you set the right headers.

Cache-Control: max-age=3600
  → Cache for 1 hour. Use for public, slow-changing data.

Cache-Control: no-store
  → Never cache. Use for auth responses, personal data.

Cache-Control: no-cache
  → Revalidate with server before using cached version.

ETag: "abc123"
  → Version identifier. Client sends If-None-Match: "abc123"
  → If unchanged, server returns 304 Not Modified (no body!)

Last-Modified: Tue, 15 Jan 2024 10:30:00 GMT
  → Client sends If-Modified-Since on subsequent requests.

CDN caching: Put CloudFlare/Fastly in front of static or public API responses. Cache at the edge, close to users. Massive performance gains.