Intermediate LevelSystem Design & Architecture
What is the Cache Stampede (Thundering Herd) problem and how do you prevent it with Probabilistic Early Expiration?
❓ Interview Prompt: Explain Cache Stampede (Thundering Herd), hot key expiration, database spikes, XFetch algorithm, and mutual exclusion locking.
A Cache Stampede (Thundering Herd) occurs when a heavily accessed hot cache key expires, causing thousands of concurrent user requests to hit the database simultaneously to recompute the key:
Mitigation Strategies:
1
Mutex Lock (Single-Flight): The first request acquires a lock to recompute the cache value; subsequent requests wait for the cache update.
2
XFetch (Probabilistic Early Expiration): Randomly recomputes hot cache keys in the background shortly before actual TTL expiration based on access frequency!
Code Implementation Example
typescript
export async function getWithMutex(key: string, fetchFn: () => Promise<any>) {
const value = await redis.get(key);
if (value) return JSON.parse(value);
// Acquire Mutex Lock to prevent Thundering Herd on database!
const acquired = await redis.set('lock:\${key}', '1', 'NX', 'EX', 5);
if (!acquired) {
await sleep(50);
return getWithMutex(key, fetchFn); // Retry read from cache
}
const freshData = await fetchFn();
await redis.setex(key, 3600, JSON.stringify(freshData));
await redis.del('lock:\${key}');
return freshData;
}💡Preventing Cache Stampede with mutual exclusion locking.
🎯 Interview Key Takeaway
Use Mutex Locks or Probabilistic Early Expiration (XFetch) to prevent Thundering Herd spikes on database servers during hot cache key expirations.
#System Design#Cache Stampede#Thundering Herd#Caching#Intermediate
Back to all System Design & Architecture Questions→