Intermediate LevelSystem Design & Architecture
How do you handle Stream Backpressure when data producers overwhelm data consumers?
❓ Interview Prompt: Explain Backpressure in streaming systems, push vs pull flow control, Reactive Streams specification, and buffer shedding strategies.
Backpressure occurs when a high-speed data Producer emits events faster than a downstream Consumer can process them:
3 Handling Strategies:
1
Buffering: Queue incoming messages up to a memory threshold (risk of OOM crashes under sustained spikes).
2
Drop / Shedming: Drop excess messages or apply sampling (acceptable for non-critical telemetry logs).
3
Pull-Based Flow Control (Reactive Streams): Consumer explicitly requests chunk capacity ('subscription.request(100)'), forcing the Producer to pause emission until requested!
Code Implementation Example
typescript
// Reactive Stream Backpressure Pull-based flow control
class BackpressureConsumer {
onSubscribe(subscription: any) {
this.subscription = subscription;
this.subscription.request(50); // Request only 50 items at a time!
}
onNext(item: any) {
this.process(item);
this.subscription.request(1); // Ask for 1 more item after processing
}
}💡Implementing pull-based backpressure in streaming consumers.
🎯 Interview Key Takeaway
Use pull-based backpressure mechanisms to prevent fast producers from exhausting consumer memory resources.
#System Design#Backpressure#Reactive Streams#Flow Control#Intermediate
Back to all System Design & Architecture Questions→