Senior LevelExpress JS
How do you stream a 10,000,000-row database query export directly to an Express HTTP response as CSV?
❓ Interview Prompt: Explain backpressure-aware streaming, Node.js stream piping, database cursors, and memory-efficient massive CSV exports.
Attempting to load 10,000,000 database rows into a JavaScript array in memory before sending a response throws a fatal Out of Memory error.
Backpressure-Aware Streaming:
1
Execute query using a database Cursor Stream ('pg-query-stream').
2
Pipe database cursor through a CSV Transform stream directly into the Express response object ('dbStream.pipe(csvTransform).pipe(res)').
3
The Express response socket exerts backpressure automatically, pausing database row reading whenever the client network buffer fills!
Code Implementation Example
javascript
import express from 'express';
import QueryStream from 'pg-query-stream';
import { Transform } from 'stream';
const app = express();
app.get('/api/export-csv', async (req, res) => {
res.setHeader('Content-Type', 'text/csv');
res.setHeader('Content-Disposition', 'attachment; filename="export.csv"');
const client = await pool.connect();
const query = new QueryStream('SELECT id, email, created_at FROM users');
const stream = client.query(query);
const csvTransform = new Transform({
objectMode: true,
transform(row, encoding, callback) {
callback(null, `${row.id},${row.email},${row.created_at}\n`);
}
});
// Pipe database stream directly to HTTP response with automatic backpressure!
stream.pipe(csvTransform).pipe(res);
stream.on('end', () => client.release());
});💡Streaming 10M database rows to HTTP response as CSV with backpressure handling.
🎯 Interview Key Takeaway
Pipe database cursor streams through transform streams directly to res to handle massive exports with automatic backpressure.
#Backpressure#Streaming#Database Export#Senior
Back to all Express JS Questions→