Intermediate LevelNode JS
How do you process multi-gigabyte text files line-by-line without memory exhaustion using readline?
❓ Interview Prompt: import fs from "node:fs"; import readline from "node:readline"; const rl = readline.createInterface({ input: fs.createReadStream("massive.log") }); for await (const line of rl) { processLine(line); }. RAM usage remains flat under 20MB!
Loading a multi-gigabyte log file into RAM via 'fs.readFile()' causes memory exhaustion crashes ('JavaScript heap out of memory').
Line-by-Line Streaming:
Combining 'fs.createReadStream()' with 'node:readline' streams and parses one line at a time, keeping RAM consumption flat under 20MB regardless of file size!
Code Implementation Example
javascript
import fs from 'node:fs';
import readline from 'node:readline';
async function processMassiveLogFile(filePath) {
const fileStream = fs.createReadStream(filePath);
const rl = readline.createInterface({
input: fileStream,
crlfDelay: Infinity
});
for await (const line of rl) {
if (line.includes('ERROR')) {
console.log('Error line found:', line);
}
}
}💡Streaming multi-gigabyte log files line-by-line with node:readline.
🎯 Interview Key Takeaway
Combine fs.createReadStream with node:readline to parse multi-gigabyte files line-by-line with flat memory usage.
#readline#Streaming#Log Processing#Memory Efficiency#Intermediate
Back to all Node JS Questions→