Senior LevelPython
How do you achieve Zero-Copy inter-process communication for massive NumPy arrays across Python processes using multiprocessing.shared_memory?
❓ Interview Prompt: Explain Zero-Copy IPC, `multiprocessing.shared_memory.SharedMemory`, zero-copy array sharing, and eliminating IPC serialization overhead.
Passing massive NumPy arrays across processes using standard 'multiprocessing.Queue' requires serializing ('pickle') and copying data across process IPC pipes.
Zero-Copy Shared Memory ('SharedMemory'):
Allocates shared OS RAM buffers ('SharedMemory(create=True, size=...)'). Worker processes attach to the shared buffer using 'np.ndarray(..., buffer=shm.buf)', reading and mutating RAM arrays instantly with ZERO copy overhead!
Code Implementation Example
python
from multiprocessing.shared_memory import SharedMemory
import numpy as np
# Create 100MB NumPy array in Shared Memory
shm = SharedMemory(create=True, size=100_000_000)
shared_arr = np.ndarray((1000, 1000, 25), dtype=np.float64, buffer=shm.buf)
shared_arr.fill(1.5)
print('Shared Memory Block Name:', shm.name)
# Worker process attaches to shm.name with ZERO data copying!💡Sharing massive NumPy arrays across processes with zero-copy shared memory.
🎯 Interview Key Takeaway
Use multiprocessing.shared_memory for zero-copy IPC array sharing across Python worker processes.
#SharedMemory#Zero-Copy#IPC#NumPy#Senior
Back to all Python Questions→