Intermediate LevelPython
How do you distribute CPU-bound calculations across multiple CPU cores using multiprocessing.Pool.starmap() or ProcessPoolExecutor?
❓ Interview Prompt: Explain `multiprocessing.Pool`, `ProcessPoolExecutor`, distributing work across CPU cores, bypassing the GIL, and IPC serialization.
Because of the GIL, CPU-bound tasks must be executed across separate OS processes:
Process Pools:
•
'ProcessPoolExecutor' (from 'concurrent.futures'): Manages a pool of worker processes, distributing task payloads across available CPU cores to execute in parallel.
Code Implementation Example
python
from concurrent.futures import ProcessPoolExecutor
import math
def heavy_calculation(n):
return math.factorial(n)
if __name__ == '__main__':
numbers = [50000, 60000, 70000, 80000]
with ProcessPoolExecutor() as executor:
results = list(executor.map(heavy_calculation, numbers))
print('Calculated', len(results), 'results in parallel!')💡Distributing CPU-bound factorial tasks across CPU cores using ProcessPoolExecutor.
🎯 Interview Key Takeaway
Use ProcessPoolExecutor to bypass the GIL and parallelize heavy CPU calculations across all CPU cores.
#multiprocessing#ProcessPoolExecutor#Parallelism#Intermediate
Back to all Python Questions→