Primer on Parallelism

Implementation Approaches

🧵 Multi-Threading

Shared memory within process

✅ Fast communication
✅ Low overhead
⚠️ Race conditions
⚠️ Python GIL limitation

Tightly coupled tasks

⚙️ Multi-Processing

Isolated processes with IPC

✅ True parallelism
✅ No race conditions
⚠️ Communication overhead
⚠️ Higher memory use

Embarrassingly parallel

Communication Trade-off

Tip

Litmus test: “Do the same thing to 1000 inputs independently” → Multi-processing