A8. OpenMP vs pthreads

The Mystery

When scaling the dummy Multi-Layer Perceptron (MLP) up to full-batch MNIST size, a bizarre energy anomaly emerged. In my pure matrix multiplication benchmarks, the hand-rolled C pthreads engine consumed slightly less energy than the C++ OpenMP engine. However, when placing these identical engines inside a deep learning training loop, the situation violently flipped.

Thread Sleep vs. Spinlocks

In raw, sustained matrix multiplication, both thread pools are spun up once and churn through massive blocks of math for seconds. In that scenario, pthreads is highly efficient. However, an ML training loop executes a forward pass, stops, evaluates loss, executes a backward pass, and stops again. The matrix math is broken up into tiny microsecond chunks separated by thread synchronization barriers.

The difference lies in how the threads wait at these barriers:

• OpenMP: When a thread finishes its tiny chunk of work, OpenMP's runtime evaluates if it should yield the CPU. It aggressively puts idle threads to sleep, allowing the CPU to power down cores and save energy between layers.
• My pthreads implementation: I implemented a naive Busy-Wait Spinlock. When a worker thread finishes its math, it executes a while(true) loop waiting for the other threads. This busy loop forces the CPU core to stay active at 100% utilization and maximum turbo frequency, burning massive amounts of power while doing absolutely no useful work.

The Power Profiling Proof

To prove this, we can look at the average power draw during the dummy MLP training loop:

The OpenMP implementation finished in ~0.37s and burned 43.1J. This is roughly 116 Watts of average power draw. It spiked the CPU to maximum power, finished instantly, and went back to sleep.

The pthreads implementation took 3.45s and burned 239.3J. This is roughly 69 Watts of average power draw over a prolonged period. Because the threads are trapped in a spinlock during the microsecond gaps between layer multiplications, the cores are never allowed to power down, causing severe thermal and power inefficiency across thousands of loop iterations.