Systems

This post is my own summary of Netflix’s tech blog post Seeing through hardware counters: a journey to threefold performance increase, written the way I understood it, with some background I added along the way. The original post is the authoritative source, so check it directly for exact figures and details. The Problem A Netflix service was running short on CPU, so the team scaled its nodes up 3x. Given the CPU-intensive workload, they expected throughput to scale roughly in step. Instead, they got about a 25% improvement, and tail latency got worse. ...

Locks split into two broad families: spinlocks, which run entirely in userspace on raw CPU instructions, and mutexes, which lean on the kernel. Both exist to guarantee mutual exclusion, but they get there differently, and that difference determines which workload each one fits. Spinlock A spinlock lives entirely in userspace. The mechanism is simple: loop forever until a CAS (Compare-And-Swap) succeeds. while (!CAS(lock, 0, 1)) Most CPUs ship a dedicated instruction for this. On x86 it’s LOCK CMPXCHG. The catch is that a CAS only succeeds once the core holds exclusive write access to that cache line. Getting there pulls the cache coherence protocol into play, and passing ownership of a cache line back and forth triggers frequent invalidation, what people call cache line bouncing. That round trip usually costs somewhere between 4 and 80 nanoseconds. ...