Skip to content

Rate limit HTTP requests to Azure Container Registry APIs#2137

Open
lbussell wants to merge 10 commits into
mainfrom
lbussell/registry-429
Open

Rate limit HTTP requests to Azure Container Registry APIs#2137
lbussell wants to merge 10 commits into
mainfrom
lbussell/registry-429

Conversation

@lbussell

@lbussell lbussell commented Jun 5, 2026

Copy link
Copy Markdown
Member

Ever since #2050, checking for image lifecycle annotations has sped up. Checking for existing EOL annotations can exceed rate ACR's per-identity rate limit.

The original LifecycleAnnotationService assumed that error while checking for annotations/referrers means that the annotation does not exist. If an HTTP 429 error occured while checking for lifecycle annotations, then it would report that the annotation doesn't exist. This caused issues in the EOL annotation step of the golang image publishing pipeline.

This PR adds:

  • A regression test that ensures that the LifecycleAnnotationService doesn't swallow errors.
  • Global ACR rate-limiting via standard HttpClient configuration.

For the rate limiting, I decided to migrate away from our ad-hoc HttpClientProvider. It existed since before we adopted Microsoft.Extensions.Hosting in #1860. M.E.Hosting contains an IHttpClientProvider by default. It already includes logging as well. Now, we configure HttpClient configuration and rate limiting through standard means.

lbussell and others added 4 commits June 3, 2026 15:38
… errors

LifecycleMetadataService.IsDigestAnnotatedForEolAsync catches all
exceptions and returns null, so a transient registry failure (e.g. an
ACR HTTP 429 rate-limit error) is silently treated as "not annotated".
This false negative lets already-EOL-annotated digests leak back into
the annotation list and fail the publish stage with a conflicting EOL
date.

Make OciArtifactType public so the test can reference it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the catch-all in IsDigestAnnotatedForEolAsync that logged and
returned null for any exception. A transient registry failure (e.g. an
ACR HTTP 429 rate-limit error) was being treated as "not annotated",
causing already-EOL-annotated digests to leak into the annotation list
and fail the publish stage with a conflicting EOL date. The legitimate
"not annotated" case is still represented by a null return when no
lifecycle referrer is present; genuine registry failures now propagate
so callers fail loudly instead of producing a false negative.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the ad-hoc IHttpClientProvider/HttpClientProvider abstraction
with the standard Microsoft.Extensions IHttpClientFactory. Consumers now
inject IHttpClientFactory and call CreateClient(). The custom logging
DelegatingHandler is dropped in favor of the factory's built-in HTTP
request/response logging.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lbussell lbussell requested review from dagood and mthalman June 5, 2026 21:18
@lbussell lbussell requested a review from a team as a code owner June 5, 2026 21:18
Comment thread src/ImageBuilder.Tests/AcrRateLimitingHandlerTests.cs Fixed
Comment thread src/ImageBuilder.Tests/AcrRateLimitingHandlerTests.cs Fixed
Comment thread src/ImageBuilder.Tests/AcrRateLimitingHandlerTests.cs Fixed
Comment thread src/ImageBuilder.Tests/AcrRateLimitingHandlerTests.cs Fixed
Comment thread src/ImageBuilder.Tests/AcrRateLimitingHandlerTests.cs Fixed
Comment thread src/ImageBuilder.Tests/LifecycleMetadataServiceTests.cs Fixed
)
{
Interlocked.Increment(ref _requestCount);
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK));

@mthalman mthalman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that ContainerRegistryClient and ContainerRegistryContentClient are not getting the benefit of these changes to support rate limiting. Consider updating their respective factory classes to configure an options instance for the client.

var options = new ContainerRegistryClientOptions
{
    Transport = new HttpClientTransport(httpClientFactory.CreateClient())
};

Comment thread src/ImageBuilder/LifecycleMetadataService.cs
Comment thread src/ImageBuilder/RateLimiting/AcrRateLimitingHandler.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants