[acrcssc] Cannot respond to request for authentication during OCI artifact push - root cause identified, not Cloud Shell-specific

[serbathome]

Related issues

• [acrcssc] Awareness: az acr supply-chain workflow create --run-immediately fails in Azure Cloud Shell: ORAS receives 403 without WWW-Authenticate during OCI artifact push to csscpolicies #10003
• Awareness: az acr supply-chain workflow create --run-immediately fails in Azure Cloud Shell: ORAS receives 403 without WWW-Authenticate during OCI artifact push to csscpolicies azure-cli#32817

Those reports attribute this to Azure Cloud Shell's credential handoff. I hit the identical failure on a plain local Linux install (non-Cloud-Shell) and traced it to a concrete bug in how acrcssc uses oras-py's TokenAuth backend — it does not appear to be environment-specific.

Environment

azure-cli: 2.75.0 (also reproduced on 2.90.0 before downgrading for an unrelated SDK issue)
acrcssc: 1.0.0b7
oras (vendored): 0.2.25

Repro

az acr supply-chain workflow create \
  -r <registry> -g <resource-group> -t continuouspatchv1 \
  --config ./continuouspatching.json --schedule 1d --run-immediately

Fails with:

Failed to push OCI artifact to ACR: Cannot respond to request for authentication.

--dry-run succeeds (it never pushes), confirming the bug is isolated to the OCI artifact push step.

Root cause

_ociartifactoperations.py::_get_acr_token() shells out to az acr login --expose-token, which returns an ACR refresh token (explicitly not a usable access token — az itself warns about this). _oras_client() then does:

token = _get_acr_token(registry.name, subscription)
client = OrasClient(hostname=str.lower(registry.login_server), auth_backend="token")
client.login(BEARER_TOKEN_USERNAME, token)

But oras.provider.Registry.login() only ever calls self.auth.set_basic_auth(username, password) — it never calls set_token_auth(). So TokenAuth.token stays None after login.

Separately, oras.provider.Registry.do_request() contains:

if headers is not None and isinstance(self.auth, oras.auth.TokenAuth):
    headers.update(self.auth.get_auth_header())

TokenAuth.get_auth_header() unconditionally returns {"Authorization": "Bearer %s" % self.token}. Since self.token is None, this attaches a literal Authorization: Bearer None header on the first request, before any real 401 challenge/response cycle happens.

ACR sees a malformed-but-present Bearer credential and responds 403 without a WWW-Authenticate header (that's only sent on clean, unauthenticated 401s). TokenAuth.authenticate_request() then can't recover:

authHeaderRaw = original.headers.get("Www-Authenticate")
if not authHeaderRaw:
    logger.debug("Www-Authenticate not found in original response, cannot authenticate.")
    return headers, False   # -> do_request() raises ValueError("Cannot respond to request for authentication.")

I confirmed this by replaying the exact same "raw refresh token as Bearer" request manually — ACR returns 401 insufficient_scope with WWW-Authenticate in isolation, but the premature/poisoned first request in the real flow behaves differently and the extension never gets a chance to recover.

I also confirmed the oras-py "basic" backend (auth_backend="basic") is not a valid alternative — its authenticate_request() just resends raw HTTP Basic auth to the blob endpoint, which ACR's data-plane API doesn't accept; no real OAuth2 exchange happens there either.

Fix

Rather than relying on oras's broken token negotiation, perform the ACR refresh_token → scoped access_token exchange explicitly, and hand the resulting token directly to the auth backend via set_token_auth() (bypassing login()'s broken basic-auth-only path entirely):

Before (azext_acrcssc/helper/_ociartifactoperations.py):

def _oras_client(registry):
    resourceid = parse_resource_id(registry.id)
    subscription = resourceid[SUBSCRIPTION]

    try:
        token = _get_acr_token(registry.name, subscription)
        client = OrasClient(hostname=str.lower(registry.login_server), auth_backend="token")
        client.login(BEARER_TOKEN_USERNAME, token)
        logger.debug(f"Login to ACR {registry.name} completed successfully.")
    except Exception as exception:
        raise AzCLIError(f"Failed to login to Artifact Store ACR {registry.name}: {exception}")

    return client

After:

import requests

def _oras_client(registry):
    resourceid = parse_resource_id(registry.id)
    subscription = resourceid[SUBSCRIPTION]

    try:
        refresh_token = _get_acr_token(registry.name, subscription)
        login_server = str.lower(registry.login_server)
        # oras-py's TokenAuth backend does not perform the ACR
        # refresh_token -> access_token exchange correctly (it sends a
        # premature "Bearer None" header on the first request, and cannot
        # recover afterwards). Perform the exchange explicitly instead.
        scope = f"repository:{CSSC_WORKFLOW_POLICY_REPOSITORY}/{CONTINUOUSPATCH_OCI_ARTIFACT_CONFIG}:pull,push"
        exchange_resp = requests.post(
            f"https://{login_server}/oauth2/token",
            data={
                "grant_type": "refresh_token",
                "service": login_server,
                "scope": scope,
                "refresh_token": refresh_token,
            },
            timeout=30,
        )
        exchange_resp.raise_for_status()
        access_token = exchange_resp.json()["access_token"]

        client = OrasClient(hostname=login_server, auth_backend="token")
        client.auth.set_token_auth(access_token)
        logger.debug(f"Login to ACR {registry.name} completed successfully.")
    except Exception as exception:
        raise AzCLIError(f"Failed to login to Artifact Store ACR {registry.name}: {exception}")

    return client

I applied this locally against ~/.azure/cliextensions/acrcssc/ and confirmed az acr supply-chain workflow create ... --run-immediately now completes successfully end-to-end (artifact push, ARM task deployment, and workflow trigger all succeed).

Suggested longer-term fix

Ideally this gets fixed upstream in oras-py itself (TokenAuth/do_request shouldn't attach a Bearer None header when no token has been obtained yet), but the workaround above avoids depending on a fix there and is a minimal, self-contained change to acrcssc.

Happy to open a PR with this change if useful.

[yonzhan]

Thank you for opening this issue, we will look into it.