[Bug]: 502 instead of 403 when the credential is not authorized for the X-Org-Id organization #565

Bug Description

When a credential is not authorized for the organization named in the X-Org-Id header, every authenticated automation endpoint (including the KV API) answers 502 Bad Gateway instead of 403 Forbidden.

authenticate_request in openhands/automation/auth.py forwards X-Org-Id to OpenHands GET /api/v1/users/me (added for #402). OpenHands correctly answers 403 {"detail":"API key is not authorized for this organization"}. The service only handles 200, 401 and 429 from that call, so the 403 falls into if resp.status_code != 200: and becomes 502 "Unexpected response from OpenHands API". The status and the explanation are lost.

A 403 from the identity service is an authorization result about the caller, not a gateway failure. A 502 tells clients and operators to look for an outage, and it is the status clients usually retry.

Expected Behavior

• 403 when /users/me answers 403, passing the upstream detail through (for example "API key is not authorized for this organization").
• 502 stays for unreachable upstream and for upstream 5xx.
• Other upstream 4xx (for example 404 for an unknown org) should be a client error and not a 502 either. Which status is right for those is a call for the maintainers.

Actual Behavior

Reproduced on 2026-10-07 against a beta instance (automation service 1.15.1) and against SaaS production (https://app.all-hands.dev). Use any API key and any organization UUID the key is not authorized for. A random UUID works:

H=https://app.beta.staging.all-hands-testing.dev   # or https://app.all-hands.dev
RND=$(python3 -c "import uuid; print(uuid.uuid4())")

# 1. Automation list: 502
curl -s -i -H "Authorization: Bearer $OPENHANDS_API_KEY" -H "X-Org-Id: $RND" \
  "$H/api/automation/v1?limit=1"

# 2. The upstream call the service makes on the caller's behalf: 403 with a clear message
curl -s -i -H "Authorization: Bearer $OPENHANDS_API_KEY" -H "X-Org-Id: $RND" "$H/api/v1/users/me"

# 3. KV API: 502 as well
AID=<any automation id in the key's org>
curl -s -i -H "Authorization: Bearer $OPENHANDS_API_KEY" -H "X-Org-Id: $RND" \
  "$H/api/automation/v1/kv?automation_id=$AID"

# 4. For contrast, a malformed header is handled correctly
curl -s -w " -> %{http_code}\n" -H "Authorization: Bearer $OPENHANDS_API_KEY" \
  -H "X-Org-Id: nope" "$H/api/automation/v1?limit=1"     # 400 "Invalid X-Org-Id header (must be a UUID)"

On SaaS production the same request also returns 502, but the body is an nginx "Service Temporarily Unavailable" HTML page, so even the JSON detail is gone. That page makes an authorization error look like an outage.

The same happens for organizations the user really belongs to. The test account belongs to four organizations (one personal, three shared). The API key was minted in the personal organization:

So the service cannot tell "this key belongs to another org" from "the platform is down".

Code path (as of main at b43294c):

# openhands/automation/auth.py, authenticate_request()
if resp.status_code == 401: ...   # -> 401
if resp.status_code == 429: ...   # -> 429
if resp.status_code != 200:       # 403, 404, 5xx, ... all land here
    raise HTTPException(status_code=502, detail="Unexpected response from OpenHands API")

Acceptance Criteria

• An X-Org-Id that /users/me rejects with 403 results in 403 from every authenticated endpoint, with the upstream detail preserved.
• Upstream 5xx and connection errors still produce 502, with a message that distinguishes them from an authorization failure.
• A test covers 403 from /users/me (and one other 4xx) through authenticate_request.
• Optional: retry 502/503/504 from /users/me in _make_auth_request_with_retry. Today only 429 is retried, so one transient upstream 5xx fails the caller's request (see the note below).

Additional Context

• Found while testing the KV client's user-auth support: https://github.com/OpenHands/automation-kv-client/pull/41 (comment with the same repro) and https://github.com/OpenHands/automation-kv-client/pull/40.
• Related: Automation API ignores Canvas-selected X-Org-Id when resolving organization #402 (forwarding X-Org-Id, now closed) and feat: add user-authenticated KV access #445 (user-authenticated KV access).
• The first key I used is organization-bound (GET /api/keys/current shows bound_org_id equal to the personal org). Per the OpenHands OpenAPI description of POST /api/keys, a key is bound to the creating org unless it is created with an explicit org_id: null. A bound key is refused (403) with X-Org-Id set to any other org, even one the user belongs to, and that 403 is what turns into the 502 above.
• I then created a temporary unbound key (org_id: null, deleted afterwards). With it, X-Org-Id switches organization correctly on /users/me and /api/automation/v1 for each of the four orgs the user belongs to (including a different role in one of them), and a random UUID gives the same 403 / 502 as above. So the bug affects every key and org combination that is not authorized; it is not limited to bound keys.
• One unexplained 502 "Unexpected response from OpenHands API" on a cold-cache request without X-Org-Id, with the key valid, was seen once on the beta instance and could not be reproduced in eight further cold-cache attempts. It suggests a transient non-200 from /users/me. The service log line Unexpected status from OpenHands /api/v1/users/me: <status> would show which status. This is what the optional retry above would help with.

This issue was created by an AI agent (OpenHands) on behalf of @jpshackelford.