hubble-relay image v1.15.0 missing h2 ALPN support that breaks hubble CLI (grpc-go >= 1.67)

[pesarkhobeee]

Description

The retina-hubble chart v1.1.0 defaults to mcr.microsoft.com/oss/cilium/hubble-relay:v1.15.0. This relay image is built from Cilium v1.15.0, which does not include the h2 ALPN fix from cilium/cilium#33616.
As a result, the hubble CLI cannot connect to the relay, the TLS handshake fails because the relay does not advertise the h2 ALPN protocol, which grpc-go >= 1.67 requires.

This fix has been backported to all maintained Cilium branches, including v1.15.19:
https://github.com/cilium/cilium/blob/v1.15.19/pkg/crypto/certloader/server.go

The current default image (v1.15.0) is 19 patch versions behind.

I initially reported this as a Cilium issue at cilium/cilium#45235, where a maintainer confirmed that the fix already exists upstream since v1.16.0 and has been backported to v1.15.x.

Environment

• Retina chart: oci://ghcr.io/microsoft/retina/charts/retina-hubble v1.1.0
• Default relay image: mcr.microsoft.com/oss/cilium/hubble-relay:v1.15.0
• Kubernetes: GKE (europe-west3)
• Hubble CLI tested: v1.18.6 (Go 1.25.7), v1.16.5 (Go 1.23.4), v0.13.5 (Go 1.21.11), all fail

Steps to reproduce

1. Deploy retina-hubble chart v1.1.0 with default values (relay image v1.15.0)
2. Port-forward to hubble-relay:
   kubectl port-forward -n retina-hubble svc/hubble-relay 4245:443
3. Extract client certs and connect:

  hubble observe --server localhost:4245 --tls \
    --tls-ca-cert-files ca.crt --tls-client-cert-file client.crt \
    --tls-client-key-file client.key --tls-server-name "ui.hubble-relay.cilium.io" -f

Actual behavior

  rpc error: code = Unavailable desc = connection error: desc = "transport: authentication
  handshake failed: credentials: cannot check peer: missing selected ALPN property"

Verified with openssl:

  echo | openssl s_client -connect localhost:4245 -alpn h2 2>&1 | grep -i alpn
  # Output: No ALPN negotiated

Expected behavior

The relay should advertise h2 via ALPN during the TLS handshake, allowing hubble CLI to connect.

The proper fix already exists in pkg/crypto/certloader/server.go via cilium/cilium#33616 — the constructWithH2ProtoIfNeed function adds h2 to NextProtos inside GetConfigForClient. This is present in Cilium v1.15.19 but absent in v1.15.0.

Suggested fix

Bump the default hubble-relay image in the retina-hubble chart from v1.15.0 to v1.15.19 (or later). The MCR mirror has v1.16.1 available:

curl -s "https://mcr.microsoft.com/v2/oss/cilium/hubble-relay/tags/list" | jq .

Impact

• hubble CLI is completely unusable with the default chart configuration
• Affects all hubble CLI versions compiled with Go >= 1.22
• Hubble UI is unaffected (connects from within the cluster using its own gRPC client)
• The relay image is also 19 patch versions behind on security fixes

[nddq]

@pesarkhobeee this should be a simple bump. Feel free to open a PR for this!

[pesarkhobeee]

@nddq Sure thing! One suggestion though, could we jump to the latest version rather than staying on a minor version? There have been many fixes added over time, and I recently contributed one that's essential for our Retina-Hubble setup:
cilium/cilium#45021

Is there anything blocking us from using the latest version, or any considerations I should be aware of?

[nddq]

@pesarkhobeee nothing that should be blocking us as far as I'm aware. You can definitely run some tests locally and show that in the PR.