Skip to content

Commit 25fd5ae

Browse files
FrostGodFrostGod
committed
feat(helm): add opt-in per-replica GRPCRoute routing
Signed-off-by: divesh <dgude@nvidia.com>
1 parent 19fc049 commit 25fd5ae

9 files changed

Lines changed: 192 additions & 3 deletions

File tree

‎deploy/helm/openshell/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -375,6 +375,7 @@ discovery endpoint or its TLS CA.
375375
| grpcRoute.gateway.name | string | `""` | Name of the Gateway resource. Defaults to the chart fullname. |
376376
| grpcRoute.gateway.namespace | string | `""` | Namespace of the Gateway referenced by the GRPCRoute parentRef. Defaults to the release namespace. |
377377
| grpcRoute.hostnames | list | `[]` | Hostnames the GRPCRoute matches on. Leave empty to match all hosts. |
378+
| grpcRoute.replicaRouting.enabled | bool | `false` | Route requests carrying an `x-openshell-replica` header straight to that gateway replica. The CLI sets the header on long-lived sandbox connections (SSH, port forwards, exec) using the owner the gateway reports, so they skip the relay through a peer replica. Renders one Service per replica. Requires workload.kind=statefulset, workload.allowMultiReplicaStatefulSet=true, and at most 15 replicas. |
378379
| imagePullSecrets | list | `[]` | Image pull secrets attached to gateway and helper pods. |
379380
| nameOverride | string | `"openshell"` | Override the chart name used in generated resource names. |
380381
| networkPolicy.enabled | bool | `true` | Restrict SSH ingress on sandbox pods to the gateway. In managed mode, the driver applies the equivalent policy to each workspace namespace. |

‎deploy/helm/openshell/templates/_helpers.tpl‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -432,6 +432,14 @@ Validate chart values that Helm would otherwise accept silently.
432432
{{- if and (eq $workloadKind "statefulset") (gt $maxReplicas 1) (not (get $workload "allowMultiReplicaStatefulSet" | default false)) -}}
433433
{{- fail (printf "%s > 1 with workload.kind=statefulset requires workload.allowMultiReplicaStatefulSet=true; use workload.kind=deployment for external database-backed multi-replica gateways." $maxReplicasSource) -}}
434434
{{- end -}}
435+
{{- if and .Values.grpcRoute.enabled (dig "replicaRouting" "enabled" false .Values.grpcRoute) -}}
436+
{{- if ne $workloadKind "statefulset" -}}
437+
{{- fail "grpcRoute.replicaRouting.enabled requires workload.kind=statefulset so each replica has a stable name." -}}
438+
{{- end -}}
439+
{{- if gt $maxReplicas 15 -}}
440+
{{- fail (printf "grpcRoute.replicaRouting.enabled supports %s of at most 15; a GRPCRoute holds at most 16 rules." $maxReplicasSource) -}}
441+
{{- end -}}
442+
{{- end -}}
435443
{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}}
436444
{{- if not (has $workspaceMode (list "shared" "managed" "operator")) -}}
437445
{{- fail "server.drivers.kubernetes.workspaceMode must be one of: shared, managed, operator." -}}

‎deploy/helm/openshell/templates/backend-tls-policy.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,13 @@ spec:
2020
- group: ""
2121
kind: Service
2222
name: {{ include "openshell.fullname" . }}
23+
{{- if and .Values.grpcRoute.enabled (dig "replicaRouting" "enabled" false .Values.grpcRoute) }}
24+
{{- range $i := until (int (include "openshell.maxReplicas" $)) }}
25+
- group: ""
26+
kind: Service
27+
name: {{ printf "%s-replica-%d" (include "openshell.fullname" $) $i }}
28+
{{- end }}
29+
{{- end }}
2330
validation:
2431
caCertificateRefs:
2532
- group: ""

‎deploy/helm/openshell/templates/grpcroute.yaml‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,36 @@
11
{{- if .Values.grpcRoute.enabled }}
22
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
33
# SPDX-License-Identifier: Apache-2.0
4+
{{- $fullname := include "openshell.fullname" . }}
45
apiVersion: gateway.networking.k8s.io/v1
56
kind: GRPCRoute
67
metadata:
7-
name: {{ include "openshell.fullname" . }}
8+
name: {{ $fullname }}
89
namespace: {{ .Release.Namespace }}
910
labels:
1011
{{- include "openshell.labels" . | nindent 4 }}
1112
spec:
1213
parentRefs:
13-
- name: {{ default (include "openshell.fullname" .) .Values.grpcRoute.gateway.name }}
14+
- name: {{ default $fullname .Values.grpcRoute.gateway.name }}
1415
namespace: {{ default .Release.Namespace .Values.grpcRoute.gateway.namespace }}
1516
{{- if .Values.grpcRoute.hostnames }}
1617
hostnames:
1718
{{- toYaml .Values.grpcRoute.hostnames | nindent 4 }}
1819
{{- end }}
1920
rules:
21+
{{- if (dig "replicaRouting" "enabled" false .Values.grpcRoute) }}
22+
{{- range $i := until (int (include "openshell.maxReplicas" $)) }}
23+
- matches:
24+
- headers:
25+
- type: Exact
26+
name: x-openshell-replica
27+
value: {{ printf "%s-%d" $fullname $i }}
28+
backendRefs:
29+
- name: {{ printf "%s-replica-%d" $fullname $i }}
30+
port: {{ $.Values.service.port }}
31+
{{- end }}
32+
{{- end }}
2033
- backendRefs:
21-
- name: {{ include "openshell.fullname" . }}
34+
- name: {{ $fullname }}
2235
port: {{ .Values.service.port }}
2336
{{- end }}

‎deploy/helm/openshell/templates/replica-services.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{{- if and .Values.grpcRoute.enabled (dig "replicaRouting" "enabled" false .Values.grpcRoute) }}
2+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# SPDX-License-Identifier: Apache-2.0
4+
{{- $fullname := include "openshell.fullname" . }}
5+
{{- range $i := until (int (include "openshell.maxReplicas" $)) }}
6+
---
7+
apiVersion: v1
8+
kind: Service
9+
metadata:
10+
name: {{ printf "%s-replica-%d" $fullname $i }}
11+
labels:
12+
{{- include "openshell.labels" $ | nindent 4 }}
13+
spec:
14+
ports:
15+
- port: {{ $.Values.service.port }}
16+
targetPort: grpc
17+
protocol: TCP
18+
name: grpc
19+
appProtocol: grpc
20+
selector:
21+
{{- include "openshell.selectorLabels" $ | nindent 4 }}
22+
statefulset.kubernetes.io/pod-name: {{ printf "%s-%d" $fullname $i }}
23+
{{- end }}
24+
{{- end }}

‎deploy/helm/openshell/tests/replica_routing_test.yaml‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
suite: GRPCRoute replica routing
5+
templates:
6+
- templates/grpcroute.yaml
7+
- templates/replica-services.yaml
8+
- templates/gateway-config.yaml
9+
- templates/statefulset.yaml
10+
release:
11+
name: openshell
12+
namespace: my-namespace
13+
set:
14+
grpcRoute.enabled: true
15+
replicaCount: 2
16+
workload.allowMultiReplicaStatefulSet: true
17+
server.externalDbSecret: db
18+
19+
tests:
20+
- it: routes every request to the shared Service by default
21+
template: templates/grpcroute.yaml
22+
asserts:
23+
- lengthEqual:
24+
path: spec.rules
25+
count: 1
26+
- equal:
27+
path: spec.rules[0].backendRefs[0].name
28+
value: openshell
29+
30+
- it: renders no per-replica Services by default
31+
template: templates/replica-services.yaml
32+
asserts:
33+
- hasDocuments:
34+
count: 0
35+
36+
- it: renders when values from an older release have no replicaRouting key
37+
template: templates/grpcroute.yaml
38+
set:
39+
grpcRoute.replicaRouting: null
40+
asserts:
41+
- lengthEqual:
42+
path: spec.rules
43+
count: 1
44+
45+
- it: routes a named replica to its own Service ahead of the shared rule
46+
template: templates/grpcroute.yaml
47+
set:
48+
grpcRoute.replicaRouting.enabled: true
49+
asserts:
50+
- lengthEqual:
51+
path: spec.rules
52+
count: 3
53+
- equal:
54+
path: spec.rules[1].matches[0].headers[0]
55+
value:
56+
type: Exact
57+
name: x-openshell-replica
58+
value: openshell-1
59+
- equal:
60+
path: spec.rules[1].backendRefs[0].name
61+
value: openshell-replica-1
62+
- equal:
63+
path: spec.rules[2].backendRefs[0].name
64+
value: openshell
65+
66+
- it: selects exactly one pod per replica Service
67+
template: templates/replica-services.yaml
68+
set:
69+
grpcRoute.replicaRouting.enabled: true
70+
asserts:
71+
- hasDocuments:
72+
count: 2
73+
- equal:
74+
path: metadata.name
75+
value: openshell-replica-1
76+
documentIndex: 1
77+
- equal:
78+
path: spec.selector["statefulset.kubernetes.io/pod-name"]
79+
value: openshell-1
80+
documentIndex: 1
81+
82+
- it: routes every replica the autoscaler can create
83+
template: templates/grpcroute.yaml
84+
set:
85+
grpcRoute.replicaRouting.enabled: true
86+
autoscaling.enabled: true
87+
resources.requests.cpu: 100m
88+
asserts:
89+
- lengthEqual:
90+
path: spec.rules
91+
count: 5
92+
93+
- it: caps autoscaled replica routing at 15 replicas
94+
template: templates/statefulset.yaml
95+
set:
96+
grpcRoute.replicaRouting.enabled: true
97+
autoscaling.enabled: true
98+
autoscaling.maxReplicas: 16
99+
resources.requests.cpu: 100m
100+
asserts:
101+
- failedTemplate:
102+
errorMessage: "grpcRoute.replicaRouting.enabled supports autoscaling.maxReplicas of at most 15; a GRPCRoute holds at most 16 rules."
103+
104+
- it: requires a StatefulSet for stable replica names
105+
template: templates/statefulset.yaml
106+
set:
107+
grpcRoute.replicaRouting.enabled: true
108+
workload.kind: deployment
109+
asserts:
110+
- failedTemplate:
111+
errorMessage: "grpcRoute.replicaRouting.enabled requires workload.kind=statefulset so each replica has a stable name."

‎deploy/helm/openshell/values.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -714,6 +714,14 @@ grpcRoute:
714714
# or the existing openshell-server-tls Secret (its SANs must include the
715715
# external hostname).
716716
certificateRefs: []
717+
replicaRouting:
718+
# -- Route requests carrying an `x-openshell-replica` header straight to
719+
# that gateway replica. The CLI sets the header on long-lived sandbox
720+
# connections (SSH, port forwards, exec) using the owner the gateway
721+
# reports, so they skip the relay through a peer replica. Renders one
722+
# Service per replica. Requires workload.kind=statefulset,
723+
# workload.allowMultiReplicaStatefulSet=true, and at most 15 replicas.
724+
enabled: false
717725
# BackendTLSPolicy for end-to-end TLS between the Gateway proxy and the
718726
# OpenShell gateway pod. When enabled, the Gateway proxy terminates
719727
# client-facing TLS at the listener and re-encrypts when connecting to the

‎docs/kubernetes/ingress.mdx‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,22 @@ This situation can occur if you set `pkiInitJob.failOnTimeout=false` and cert-ma
206206

207207
For OpenShift 4.22+, see [OpenShift](/kubernetes/openshift#end-to-end-tls-using-gateway-api-and-backendtlspolicy-openshift-422) for platform-specific instructions including Gateway and GatewayClass setup.
208208

209+
## Replica Routing
210+
211+
With more than one gateway replica, a sandbox's SSH, port-forward, and exec connections can land on a replica that does not hold the sandbox's supervisor session, and that replica relays them to the one that does. Enable replica routing to send those connections straight to the owning replica:
212+
213+
```shell
214+
helm upgrade <release-name> oci://ghcr.io/nvidia/openshell/helm-chart \
215+
--reuse-values --namespace <namespace> \
216+
--set workload.kind=statefulset \
217+
--set workload.allowMultiReplicaStatefulSet=true \
218+
--set grpcRoute.replicaRouting.enabled=true
219+
```
220+
221+
The chart renders one Service per replica and a GRPCRoute rule that matches the `x-openshell-replica` request header. The CLI sets that header from the owner the gateway reports and retries without it if the named replica is unavailable. Replica routing requires `workload.kind=statefulset`, `workload.allowMultiReplicaStatefulSet=true`, and at most 15 replicas.
222+
223+
If the release uses `workload.kind=deployment`, this upgrade replaces the Deployment with a StatefulSet, so every gateway pod restarts at once. The StatefulSet also requests a 1Gi PersistentVolumeClaim per replica, which stays `Pending` on clusters without a default StorageClass.
224+
209225
## SSH Relay
210226

211227
Sandbox SSH uses the gateway endpoint registered with the CLI. No separate Helm SSH host or port values are required.

‎skills/debug-openshell-cluster/SKILL.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1022,6 +1022,7 @@ credential failures.
10221022
| Image pull failure | Gateway or sandbox image cannot be pulled | Runtime events and image pull credentials |
10231023
| Gateway API resources fail with `the server could not find the requested resource` | Optional Gateway API resources were applied without Envoy Gateway CRDs | Install Envoy Gateway and enable `grpcRoute` before applying the optional ingress resources |
10241024
| HTTPS ingress (`grpcRoute.gateway.listener.protocol=HTTPS`) connection resets or TLS handshake hangs | Envoy terminates TLS but the gateway pod still expects TLS, so the plaintext backend hop fails | Set `server.disableTls=true` so Envoy forwards plaintext to the pod; verify the listener `certificateRefs` Secret exists in the release namespace and `openshell status` over `https://<host>` |
1025+
| With `grpcRoute.replicaRouting.enabled=true`, sandbox SSH, forward, or exec still relay through a peer replica | A `<release>-replica-<i>` Service has no endpoint, so Envoy returns `Unavailable` and the CLI retries unrouted | `kubectl -n openshell get endpoints <release>-replica-0 <release>-replica-1`; confirm `workload.kind=statefulset` and the GRPCRoute status is `Accepted`/`ResolvedRefs` |
10251026
| HTTPS ingress returns `Unauthenticated` after connecting | TLS terminates at Envoy, so the gateway never sees a client cert; no OIDC issuer is configured for identity | Configure `server.oidc.issuer` and register with `openshell gateway add https://<host> --oidc-issuer <url>`, or set `server.auth.allowUnauthenticatedUsers=true` for a trusted-proxy/dev cluster |
10261027
| External server `Certificate` never becomes Ready with `certManager.serverIssuerRef` set | ACME issuer rejected internal-only SANs, a loopback IP, or a `commonName` absent from the SANs | `kubectl -n openshell describe certificate openshell-server-external`; confirm `certManager.serverDnsNames` lists only real, externally-resolvable hostnames |
10271028
| Sandbox supervisors fail TLS handshake with `UnknownCA` after configuring `certManager.serverIssuerRef` | `server.grpcEndpoint` is set to the external hostname, forcing supervisors to receive the ACME cert (via SNI) which they can't verify against chart CA | Remove `server.grpcEndpoint` or set it to the internal service name; supervisors should connect via internal service name to receive the internal cert |

0 commit comments

Comments
 (0)