Skip to content

feat: server-selected Kubernetes namespace in run and app specs - #8300

Merged
EngHabu merged 9 commits into
mainfrom
haytham/k8s-metadata-run-app-spec
Nov 3, 2026
Merged

EngHabu merged 9 commits into
mainfrom
haytham/k8s-metadata-run-app-spec

Conversation

@EngHabu

@EngHabu EngHabu commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Why are the changes needed?

A control plane may need to choose the Kubernetes namespace a run's or an app's pods are created in, for example to place a project's workloads in a dedicated namespace. Once chosen, the namespace must stay fixed while the workload runs:

  • Runs: for the run's whole life. If the rule that picked it changes mid-run, the run's remaining actions and finalizers still go where it started.
  • Apps: while the app runs. A new choice can apply the next time the app is turned on.

That needs a field where the server records the namespace it selected, so the choice persists with the run or app spec and reaches every action.

What changes were proposed in this pull request?

  • New flyteidl2.common.K8sMetadata (flyteidl2/common/kubernetes.proto): Kubernetes metadata the server sets for a run or app. Its one field, namespace, requires min_len = 1 when the message is present.
  • RunSpec.k8s_metadata = 21: set when the run is created and fixed afterwards; it applies to every action and finalizer.
  • app.Spec.k8s_metadata = 17: set when the app is created or turned on, and fixed while it runs.
  • An absent message keeps today's namespace resolution, so existing clients and stored specs behave as before.
  • Regenerated Go (including validators), Python, Rust and TypeScript code, and the gateway swagger.

The field is server-owned: a server should ignore any value a client sends and set it itself.

How was this patch tested?

  • make buf outputs regenerated for every language; check-generate passes.
  • A downstream service consumed these types at this branch's head: CreateRun and app create/update setting the field, and workers placing pods in the recorded namespace. Its unit and end-to-end tests pass.

Labels

  • added

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

🤖 Generated with Claude Code

EngHabu and others added 9 commits October 31, 2026 12:44
Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
(cherry picked from commit 0acbda7)
(cherry picked from commit c44d1e6)
Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
(cherry picked from commit 08e739f)
(cherry picked from commit c45bd10)
Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
(cherry picked from commit 8e5b7c7)
(cherry picked from commit 0d28f8b)
Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
(cherry picked from commit 6b1b0a7)
(cherry picked from commit 1fa5084)
Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
(cherry picked from commit 11b62b8)
(cherry picked from commit 4e7393d)
Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
(cherry picked from commit 412c5c2)
(cherry picked from commit 8822893)
Regenerating on CI (check-generate) reproduces main's validate_pb.ts; the
earlier regeneration here had changed one line of it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EuLd1BpsMEzMwhnd2bYhsK
Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
check-generate's make buf adds the field to gen/rust/src/flyteidl2.task.rs,
which the earlier Rust regeneration had missed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EuLd1BpsMEzMwhnd2bYhsK
Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
…d on

The run and app fields are the metadata message, of which the namespace is
the one field today. An app's namespace is set when it is created or turned
on and is fixed only while it runs, so a binding changed while it is off
applies at its next start.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EuLd1BpsMEzMwhnd2bYhsK
Signed-off-by: Haytham Abuelfutuh <haytham@union.ai>
@github-actions github-actions Bot added the flyte2 label Nov 2, 2026
@EngHabu EngHabu added the added Merged changes that add new functionality label Nov 2, 2026
@EngHabu
EngHabu merged commit 5f50364 into main Nov 3, 2026
48 of 51 checks passed
@EngHabu
EngHabu deleted the haytham/k8s-metadata-run-app-spec branch November 3, 2026 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

added Merged changes that add new functionality flyte2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants