Skip to content

feat: add optional admin HTTP server on separate port#5191

Open
simonswine wants to merge 2 commits into
grafana:mainfrom
simonswine:cw/admin-server
Open

feat: add optional admin HTTP server on separate port#5191
simonswine wants to merge 2 commits into
grafana:mainfrom
simonswine:cw/admin-server

Conversation

@simonswine

@simonswine simonswine commented May 26, 2026

Copy link
Copy Markdown
Contributor

Adds an optional secondary HTTP server (-admin-server.mode=disabled|additional|exclusive) that exposes operational endpoints (metrics, pprof, rings, config, admin/ops) on a dedicated port (default localhost:4042), so they can be firewalled independently from the public API port.

This is step one and a couple of extra things needs adressing:

  • Include this in the helm chart so we can roll this out
  • Decide how to handle query diagnostics, they currently sit weirdly in between admin and normal query API

@simonswine simonswine marked this pull request as ready for review May 27, 2026 08:01
@simonswine simonswine requested review from a team as code owners May 27, 2026 08:01
Comment thread pkg/api/api.go
}

func (a *API) RegisterReadyHandler(handler http.Handler) {
a.RegisterRoute("/ready", handler, WithMethod("GET"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was moving /ready onto the admin server intentional, or a side effect of routing everything non-data-plane through registerAdminRoute?

In exclusive mode it's removed from the main port and only served on the admin server (defaults to localhost:4042), so the helm readiness probe on the main service port would 404 and pods would never go Ready.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that was intentional as readiness error messages might return errors not to be seen for the public, there needs to be helm changes for metrics/profile collection anyhow in exclusive mode, so that is totally fine.

@simonswine simonswine requested a review from marcsanmi June 9, 2026 11:35
Adds an optional secondary HTTP server that exposes operational
endpoints (metrics, pprof, admin/ops, rings, config) on a separate
port, so they can be firewalled independently from the public-facing
API port.

Flags:
  -admin-server.mode              disabled|additional|exclusive (default: disabled)
  -admin-server.http-listen-address  (default: localhost)
  -admin-server.http-listen-port     (default: 4042)

Modes:
- disabled:   no change, all routes on main port (backward compatible)
- additional: admin server starts, operational routes served on both ports
- exclusive:  admin server starts, operational routes removed from main port

Routes moved to admin server:
- /metrics, /debug/pprof (exclusive mode only)
- /admin index page, /ready, /runtime_config
- /api (grpc-gateway: status/config/buildinfo)
- /debug/fgprof
- All ring pages (/ring, /distributor/ring, /store-gateway/ring, etc.)
- /memberlist, /overrides-exporter/ring, /compactor/ring
- /ops/object-store/* (admin ops)
- /metastore-nodes, /metastore-client-test
- /ring-segment-writer
- /query-diagnostics/*

The main server /admin page retains swagger and UI links.
The admin server /admin page shows operational links only.
Static assets (/static/, /assets/) are served on both servers
so CSS/JS works for query-diagnostics pages on the admin port.

Uses a plain net/http server (not dskit) wrapped as a dskit IdleService
for lifecycle management, with a gorilla mux router for correct
path parameter handling.

@marcsanmi marcsanmi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants