Skip to content

feat: apply per-query-class ClickHouse settings to endpoint queries - #4120

Open
djwhitt wants to merge 1 commit into
feat/enforced-clickhouse-endpoint-settingsfrom
feat/query-class-clickhouse-settings
Open

djwhitt wants to merge 1 commit into
feat/enforced-clickhouse-endpoint-settingsfrom
feat/query-class-clickhouse-settings

Conversation

@djwhitt

@djwhitt djwhitt commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Stack

Depends on #4052 (feat/enforced-clickhouse-endpoint-settings). This PR contains only the dependent O11Y-2661 change. Merge #4052 first, then retarget this PR to main.

Closes https://linear.app/supabase/issue/O11Y-2661/per-query-class-clickhouse-settings-for-logflare-endpoint-queries

Summary

  • Add admin-only backend query_class_settings, including a required default priority and the eight existing workload classes. Ordinary backend saves cannot change or erase policy, including through stale snapshots or backend-type changes.
  • Select policy using the original requested read-cluster label, independently of routing fallback or connection retries. Missing/unknown/unconfigured classes inherit the default; recognized class labels without dedicated routes no longer produce routing warnings.
  • Carry trusted endpoint settings separately from consumer SQL. Combine default, class, and endpoint policy, use stricter-wins resource ceilings, reject consumer overrides recursively, and inject settings once at the final ClickHouse execution boundary. Previews reuse that boundary.
  • Support positive priorities/threads, existing resource limits, fractional execution times, and only throwing overflow modes. Reject unlimited values and semantic-changing or sandbox-weakening settings.
  • Keep successful result caches class-agnostic. Refreshes retain their original class and read current backend policy. Class-only saves invalidate backend metadata without restarting ingesters or active read connections.
  • Document operator configuration, query-user/profile preflight, coordinated rollout, verification, and limitations.

Validation

  • Combined affected ten-file suite: 548 tests, 0 failures, 2 skipped (6 excluded).
  • Backend/execution regressions after mechanical lint extraction: 134 tests, 0 failures.
  • Full MIX_ENV=test ../bin/x mix ci passes: compilation, formatting, lint, security, clone ratchet, and structural checks.
  • Local integration verifies effective API settings in system.query_log.Settings, every allowlisted key under a temporary readonly = 2 user, and rejection above server profile ceilings. Temporary users are removed and an independent cleanup audit returns no residual test users.

Rollout boundaries

No production ClickHouse configuration has changed, and local checks do not establish production profile compatibility or competing-load dashboard latency. Deploy support on every instance before enabling class policy, then coordinate backend policy and the matching profile priority baseline. Do not weaken readonly/sandbox permissions to enable settings.

The existing header remains a trusted-gateway scheduling hint, not an authorization boundary. Priority does not provide tenant isolation or aggregate admission control; lower-priority work retains memory/connections while paused. Aggregate throttling (O11Y-2662) and separate API/MCP compute remain follow-ups.

@claude

claude Bot commented Oct 8, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@djwhitt
djwhitt added this pull request to stack #4121 October 8, 2026 19:04

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant