Skip to content

feat(auth): define provider-neutral authorization contracts - #144

Draft
djwhitt wants to merge 12 commits into
mainfrom
oidc-auth-contracts
Draft

feat(auth): define provider-neutral authorization contracts#144
djwhitt wants to merge 12 commits into
mainfrom
oidc-auth-contracts

Conversation

@djwhitt

@djwhitt djwhitt commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

Introduce the provider-neutral authentication and authorization contracts that the OIDC stack will build on:

  • Smolquery.Auth.Principal — a structurally validated stable identity with no provider claim bag or token material
  • Smolquery.Auth.Context — one principal, an explicit current scope, a closed capability set, and assertion expiry that is required for OIDC and optional for static credentials
  • Smolquery.Auth.Policy — fail-closed, expiry-aware authorization that preserves the 401/403 distinction and identifies unknown internal capabilities
  • Smolquery.Auth — one attachment seam for Plug connections and LiveView sockets

This is deliberately a behavior-neutral foundation. The existing static API-key and web Basic-auth flows are not rewired in this PR. OIDC discovery, JWT verification, browser redirects, role mapping, and route enforcement are also outside this PR.

Closes T-229. Refs PL-27.

Why this is a separate layer

Without an internal contract, adding OIDC directly to SmolqueryApi.Auth or SmolqueryWeb.Auth would make provider-specific claims the application's authorization model. Controllers and LiveViews would eventually learn about groups, roles, scope, email addresses, or one provider's token shape.

This layer creates the boundary first:

flowchart LR
    subgraph Authenticators[Authentication adapters — outside this PR]
        APIKey[Static API key]
        Basic[Web Basic auth]
        OIDC[OIDC / OAuth tokens]
    end

    APIKey -.-> Principal
    Basic -.-> Principal
    OIDC -.-> Principal

    Principal[Auth.Principal<br/>who authenticated]
    Principal --> Context[Auth.Context<br/>principal + scope + capabilities + expiry]
    Context --> Attach{Request carrier}
    Attach --> Conn[Plug.Conn assigns]
    Attach --> Socket[LiveView.Socket assigns]
    Conn --> Policy[Auth.Policy]
    Socket --> Policy
    Policy --> App[Controllers / LiveViews]
Loading

Authentication adapters are responsible for interpreting provider-specific inputs. Consumers of this contract receive a Principal and ask Policy about a normalized capability.

Design

Principal: identity without provider leakage

Smolquery.Auth.Principal carries:

Field Meaning
id Stable opaque identifier used by smolquery
authn :oidc, :api_key, or :basic
kind :user or :service
issuer / subject Canonical external identity for OIDC principals
display_name Optional descriptive metadata; never identity or authorization input
client_id Optional calling-client metadata

There is intentionally no field for raw tokens, refresh tokens, provider claims, email, username, groups, or roles.

OIDC identity

The canonical external identity is the exact pair {issuer, subject}. Neither value is trimmed, case-folded, or otherwise normalized.

Principal.oidc/4 derives an opaque, stable pseudonymous identifier using a versioned encoding:

oidc:v1:<base64url(sha256(
  uint32be(byte_size(issuer)) || issuer ||
  uint32be(byte_size(subject)) || subject
))>

Length framing prevents delimiter ambiguity, so {a\0b, c} cannot collide at the encoding layer with {a, b\0c}. Lengths are unsigned big-endian 32-bit byte counts; the full SHA-256 digest uses unpadded base64url. The v1 prefix makes the durable derivation explicit. A fixed test vector prevents accidental changes to this encoding.

Principal.well_formed?/1 recomputes the OIDC ID from issuer and subject; a struct with a substituted ID is structurally rejected. This check does not prove authentication provenance. Only trusted authentication adapters may construct principals, and client input is never decoded directly into these structs.

Smolquery treats differing exact {issuer, subject} pairs as distinct identities, including pairwise subjects issued to different clients or sectors. It never links identities through mutable display claims such as email or username. The derived ID relies on SHA-256 collision resistance and is pseudonymous audit data, not anonymous or categorically non-PII.

Local identities

Principal.local/4 represents credentials managed by smolquery. It accepts only :api_key or :basic; callers cannot create a caller-selected :oidc identity. The caller supplies a stable non-secret source key such as static:api, never the credential itself. The constructor does not retain that source key; it derives the final ID in an authentication-specific namespace:

api_key:v1:<base64url(sha256(frame("api_key:v1:", source_key)))>
basic:v1:<base64url(sha256(frame("basic:v1:", source_key)))>

The disjoint constructor-owned namespaces prevent local identities from colliding with OIDC or with one another. Local ID validation also requires one canonical unpadded base64url encoding.

Both constructors:

  • return tagged results;
  • require non-empty binary identifiers;
  • accept only display_name and client_id options;
  • reject duplicate or unknown options;
  • reject invalid kinds, authentication methods, and metadata.

Context: authorization is scoped

Smolquery.Auth.Context combines:

%Smolquery.Auth.Context{
  principal: principal,
  scope: :single_tenant,
  capabilities: MapSet.new([:query, :ingest]),
  expires_at: 1_786_800_000
}

The capability vocabulary is closed:

Capability Intended boundary
:web_access Enter the human UI
:query Read metadata and run/read query jobs
:ingest Insert and load rows
:catalog_manage Create datasets/tables and update table policy
:platform_operate Fleet-wide kill, restart, and drain operations

:platform_operate is deliberately separate from tenant-administrator permissions.

Capabilities are stored in a MapSet, so duplicates collapse. Unknown atoms and strings fail construction rather than becoming dynamic atoms or implicit grants. granted?/2 also fails closed for unknown capabilities and malformed contexts.

expires_at is a Unix epoch time in integer seconds and belongs to the request or session context rather than the stable principal identity. It is required for OIDC contexts and optional for local static contexts, which may use nil. Context.active?/2 uses the strict boundary now < expires_at; the context is expired at equality. OIDC adapters must validate token timestamps and apply their bounded clock-skew policy before constructing the context. Policy rechecks the absolute expiry on every authorization decision so a long-lived LiveView socket cannot retain capability access indefinitely.

The single-tenant sentinel

The current scope is explicitly :single_tenant. It is not derived from:

  • OIDC issuer;
  • email domain;
  • provider organization;
  • groups or roles.

This makes the scope explicit while leaving a replacement seam for a membership lookup:

(principal_id, tenant_id) -> tenant-scoped capabilities

The PR does not make catalog keys, storage paths, query engines, or datasets tenant-isolated.

Policy: preserve authentication versus authorization

Smolquery.Auth.Policy.authorize/2 has one fail-closed decision path:

flowchart TD
    Start[authorize context, capability] --> Active{Well-formed active context?}
    Active -- No --> Unauthenticated[error: unauthenticated<br/>HTTP adapter maps to 401]
    Active -- Yes --> Known{Known internal capability?}
    Known -- No --> Invalid[error: invalid_capability<br/>programmer/configuration defect]
    Known -- Yes --> Granted{Capability granted?}
    Granted -- Yes --> Allowed[ok]
    Granted -- No --> Forbidden[error: forbidden<br/>HTTP adapter maps to 403]
Loading

The domain layer returns tagged decisions rather than HTTP responses. At HTTP integration boundaries, these decisions map as follows:

  • :unauthenticated to 401;
  • :forbidden to 403;
  • :invalid_capability to an internal programming/configuration failure, never an end-user denial.

A missing, malformed, or expired context is not treated as an authenticated principal with insufficient permissions; it remains unauthenticated. authorize/2 reads Unix time from the system clock, while authorize/3 accepts an explicit timestamp for deterministic tests.

One Plug and LiveView attachment seam

Smolquery.Auth stores a well-formed context under the stable assign key:

:smolquery_auth_context

The same API works for both request carriers:

conn = Smolquery.Auth.assign_context(conn, context)
{:ok, context} = Smolquery.Auth.fetch_context(conn)

socket = Smolquery.Auth.assign_context(socket, context)
{:ok, context} = Smolquery.Auth.fetch_context(socket)

This avoids putting identity into conn.private, which already carries API runtime-instance state, and provides the same normalized attachment contract for Plug connections and LiveView sockets. Route and mount enforcement remain outside this PR.

Assignment rejects malformed contexts. Fetching a missing or malformed assignment returns :error. These attachment checks prove structure only; they do not authenticate a caller or authorize a capability. Trusted adapters establish authentication provenance before assignment, and every access decision goes through Policy, which checks expiry and capabilities. Regression coverage includes partial struct-tagged principal maps and malformed MapSet internals so structural checks fail closed without raising.

Security properties

  • Exact {issuer, subject} is the OIDC identity; display metadata cannot change or merge it.
  • Derived IDs are opaque stable pseudonymous identifiers, use disjoint authentication namespaces, and are treated as potentially sensitive audit data.
  • Raw tokens and arbitrary provider claims have no place in the principal struct.
  • Authentication methods and capabilities are closed atom sets; request data is never converted with String.to_atom/1.
  • Unknown capability grants fail construction; unknown capabilities requested by trusted application code are distinguished as internal defects.
  • Substituted OIDC IDs, invalid local ID encodings, partial struct-tagged maps, invalid scopes, malformed capability sets, and malformed assignments fail structural checks without raising.
  • Missing, malformed, or expired identity remains distinguishable from insufficient permission.
  • Structural checks do not establish authentication provenance; only trusted adapters construct and attach contexts.
  • Principal and tenant identifiers are not introduced as telemetry labels.

What this PR does not do

  • No OIDC discovery or JWKS cache
  • No JWT signature or claim validation
  • No authorization-code/PKCE browser flow
  • No provider-role or scope mapping
  • No route or LiveView authorization changes
  • No job ownership or actor persistence
  • No tenant provisioning or data isolation
  • No change to internal-secret, gen_rpc TLS, or Erlang-distribution TLS boundaries

Those concerns are outside this PR so the provider-neutral contract remains independently reviewable.

Validation

  • 31 focused auth-contract checks pass
  • Checks (mix ci), the fast test suite, and Dialyzer pass
  • Integration and cluster workflows pass

The tests cover:

  • stable OIDC ID derivation and a fixed compatibility vector;
  • stable local ID derivation, fixed API-key and Basic compatibility vectors, and disjoint API-key, Basic, and OIDC namespaces;
  • canonical local base64url encoding and repeated-prefix rejection;
  • cross-issuer and length-boundary identity separation;
  • display metadata not affecting or merging identity;
  • invalid constructors and structurally inconsistent principals;
  • capability normalization, deduplication, and rejection;
  • strict expiry behavior before and at the timestamp boundary;
  • partial struct-tagged principal maps and malformed context and MapSet internals failing closed without raising;
  • 401, 403, and invalid-internal-capability policy decisions;
  • Plug and LiveView assignment round trips and malformed-assignment rejection.

Stack created with GitHub Stacks CLIGive Feedback 💬

Add validated principal and single-tenant context types, a closed capability policy with distinct unauthenticated and forbidden results, and shared Plug/LiveView attachment seams. OIDC identities derive a versioned opaque id from the exact issuer and subject without retaining raw claims or token material.

Closes T-229. Refs PL-27.
Namespace local principal identities, separate assertion expiry from stable principals, enforce expiry during policy decisions, and document the structural-validation and pseudonymous-identifier boundaries.

Refs T-229 and PL-27.
Fail closed when an OIDC-derived authorization context omits assertion expiry, and strengthen the identity framing regression coverage.

Refs T-229 and PL-27.
Reject partial struct-tagged maps without raising through context attachment and authorization paths, and pin the durable Basic-auth principal ID derivation with a compatibility vector.

Refs T-229 and PL-27.
Describe the shared Plug and LiveView storage location as a stable assign key rather than implying that it uses conn.private.

Refs T-229 and PL-27.
@djwhitt
djwhitt marked this pull request as ready for review August 15, 2026 15:43
djwhitt and others added 6 commits August 15, 2026 17:29
Bring the published authorization-contract layer up to date with origin/main without rewriting its existing history.
…-262) (#166)

* Derive compact_max_rows from the compaction engine's memory budget (T-262)

A fixed 4Mi-row cap OOMs a 1 GiB compaction engine: the sandbox measured
a 4Mi-row group pinning the whole budget during the merge, with
preserve_insertion_order already off. A group the engine cannot merge
re-plans identically every sweep, so one group blocks its table forever.
Splitting the group does not converge — the halves' outputs re-enter the
next group up to the same cap.

The cap now derives from the budget at 512 bytes per row, half the
measured pin rate. The sandbox's 1 GiB budget yields 2Mi rows. An
explicit SMOLQUERY_COMPACT_MAX_ROWS still wins, a tiny budget floors at
64Ki rows, and no resolvable budget keeps the old 4Mi default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GFEnyub9GiAqwDWRRtMKBs

* The row cap adapts per table: halve on merge OOM, recover on success (T-262)

A budget-derived cap fixes the budget dimension and not the workload one.
The 512 B/row constant is calibrated on this bench's data; wide repetitive
text fields pin kilobytes per row while compressing well enough to pass
both static caps. No footer statistic predicts pin cost across workloads,
so the cap answers the workload instead: a merge OOM halves the table's
cap (floor 64Ki rows), a successful compaction doubles it back, and the
override is shed at the resolved cap. Caps live in the compactor's state;
a restart re-learns them on the next OOM.

The sweeper contract grows an optional three-tuple so a sweep can carry
state; retention's two-tuple form is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GFEnyub9GiAqwDWRRtMKBs

* Hoist split_result out of the Sweeper quote for dialyzer

Injected into each using module, the helper's unused clauses are dead
code per expansion — the compactor never returns a two-tuple error, the
retention sweeper never returns three-tuples — and dialyzer rightly
flags patterns that can never match. As one shared function the clauses
are all reachable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GFEnyub9GiAqwDWRRtMKBs

* Alias the Sweeper in its own quote for credo

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GFEnyub9GiAqwDWRRtMKBs

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The write path's 429s were correct and too late: buffer_full and
{:overloaded, _} fire after the request body is in RAM, so nothing
bounded resident bodies and memory grew with client concurrency until
the kernel OOMKilled the pod — three kills in one in-region sweep at
up to 128 VUs x 6.87 MiB.

SmolqueryApi.Admission counts POST insert/load bodies against an
in-flight byte limit between auth and parsing, while the request is
still a header. The reservation is the declared content-length capped
at the route's own body limit; no declared length reserves the limit
outright. Over the limit answers 429 with retry-after; an idle counter
always admits one request, so a limit below one body cannot brick
ingest. The counter releases when the response sends, and a monitor
releases on request crash.

The limit derives as a quarter of the cgroup memory limit, floored at
one NDJSON body; SMOLQUERY_INSERT_MAX_IN_FLIGHT_BYTES overrides it.


Claude-Session: https://claude.ai/code/session_01GFEnyub9GiAqwDWRRtMKBs

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Review fixes for the admission stack (T-264, T-265, T-266)

- Compactor: a staging-phase OOM now tightens the row cap; only the store-wrapped shape matched before, so the same group re-OOMed every sweep.
- Compactor: a cap raise now needs evidence — a cap-filling success or a cap-forced skip — and `patience` such sweeps; each OOM doubles the patience, so a table at its true limit probes it rarely instead of every second sweep, and a cap wedged at the floor unwedges.
- Compactor: a timed-out final COPY (store-wrapped CallExited) now recycles the wedged compaction engine.
- Compactor: `compact_max_rows` resolves once at start, not on every sweep.
- Storage runtime: the row cap derives from the cgroup bytes directly, with no string round-trip; fractional budgets like "1.5GB" parse; an unreadable explicit budget logs a warning instead of a silent fallback.
- Admission: a stray message no longer crashes the server (and, under rest_for_one, the Endpoint with it).
- Admission: a server that dies between the lookup and the call passes the request uncounted instead of a 500.
- API runtime: an explicit `insert_max_in_flight_bytes` is validated at boot; the derivation floor reuses `InsertController.max_ndjson_bytes/0`.
- Whole-request load reservations stay by design; the moduledoc now states the tradeoff.
- Docs: the `SMOLQUERY_COMPACT_MAX_ROWS` derivation, the new `SMOLQUERY_INSERT_MAX_IN_FLIGHT_BYTES` row, and the admission 429 in api.md.

Refs T-264, T-265, T-266.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* The row cap starts at the default, not at a bench-derived pin rate (T-266)

- The 512 B/row figure came from a bench with very large rows. A cap derived from it starves typical workloads.
- An unset `compact_max_rows` now starts at `4194304` rows. The per-table adaptation — halve on OOM, earn it back with evidence — fits the cap to each workload instead.
- This removes the budget-string parser and the per-sweep cgroup reads with it; `with_compact_max_rows/1` no longer takes a cgroup argument.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* One helper owns the 429 shed-load contract (T-265)

- `Errors.send_resource_exhausted/3` pairs the 429 envelope with its retry-after.
- The four hand-built copies — admission, buffer full, overload, job ceiling — now call it, so the contract cannot drift.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@djwhitt
djwhitt marked this pull request as draft August 16, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants