Skip to content

[Feature] BanyanDB: BydbQL surface, integration fixtures, docs for tag aggregation #14091

Description

@hanahmily

Part 4 of 5 of the tag aggregation and time bucketing design, shipped in apache/skywalking-banyandb#1360.

Design references: §5.4 BydbQL · §10 Compatibility (limit) · §11 Testing strategy · §12 Delivery order (stages 7–8)

Depends on: #14089 (time-bucket grouping) and #14090 (COUNT_DISTINCT) — this issue makes both reachable from BydbQL and proves them end to end together.

Scope

BydbQL

  • COUNT(DISTINCT col) — DISTINCT valid inside COUNT only; SUM(DISTINCT col) is a parse error.
  • Aggregate column resolves against tags as well as fields (convertAggregation currently hard-errors outside allFields).
  • TIME_BUCKET(<duration>) pseudo-column, valid only in GROUP BY, at most once, argument optional (empty → the measure's own interval); not selectable in the projection.
  • bydbctl TUI planner constant so the plan renders the new function/pseudo-column.

Integration

  • The load-bearing multi-bucket cardinality fixture (one user × 3 api-keys / 3 hour buckets, a second user × 2), asserting the scalar case and the correct order for the ranked+paged case — run in both test/integration/distributed/query/vectorized_test.go (liaison + 2 data nodes) and standalone.
  • The same fixture reused at a second bucket width, to catch a bucket key that's computed but ignored.
  • The decomposition oracle: a bucketed query over a window must equal the union of per-bucket queries over the same window, row-for-row on (bucket, tags…) — each per-bucket query is independently trustworthy, so this is a real oracle rather than a golden file.
  • data.go's golden comparison currently ignores timestamps; these new cases need it included, or every bucket silently collapses to zero and the suite still passes.
  • The generated feature matrix (test/cases/measure/cmd/generate/layer3_features.go, ql_renderer.go) gains the sixth aggregate function — it's hard-coded today and won't be caught by the exhaustive linter (the switches have default branches).

Docs

  • docs/interacting/bydbql.md: the new grammar, and prominently, the limit-with-bucketing note (default 100 truncates a bucketed series — this must be visible in the docs, not just the API reference).
  • docs/api-reference.md regenerated, never hand-edited.

Out of scope

  • The optional local-scan performance follow-up (run-folding, metadata-only scan) — separate issue.

Test / Definition of done

  • Parser: COUNT(DISTINCT x) parses; SUM(DISTINCT x) errors; TIME_BUCKET()/TIME_BUCKET('5m') both parse; a second TIME_BUCKET in one GROUP BY errors; TIME_BUCKET in SELECT errors.
  • Binder/transformer protocmp equivalence tests against the equivalent hand-built QueryRequest.
  • bydbctl planner test rendering a query using both features.
  • The integration fixtures above green in standalone and distributed CI.
  • Doc diff reviewed specifically for the limit callout.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    databaseBanyanDB - SkyWalking native databasefeatureNew feature

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions