Skip to content

[Feature] BanyanDB: local-scan performance follow-up for tag aggregation (optional) #14092

Description

@hanahmily

Part 5 of 5 of the tag aggregation and time bucketing design, shipped in apache/skywalking-banyandb#1360.

Design references: §7.7 The local scan: entity and indexed tags never touch the data file · §12 Delivery order (stages 8a–9, both marked optional in the design)

Depends on: #14089 (time-bucket grouping) and #14090 (COUNT_DISTINCT) — this optimizes both. Not required for either to ship; the design explicitly defers this and asks that the payoff be measured before it's built.

Scope

Phase A — run-folding (design's "intermediate step," no new execution path)

  • Fold each maximal (series, bucket) span once instead of per row, using a plan-time flag marking which key columns are index-resolved — derived from searchSeriesList's existing three-way projection split, never inferred at runtime by sampling column values.
  • Applies whenever an index-resolved tag is a group key: COUNT_DISTINCT collapses n − 1 redundant set inserts per span into one; COUNT becomes count += spanLength.

Phase B — metadata-only scan (after Phase A lands)

  • Bypass block decode entirely when every tag the query touches (group keys, aggregation target, any tag a residual predicate touches) resolves from the series index — gated by the range-containment guard (a block whose [min,max] isn't fully inside the query range must still read timestamps.bin) and the no-residual-predicate guard.
  • index_mode measures are the structural first case: they hold no part-resident tags at all, so an index-mode aggregation is index-only by construction.

Out of scope

  • Anything that changes correctness or the query surface — this issue is pure performance, gated by differential tests against the existing (already-correct) path.

Test / Definition of done

  • Phase A: differential test against the unoptimized per-row path on the same fixture (identical results, fewer group-map probes); a magnitude assertion (an 8192-row block at hour buckets collapses to ~137 spans, not 8192 key computations).
  • Phase B: differential test against the decoding path across both index-mode and part-backed fixtures (identical results); the range-containment fixture — a block with rows at 09:00 and 09:59 (one hourly bucket) queried over 09:30–09:31 must yield no bucket row, while the same block fully inside the query range takes the shortcut.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    databaseBanyanDB - SkyWalking native databasefeatureNew feature

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions