Conversation
Store fields declared with a calculatedFn compute their values on the client at read time, through lazy prototype getters on record data objects - never stored, parsed, or included in the value-equality and digest comparisons used to detect unchanged records, and always current when read. - Added shared getter-class util in data/impl/CalculatedFieldSupport.ts, generalizing the exposed-leaf technique from RowDataGenerator for consumption by both the Store and (upcoming) Cube layers. - projectionOnly stores adopt raw rows via a generated wrapper class (data !== raw) carrying source-field and calculated getters. - Calculated fields are read-only: modifyRecords() throws on writes, and they are excluded from getModifiedValues() and all parse paths. - GridModel-driven grids auto-refresh columns displaying calculated fields after each transaction, repainting visible cells whose value moved via inputs outside their own row (e.g. a summary denominator), and never suppress re-sorts on calculated sort columns. - A FieldFilter testing a calculated field upgrades incremental refilters to full, keeping filter membership current as external inputs move. - Data-object generation is a re-runnable function of Store state, anticipating future runtime updates to calculated field specs.
Cube-layer form of calculated fields, computed at read time on View rows with the View's AggregationContext. Because such fields carry no aggregator, Views using them for globally-dependent values (e.g. pct-of-total) never trip aggregatorsAreSimple and retain the incremental data-only update path. - RowDataGenerator installs calculated getters on both generated data classes, reading the View's current AggregationContext live. Aggregate/bucket row data moves from a spread-cloned template to a generated class, keeping parent rows on one fixed shape. - AggregationContext.filteredRecords is now readable by calculated field functions and stays fresh on data-only ticks - rebuilt lazily from the View's leaf map (patched fresh each tick) and memoized per context. - applyLeafDataUpdate diffs only non-calculated fields. - Validations: calculatedFn is mutually exclusive with aggregator/canAggregateFn/isDimension; BucketSpec.dependentFields may not name calculated fields; stores connected to a View with calculated fields must set projectionOnly (upgraded from warning to throw). The View marks its calculated field names on connected stores, wiring them into the Store-layer grid repaint and filter refresh handling.
Dense record data on stores with calculated fields is now built from a generated class - constructor-assigned slots for id and every non-calculated field in one fixed order, calculated getters on the prototype - rather than falling back to the sparse form. All instances share a single shape, and constructor slack tracking keeps them in V8's fast-properties mode past the plain-object add limit behind denseRecordThreshold, so huge parsed stores get packed memory and fast access alongside calculated fields. Stores without calculated fields keep the existing template spread-clone unchanged.
- Columns bound to calculated fields are never editable - gated in Column.isEditableForRecord via Store.calculatedFieldNames, covering View-published fields whose grid-store Field is not itself marked calculated. - Restore grid sort order on empty-transaction syncs (e.g. summary-only updates) when sorted on a calculated column. - Clear View-marked calculated field names from connected stores on View.destroy(), so a store outliving its View stops paying repaint/refilter handling. - Doc guidance: calculated values are read by name (own-property enumeration, spread and JSON.stringify do not see them - use StoreRecord.getValues()), and calculatedFns should return primitives or stable references so grid change detection can skip unchanged cells.
…ield refinements (#4620) - Stores connected to a Cube View now default to projectionOnly: true when unset - an explicit false or a processRawData function opts out. Added internal Store.setProjectionOnly() flipping mode post-construction, with CHANGELOG breaking entry and new v88 upgrade notes. - Empty-sync sort restore now routes through GridTransactionManager.noteSortStale(), coalescing with the paced deferred-sort flush instead of an immediate unscheduled full sort. - AggregationContext now takes a lazy filtered-records source from its View, replacing the recordsAreStale flag - View._leafMap returns to private. - View partitions calc/non-calc fields once in buildIndices (now ordered before the row data generator), consumed by RowDataGenerator and syncStoreCalculatedFields. - Bucket dependentFields validation runs once per newly-seen name rather than on every bucketing pass. - Trimmed changelog to an app-developer perspective; condensed comments throughout.
Pulls forward the full form of the earlier defaulting change: View.parseStores now places every connected store into projection mode itself, throwing on an explicit `projectionOnly: false` or a `processRawData` config. The calculated-field-conditional guard in syncStoreCalculatedFields is gone along with the opt-out - parse mode was offering connected stores nothing that worked (parseVal is a no-op on already-typed view rows, and updateData never applied processRawData, so transforms were stale on every incremental update). Changelog breaking entry, v88 upgrade notes, and docs updated to match.
…option (#4620) - Store.connectView(view): validates conflicting config (digestSpec, idEncodesTreePath, projectionOnly: false, processRawData), installs the View's row digest, adopts projection mode, and marks the View's calculated field names. Idempotent - the View calls it at connection, on setStores, and on query changes. - Store.disconnectView(): clears connected-View state on View destroy - and View.setStores now disconnects outgoing stores, a pre-existing gap. - Folds setProjectionOnly into connectView (sole caller); setExternalCalculatedFieldNames becomes private; View.parseStores and syncStoreCalculatedFields deleted.
The module hosts both calculated-field getters and source-field (`_src`) getters - the old name was specific to the first.
…re projection (#4620) - Condensed FieldSpec.calculatedFn JSDoc (46 -> 28 lines), same content packed. - data/README projectionOnly section updated - connected stores are always projections set by the View, not a warned recommendation. - Small code tidies in Store/StoreRecord.
Recomputed only when the filter or calculated field names change, rather than re-flattening the filter on every transaction.
Record data generation - shared defaults/template plus the generated dense and projection classes carrying calculated field getters - moves to data/impl/RecordDataGenerator, mirroring the Cube's RowDataGenerator. Store.generateDataConfig() now just rebuilds the field map and generator. Net -119 lines in Store.
A connected store's field metadata now flows from its View rather than being independently (and typically more weakly) declared. Store.connectView merges the query's own CubeField instances into store.fields - types, displayNames and calculated status carry through to everything reading field metadata off the store - superseding same-named app or grid-inferred fields, while preserving app-declared extras (store-layer calculated fields composed over view rows, infra fields such as cubeDimension). disconnectView restores the store's own declared set. - Deletes the calc-names side channel (setExternalCalculatedFieldNames / _externalCalculatedFieldNames and the union in calculatedFieldNames) - the behavioral set now derives purely from store.fields, while the evaluation set (RecordDataGenerator) stays store-layer-only as before. - A store-layer calculatedFn field sharing a view field's name throws at connection - both would claim to compute the value. - Projection wrappers now expose source getters for every view-published field, not just those the app happened to redeclare. - Reconciliation is idempotent and skips regeneration when the field set is unchanged, so per-updateQuery calls stay cheap.
Laziness dated from the two-input era (own fields + view-marked names invalidating at different times) - with reconciled fields there is one choke point, so build there.
- New StoreConfig.view: the store is built as a projection of the View's rows - projectionOnly + cubeRowDigest set automatically, query fields merged at construction, and the store registers with (and is loaded by) the View. Conflicting config throws at construction. - Removes ViewConfig.stores, Cube.createView stores param, View.setStores, and Store.connectView/disconnectView - replaced by construction-time connection plus View.addStore/removeStore for transient detach (config and data retained; store re-attaches only to its own View). - View.updateQuery reconciles member store fields via Store.reconcileFields; forceRegen and mode-flip logic deleted - the generator is built once, correctly, at store construction. - CHANGELOG, v88 upgrade notes, and README examples updated to the new flow.
# Conflicts: # CHANGELOG.md # data/cube/Cube.ts # data/cube/View.ts
6 tasks done
lbwexler
changed the title
lbwexler
marked this pull request as ready for review
August 31, 2026 23:06
…kyt4eq # Conflicts: # CHANGELOG.md # data/cube/README.md
6 tasks done
v88.0.0 shipped on develop while this PR was open, so its breaking changes and upgrade notes move to a new 89.0.0-SNAPSHOT changelog section and docs/upgrade-notes/v89-upgrade-notes.md. develop's v88 upgrade notes, doc index row and registry entry are taken as-is. Code resolutions: - Field constructor keeps both develop's parsed defaultValue and this branch's calculatedFn. - LeafRow.applyLeafDataUpdate diffs develop's checkFields (#4723); View.dataOnlyUpdate now derives them from _nonCalcFields so calculated fields are never diffed.
Cube README gains a worked example restating a weighted average as two SUM fields plus a calculatedFn ratio, keeping the View on its incremental data-only update path (per discussion on #4747). The Custom Aggregators rules and the CubeFieldSpec.calculatedFn doc point to it, and the CHANGELOG entry names ratios of sums as a target use-case.
Member
Author
|
Closing in favor of #4701 (derived fields), which covers the calculated-field use cases here - including the weighted-average case from #4747 - in a much smaller change. A Pieces worth lifting into #4701 are tracked in #4748. Field reconciliation for connected stores and |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #4620 (calculated fields at both data layers) and #4637 (connected-store field reconciliation), grown through review into a rework of how stores connect to Cube Views. Supersedes #4636, whose full history this branch contains.
Calculated fields (#4620)
FieldSpec.calculatedFn: (data, store) => any- Store fields computed on the client at read time via lazy prototype getters on recorddata. Never stored, parsed, or included in equality/digest comparisons; always current, even when inputs live outside the row (e.g. aStore.summaryRecordsdenominator). Read-only: bound grid columns are never editable andmodifyRecords()throws.CubeFieldSpec.calculatedFn: (row, ctx) => any- the Cube-layer form, computed on View rows with theAggregationContext. Carries no aggregator, so pct-of-total-style Views retain the incremental data-only update path.ctx.filteredRecordsis readable from these fns and always current.dependsOnChildrenOnly: false, forcing every update through a full O(N) rebuild. With this PR it restates as two built-inSUMs over leaf terms (qty,priceQty) plus acalculatedFnratio on every row - O(depth) per leaf update, no aggregator state, no protocol changes toAggregator/RowUpdate. The Cube README carries the worked example; the residual gap (leaf-level derived fields that feed aggregators) is discussed on Support incremental aggregators that read other fields on the row (e.g. weighted average) #4747.FieldFilteron a calculated field upgrades incremental refilters to full; sorts on calculated columns are restored through the paced deferred-sort flush, including on empty (summary-only) syncs.data/impl/RecordDataGenerator.ts, mirroring the Cube'sRowDataGenerator; both consume the shared getter utildata/impl/FieldGetterSupport.ts. ParentViewRowDatarows are themselves now generated-class instances; dense record packing is retained on calculated stores.💥 Connected stores: construct with
StoreConfig.view(#4637)projectionOnly+cubeRowDigestset automatically), with the View's query fields merged intofieldsand the store registered with (and loaded by) the View. Conflicting config (projectionOnly: false,processRawData,digestSpec,idEncodesTreePath) throws at construction.CubeFieldinstances: types,displayNames, and calculated status flow through to everything reading field metadata off the store, superseding same-named app or grid-inferredtype: 'auto'fields, while preserving app-declared extras (store-layercalculatedFnfields composed over view rows, infra fields likecubeDimension).View.updateQueryre-reconciles.ViewConfig.stores, theCube.createViewstoresparam, andView.setStores.View.addStore()/removeStore()support transient detach/re-attach (a store retains its config and data, and re-attaches only to its own View);View.disconnect()/connect()remains the idiom for pausing updates, e.g. on hidden tabs.calculatedFncollision throws);CubeFieldinstances shared by reference; the store-view connection is fixed for the store's lifetime.See the CHANGELOG breaking section and new
docs/upgrade-notes/v89-upgrade-notes.mdfor migration detail. (v88.0.0 shipped while this PR was open - its changelog entries and upgrade notes now target89.0.0-SNAPSHOT.) Follow-on filed as needed from review.Companion Toolbox PR: xh/toolbox#896.
Hoist P/R Checklist
developbranch as of last change.breaking-changelabel + CHANGELOG if so.data/README.mdanddata/cube/README.mdsections.)