Repository navigation
fix(runner): attribute example.saved to its source example, batch rollup inserts (DEV-3146) - #400
Merged
Merged
Conversation
… rollup inserts (DEV-3146) The Examples funnel's saved column was always 0: saves carried kind=saved with the demo id, and the panel filters kind=docs. The API worker now resolves the row's forked_from to the source example (docs guide/area from a bundled generated taxonomy, starter, import, payload), following saved-demo hops. writeExampleDaily used one statement per row; D1 meters each batch() statement against its 1000-query limit. Rows now go in as 8-row INSERTs in the same batch, and an oversized day throws before the DELETE. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Resolve the docs taxonomy per bucket (overrides over a newest-bucket default, numeric bucket order), run the lineage walk inside the waitUntil chain after the D1 update, fix the hop-cap off-by-one, and document that the funnel's saved stage counts descendant saves. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
DEV-3146 follow-ups from #371 (ADR-0042 example analytics). Of the three open items, two are addressed here and one is deliberately left open.
Funnel "saved" was always 0. A save carried
kind=saved, ref=<demo id>, and the funnel panel filtersblob17='docs'. The ticket offered a rollup lineage join or carrying the source on the event. A rollup join would not fix the panel, because the panel reads Analytics Engine, not D1, so the attribution goes on theexample.savedpoint itself. The API worker resolves the row'sforked_fromto the source example (docs guide and area, starter, import, payload), following saved-demo hops (capped at 5, cycle-guarded).areais not derivable from the lineage, and the worker has no assets binding, so it bundles a compact generateddocsPath -> [guide, area]map (workers/api/src/docs-taxonomy.generated.ts, about 1,600 paths) that the import-docs workflow regenerates, plus a freshness test. Anything that does not resolve (MCP demos, unknown docs paths, broken lineage, a failed lineage read) keeps the oldkind=saved, ref=<demo id>shape and never fails the save. Saves counted before this change keep the old shape.D1 per-invocation query limit. Confirmed from the D1 limits docs: every statement inside a
batch()counts against the 1000-query limit (Workers Paid), and a statement is capped at 100 bound parameters.writeExampleDailyused one statement per row. It now uses 8-row multi-rowINSERTs (12 columns, 96 parameters) in the same single batch as theDELETE. A day needing more than 500 statements (about 4,000 rows) throws before theDELETE, so the previous rows survive and Sentry gets the error. Splitting into several batches was rejected because it saves no queries and a failure part-way would persist a half-written day.serve.shareundercount: not changed. The ticket's condition ("if a server-side share route is ever added") is not met, since/share/:idis still served by the assets-only deploy. That checkbox stays open.ADR-0042 §2 and §5 and observability-contract §5 are updated.
Types of changes
runner/) changeHow was this verified?
node --experimental-strip-types --test pipeline/*.test.mjsrun raw (not through an rtk filter): 2607 pass, 0 fail, 0 skipped.pnpm --filter @handsontable/demo-api typecheckis clean.index.tsandreconcile.tsrestored to master, 6 of the new tests fail (3 attribution, 3 statement budget); the other 4 new tests are guard cases that already pass on master.example-saved-attribution.test.mjs(docs, starter, chained, looping/too-deep/dangling, MCP/unknown path, lineage-read failure, no read when no point is written),example-daily-statement-budget.test.mjs(real SQLite from the real migrations: statement count, parameter cap, throw-before-DELETE, exactly-at-budget),docs-taxonomy-generated.test.mjs(committed file equals what the manifests produce).Checklist
runner/config/frameworks.json(see CONTRIBUTING.md); otherwise it won't appear on demos.handsontable.compnpm build(andpnpm dev) in the affected example/server-example locallyNone of the checklist items apply: no example is added or renamed, and this changes the runner only (typecheck and the pipeline suite stand in for the example build).
Related issue(s):
Note
Medium Risk
Changes server-side analytics attribution and nightly D1 rollup writes; mis-attribution or rollup failure would skew dashboards but saves and core API behavior are guarded with fallbacks and pre-DELETE budget checks.
Overview
Fixes DEV-3146 so the Examples funnel can count saves:
example.savedno longer always emitskind=savedwith the demo id. After a successful editor Save rebuild, the API worker walksforked_from(saved-demo hops, max 5, cycle-safe) insidewaitUntil—after the D1 update— and attributes the point to the source example (docsguide/area/bucket, starter, import, or payload). Unresolvable lineages or failed reads keep the old shape and never block the save.Docs guide/area for
docs:lineages come from a new generateddocs-taxonomy.generated.ts, built from docs-example manifests inimport-docs.mjs(buildDocsTaxonomy/writeDocsTaxonomy). The import-docs workflow regenerates that file alongside the bucket index; a freshness test pins it to the manifests.Separately,
writeExampleDailybatchesexample_dailywrites as multi-rowINSERTs (8 rows × 12 columns) to stay under D1’s per-statement parameter cap and per-invocation statement budget; oversized days throw before the day’sDELETEso prior rollup rows remain. ADR-0042 and the observability contract document the new attribution and rollup behavior.Reviewed by Cursor Bugbot for commit 3824cf1. Bugbot is set up for automated code reviews on this repo. Configure here.