Skip to content

A Kumo Tabular quickstart notebook for Databricks - #48

Open
adi-kumo wants to merge 8 commits into
mainfrom
kumo-tabular-databricks-quickstart
Open

adi-kumo wants to merge 8 commits into
mainfrom
kumo-tabular-databricks-quickstart

Conversation

@adi-kumo

Copy link
Copy Markdown
Collaborator

Adds a Databricks quickstart for Kumo Tabular, next to the relational one. It deploys the model to a one-GPU Model Serving endpoint that scales to zero, loads the AdventureWorks products table, and asks three questions of it: which color a product has, what its list price is, and the color question again as a predictive query. Each answer is checked against the recorded value and compared with a simple rule. The notebook is saved with the outputs of a real run.

The tabular model is the one registered in Unity Catalog as kumo_tabular. Its 16 MB limit message used to talk about entities, which a one-table request does not have, so that wording is made model-neutral too.

Changes

  • examples/databricks/kumo_tabular_quickstart.ipynb: the notebook, with every Unity Catalog name part quoted so hyphenated catalogs work.
  • The refusal for a request over 16 MB now says fewer context rows or fewer rows to score for a tabular request (databricks_serving.py), with matching text in docs/reference/limits.md and the quickstart, a changelog line, and tests.
  • Saved outputs are from the run below. The catalog, schema and endpoint names in them are the notebook's defaults. The %pip cell's output is left out; it only repeats the protobuf notice the text explains.
  • Example 2's text says the price question is an easy one, because the table's standard cost tracks the list price.

Testing

Run on Databricks (EDSP workspace), 2026-10-22 UTC (2026-10-21 local): one-time job run 107355698597869, task run 923635785761350, serverless environment version 5 (client.5.12), Python 3.12.3, pandas 2.2.3, numpy 2.1.3, databricks-sdk 0.156.0, client, api, pql and connectors built from this branch at f5ae890 (1.0.0). The notebook created the endpoint (GPU_MEDIUM, Small, scale to zero) for Kumo Tabular version 1 and ran from top to bottom in 2.7 minutes, because the endpoint image was already cached from an earlier deployment of the same model. A first deployment took about 18 minutes in an earlier smoke test. The only differences from this notebook were the install cell (wheels from a Volume, as the package is not on PyPI yet) and the constants (that workspace's model, catalog and schema, and a throwaway endpoint name). The cells are otherwise unchanged. After that run only Example 2's text and the saved outputs changed. The endpoint was deleted afterwards.

What the run showed:

  • Example 1: 47 of 60 colors correct; always guessing the most common color (Black) gets 21.
  • Example 2: average error $2.63 on the list price; always guessing the median ($337.22) is off by $544.46.
  • Example 3: 47 of 60, equal to Example 1's count, as the text says.
  • Step 8 with 200 context rows: average error $2.58.
  • The probability columns are named probability_<color>, one per color, as the text describes.

Also run for this PR: ruff check, ruff format --check, scripts/check_repo_hygiene.py all, and the five-package pytest suite (2179 passed, 330 skipped), which includes the Databricks notebook tests. The engine suite is not affected.

Follow-ups

  • The install cell installs from PyPI, so the notebook only runs as written once this client is released.
  • Splitting a large tabular request by bytes, rather than only refusing it, is not part of this PR.
  • Over Databricks the client sends rows as arrays; Arrow was tested with raw requests and is about 3.6 times smaller. A client-side Arrow encoding would raise the column limit at 10,000 rows from about 84 to about 310.

🤖 Generated with Claude Code

@adi-kumo adi-kumo self-assigned this Oct 29, 2026
Deploys the tabular model to a one-GPU Model Serving endpoint that scales
to zero, then asks the AdventureWorks products table a color
classification, a list price regression, and the color question in PQL.
Every Unity Catalog name part is quoted with backticks.

Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
… too large

A tabular request has no entities and no num_context_samples, so the
16 MB refusal now asks for fewer context rows or fewer rows to score.
The limits table and the quickstart name the tabular model too, and the
notebook test expects the new quickstart.

Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
The products table carries each product's standard cost, which tracks its
list price closely, so the regression error is small. Say so, so nobody
reads it as the model's typical error on a hard column.

Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
The notebook was run end to end on serverless environment 5 against a
GPU_MEDIUM endpoint serving Kumo Tabular. The catalog, schema and endpoint
names in the outputs are the notebook's defaults. The pip output is left out,
because it only repeats the protobuf notice the text already explains.

Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
@adi-kumo
adi-kumo force-pushed the kumo-tabular-databricks-quickstart branch from e52134d to b3f41a5 Compare October 29, 2026 19:45
@adi-kumo
adi-kumo marked this pull request as ready for review October 29, 2026 19:46
" f'Creating the endpoint {ENDPOINT_NAME}; this takes 20 to 40 minutes.'\n",
" )\n",
"else:\n",
" config = endpoint.config or endpoint.pending_config\n",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use endpoint.pending_config or endpoint.config here, so an endpoint that is already updating to this model is not refused.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Fixed in 01887cb: the initial check now prefers pending_config, so an endpoint updating from another model to the requested model is accepted. The post-wait check remains in place and rejects a final model/version mismatch before constructing the client.

Added the updating-to-requested-model regression case; it failed before the fix. Validation: 68 focused tests passed, 2,700 package/root tests passed (271 skipped), and 2,580 engine tests passed (101 skipped). Lint, formatting and hygiene passed.

Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
@adi-kumo
adi-kumo requested a review from Manushpm8 October 30, 2026 02:31
}
],
"source": [
"endpoint = workspace.serving_endpoints.wait_get_serving_endpoint_not_updating(\n",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait_get_serving_endpoint_not_updating raises the SDK's own OperationFailed when the update ends in UPDATE_FAILED or UPDATE_CANCELED, so the ready != 'READY' message below never runs for a failed deployment: the user sees failed to reach NOT_UPDATING, got UPDATE_FAILED instead. It also means an endpoint that serves fine but had a later update fail makes this cell raise on every rerun. Catching OperationFailed (from databricks.sdk.errors import OperationFailed) and raising the cell's own guidance would cover both.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 71e9b61. The cell catches OperationFailed and re-fetches the endpoint, allowing a healthy active deployment only when it is READY and matches the requested model/version. Otherwise the existing guidance is raised with the SDK exception preserved as its cause. Tests cover failed and canceled updates with healthy, not-ready, wrong-model and wrong-version active deployments.

}
],
"source": [
"products = spark.table(PRODUCTS).toPandas()\n",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toPandas() returns rows in no guaranteed order, and the held-out rows later come from sample(frac=1, random_state=0) on that order, so the seed alone does not fix which 60 rows are held out. Another workspace, or the same table after a rewrite, can get different rows and numbers other than the saved 47 of 60 and $2.63. The relational notebook orders its sample (ORDER BY md5(...)) for this reason; .sort_values('ProductID', ignore_index=True) here would do it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed; fixed in 71e9b61. Products are sorted by ProductID with a reset index before feature construction and sampling. A regression test executes the notebook preparation/split code against differently ordered inputs and verifies identical classification and regression splits. Saved outputs and execution counts were cleared rather than presenting prior metrics as results of the revised notebook. No live model rebuild or deployment was performed.

workspace = Mock()
workspace.serving_endpoints.get.return_value = SimpleNamespace(
config=config(active_model, '1'),
pending_config=config('catalog.schema.tabular', '1'),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every case sets pending_config to the requested model, so the common case, an existing endpoint with no update pending (pending_config=None), never runs, and neither does the deploy cell's own refusal. Dropping the or endpoint.config fallback would break every endpoint that is not updating, and all five cases would still pass. Could you add a case with pending_config=None, and one where the deploy cell refuses?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed; added coverage in 71e9b61 for pending_config=None with a matching active model, refusal of a mismatched active model, and refusal of a mismatched pending model. These execute the deploy cell and assert no create/wait call occurs. The existing post-wait mismatch cases remain.

Comment thread CHANGELOG.md Outdated
### Changed

- The refusal of a request over a Databricks serving endpoint's 16 MB limit
no longer names `num_context_samples` and entities alone, which a tabular

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tabular predictive-query route does take num_context_samples: materialize(indices=..., num_context_samples=...) (materializers/tabular.py, indices()). So "which a tabular request has none of" is not right, and the same goes for the new comment at test_databricks_serving.py:443. The error text itself ("where the model takes one") is fine.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct; fixed both the changelog and test comment in 71e9b61. The descriptions now explain that the guidance covers directly supplied rows as well as predictive-query requests, without claiming Tabular never supports num_context_samples. The user-facing error text is unchanged.

[`kumo_relational_quickstart`](https://github.com/NVIDIA/structured-data-models-client/blob/main/examples/databricks/kumo_relational_quickstart.ipynb) and
[`kumo_tabular_quickstart`](https://github.com/NVIDIA/structured-data-models-client/blob/main/examples/databricks/kumo_tabular_quickstart.ipynb)
deploy each model to a serving endpoint and ask it questions about the
AdventureWorks sample data.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this paragraph here, the "This needs pip install '...[databricks-serving]'" sentence right below reads as the notebooks' install line, but the notebooks need [databricks,databricks-serving] (quote_ident comes from sdm_connectors). Moving the paragraph after that sentence, or naming the notebooks' install, would avoid it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed; fixed in 71e9b61. The serving-only install guidance now directly follows its example, and the notebook paragraph explicitly names [databricks,databricks-serving] for the connectors and serving SDK.

Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
@adi-kumo
adi-kumo requested a review from JGagacheva November 3, 2026 04:07

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants