Skip to content

fix: Refresh pre-computed vectors for pushes with source field mappings - #7076

Merged
ntkathole merged 8 commits into
feast-dev:masterfrom
chandlerok:fix/precompute-refresh-pushed-entities
Nov 18, 2026
Merged

ntkathole merged 8 commits into
feast-dev:masterfrom
chandlerok:fix/precompute-refresh-pushed-entities

Conversation

@chandlerok

@chandlerok chandlerok commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

precompute_online=True must survive a store that takes pushes whose columns are named by the source. push() hands the frame to _precompute_for_push before ingestion has applied batch_source.field_mapping, so deriving entity keys from the raw frame found no join keys (raw_driver_id vs driver_id). The online row was written, the push hook returned early, and the pre-computed vector stayed stale — or stayed missing for a new entity.

This PR maps the pushed frame the same way ingestion does (batch_source.field_mapping) before extracting the distinct entity keys, so the refresh follows the write.

The rest of the pushed-entity refresh — the entity_keys parameter on precompute_feature_service, the _precompute_for_push hook, and bare-name matching in _try_precomputed_fast_path — is already on master (the follow-ups to #6463, including #7077). After merging master, the net diff here is the field-mapping fix and its regression test.

Which issue(s) this PR fixes:

Fixes #7074

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

New unit tests:

  • test_maps_source_columns_before_building_entity_keys — the push hook derives entity keys from the field-mapped frame.
  • test_push_with_field_mapping_refreshes_precomputed_vector — public push → get_online_features regression: push score=0.5, read 0.5, push score=0.9 to the same entity under raw_driver_id, read 0.9. Fails without the fix.

Locally: test_precomputed_feature_vectors.py (72 tests) and tests/unit/infra/online_store + test_feature_server.py (531 tests) pass; ruff check, ruff format --check and mypy are clean.

Misc

@ntkathole @franciscojavierarceo — conflicts are resolved by merging master, and the note about source field mappings is addressed above.

🤖 Generated with Claude Code

chandlerok and others added 2 commits October 26, 2026 19:16
precompute_feature_service accepts entity_keys and skips the keyspace scan
when they are given. The push hook now passes the distinct entity keys from
the pushed DataFrame instead of rebuilding every entity in the store.

Signed-off-by: Chandler King <chandler.king@findheadway.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Stored vectors carry "<fv>__<feature>" names while the default request
expects bare feature names, so the fast path always reported a schema
mismatch unless full_feature_names=True. Compare on the caller's naming and
derive the per-feature view name from the stored names for TTL checks.

Signed-off-by: Chandler King <chandler.king@findheadway.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@codecov-commenter

codecov-commenter commented Oct 26, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 53.43%. Comparing base (2a312cc) to head (0f1c844).
⚠️ Report is 4 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7076      +/-   ##
==========================================
+ Coverage   53.35%   53.43%   +0.08%     
==========================================
  Files         462      462              
  Lines       58534    58539       +5     
  Branches     8632     8634       +2     
==========================================
+ Hits        31230    31280      +50     
+ Misses      25180    25129      -51     
- Partials     2124     2130       +6     
Flag Coverage Δ
go-feature-server 30.96% <ø> (ø)
python-unit 55.00% <100.00%> (+0.08%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/feature_store.py 50.36% <100.00%> (+2.57%) ⬆️

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a312cc...0f1c844. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Chandler King <chandler.king@findheadway.com>
@ntkathole

Copy link
Copy Markdown
Member

@chandlerok please resolve conflicts

@franciscojavierarceo franciscojavierarceo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to preserve vector refreshes for pushes that use source field mappings.

Comment thread sdk/python/feast/feature_store.py Outdated
from feast.type_map import python_values_to_proto_values

join_keys = list(getattr(feature_view, "join_keys", []) or [])
if not join_keys or any(k not in df.columns for k in join_keys):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

push() passes the original dataframe here, but ingestion applies batch_source.field_mapping before building entity keys. with {"raw_driver_id": "driver_id"} and a pushed raw_driver_id column, the online row is updated successfully while this check returns [], so the precomputed vector stays stale (or remains missing for a new entity). we should derive these keys from the same mapped representation as ingestion and add a public push/read regression that checks the refreshed vector.

Signed-off-by: chandlerok <chandleroking@gmail.com>
push() hands _precompute_for_push the frame as the caller sent it, but
ingestion renames columns through batch_source.field_mapping before it
builds entity keys. A pushed raw_driver_id column mapped to driver_id
therefore found no join keys and skipped the refresh, leaving the
pre-computed vector stale (or missing for a new entity).

Map the columns before extracting the join keys, as ingestion does, and
add a push-then-read regression that pushes one entity twice under its
source column name and asserts the vector follows the second push.

Signed-off-by: chandlerok <chandleroking@gmail.com>
@chandlerok chandlerok changed the title fix: Refresh pre-computed vectors for pushed entities only and match bare feature names fix: Refresh pre-computed vectors for pushes with source field mappings Nov 17, 2026
Signed-off-by: chandlerok <chandleroking@gmail.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

@chandlerok

Copy link
Copy Markdown
Contributor Author

@franciscojavierarceo @ntkathole I updated the PR with the fix.

  • _entity_keys_from_dataframe maps batch_source.field_mapping before pulling join keys, same as ingestion, so a push with raw_driver_id → driver_id refreshes the vector instead of returning [].
  • Added the requested push/read regression: push 0.5, read 0.5, push 0.9 to the same entity, read 0.9. It fails without the fix.

Would you mind taking another look when you have a chance?

Signed-off-by: chandlerok <chandleroking@gmail.com>
@ntkathole
ntkathole merged commit 67cbcd5 into feast-dev:master Nov 18, 2026
18 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

precompute_online: one push rebuilds every vector in the store, and the fast path never matches without full_feature_names

5 participants