Repository navigation
Conversation
precompute_feature_service accepts entity_keys and skips the keyspace scan when they are given. The push hook now passes the distinct entity keys from the pushed DataFrame instead of rebuilding every entity in the store. Signed-off-by: Chandler King <chandler.king@findheadway.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Stored vectors carry "<fv>__<feature>" names while the default request expects bare feature names, so the fast path always reported a schema mismatch unless full_feature_names=True. Compare on the caller's naming and derive the per-feature view name from the stored names for TTL checks. Signed-off-by: Chandler King <chandler.king@findheadway.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #7076 +/- ##
==========================================
+ Coverage 53.35% 53.43% +0.08%
==========================================
Files 462 462
Lines 58534 58539 +5
Branches 8632 8634 +2
==========================================
+ Hits 31230 31280 +50
+ Misses 25180 25129 -51
- Partials 2124 2130 +6
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Signed-off-by: Chandler King <chandler.king@findheadway.com>
|
@chandlerok please resolve conflicts |
franciscojavierarceo
left a comment
There was a problem hiding this comment.
we need to preserve vector refreshes for pushes that use source field mappings.
| from feast.type_map import python_values_to_proto_values | ||
|
|
||
| join_keys = list(getattr(feature_view, "join_keys", []) or []) | ||
| if not join_keys or any(k not in df.columns for k in join_keys): |
There was a problem hiding this comment.
push() passes the original dataframe here, but ingestion applies batch_source.field_mapping before building entity keys. with {"raw_driver_id": "driver_id"} and a pushed raw_driver_id column, the online row is updated successfully while this check returns [], so the precomputed vector stays stale (or remains missing for a new entity). we should derive these keys from the same mapped representation as ingestion and add a public push/read regression that checks the refreshed vector.
Signed-off-by: chandlerok <chandleroking@gmail.com>
push() hands _precompute_for_push the frame as the caller sent it, but ingestion renames columns through batch_source.field_mapping before it builds entity keys. A pushed raw_driver_id column mapped to driver_id therefore found no join keys and skipped the refresh, leaving the pre-computed vector stale (or missing for a new entity). Map the columns before extracting the join keys, as ingestion does, and add a push-then-read regression that pushes one entity twice under its source column name and asserts the vector follows the second push. Signed-off-by: chandlerok <chandleroking@gmail.com>
Signed-off-by: chandlerok <chandleroking@gmail.com>
There was a problem hiding this comment.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
|
@franciscojavierarceo @ntkathole I updated the PR with the fix.
Would you mind taking another look when you have a chance? |
Signed-off-by: chandlerok <chandleroking@gmail.com>
What this PR does / why we need it:
precompute_online=Truemust survive a store that takes pushes whose columns are named by the source.push()hands the frame to_precompute_for_pushbefore ingestion has appliedbatch_source.field_mapping, so deriving entity keys from the raw frame found no join keys (raw_driver_idvsdriver_id). The online row was written, the push hook returned early, and the pre-computed vector stayed stale — or stayed missing for a new entity.This PR maps the pushed frame the same way ingestion does (
batch_source.field_mapping) before extracting the distinct entity keys, so the refresh follows the write.The rest of the pushed-entity refresh — the
entity_keysparameter onprecompute_feature_service, the_precompute_for_pushhook, and bare-name matching in_try_precomputed_fast_path— is already onmaster(the follow-ups to #6463, including #7077). After mergingmaster, the net diff here is the field-mapping fix and its regression test.Which issue(s) this PR fixes:
Fixes #7074
Checks
git commit -s)Testing Strategy
New unit tests:
test_maps_source_columns_before_building_entity_keys— the push hook derives entity keys from the field-mapped frame.test_push_with_field_mapping_refreshes_precomputed_vector— publicpush→get_online_featuresregression: pushscore=0.5, read0.5, pushscore=0.9to the same entity underraw_driver_id, read0.9. Fails without the fix.Locally:
test_precomputed_feature_vectors.py(72 tests) andtests/unit/infra/online_store+test_feature_server.py(531 tests) pass;ruff check,ruff format --checkandmypyare clean.Misc
@ntkathole @franciscojavierarceo — conflicts are resolved by merging
master, and the note about source field mappings is addressed above.🤖 Generated with Claude Code