Repository navigation
fix: Bind per-feature-view values in the Ray offline store closures - #7175
Conversation
| transformation_serialized: bytes = transformation_serialized, | ||
| fv_name: str = fv.name, |
There was a problem hiding this comment.
🟡 Missing required multi-view regression tests
The get_historical_features changes include no unit or retrieval regression tests. The component checklist requires both for retrieval changes.
Learn more
The component checklist requires unit coverage for component changes and a universal test when retrieval behavior changes. This PR changes both transformation selection and timestamp-column creation during multi-view historical retrieval, but its diff contains no test additions or updates. A deferred-callback unit test can capture the functions passed to map_batches and invoke them after both feature views have been processed.
Example: Use two feature views with distinct transformations and timestamp names, such as driver_time and customer_time. After constructing both callbacks, invoke each with its matching batch and verify that each transformation and timestamp column belongs to the correct view. Also exercise transformation failure to check the captured view name.
Recommended fix: Add unit coverage under sdk/python/tests/unit/infra/ using mocked dataset and store dependencies. Add a retrieval regression test that requests two views and consumes the result. Verify that the tests fail when the corresponding default bindings are removed.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #7175 +/- ##
=======================================
Coverage 53.23% 53.23%
=======================================
Files 459 459
Lines 58427 58427
Branches 8629 8629
=======================================
Hits 31103 31103
Misses 25208 25208
Partials 2116 2116
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Signed-off-by: VenishPaneliya <141703684+VenishPaneliya@users.noreply.github.com>
d94fed2 to
c7add8a
Compare
What this does
Two closures inside
RayOfflineStore.get_historical_featuresare defined in thefor fv in regular_feature_views:loop but read their per-iteration values from theenclosing scope.
Dataset.map_batchesis lazy, so the functions do not run until thedataset is consumed — after the loop has finished and those names hold the last
feature view's values.
apply_transformation_with_serialized_funcreadstransformation_serializedand
fv.name.add_timestamp_fieldreadstimestamp_field, which the field-mapping block justabove reassigns per feature view.
With two or more transformed feature views in one retrieval, every one of them gets
the last view's UDF applied. The fix binds the values as parameter defaults at
definition time; the non-transformed path is untouched.
Expected behaviour & risk
Before, with feature views
driver_hourly_stats(udf A) andcustomer_profile(udf B), consuming the first dataset applies udf B:
After, each view gets its own:
No signature or behaviour change for a single feature view, which is why the existing
tests pass either way.
How this was tested
I reproduced the loop and a lazy
map_batchesstand-in to get the output above, andchecked statically that both closures now bind their values as defaults.
ruff checkand
ruff format --checkpass on the file under the repo's config (the file wasformat-clean before and still is).
ruffalso no longer reportsB023(
function-uses-loop-variable) for either closure.I did not add a unit test: both closures are defined inline inside a ~400-line method,
so reaching them needs a live Ray cluster and a materialised feature repo rather than
a unit test. If you would prefer them lifted into module-level factories so they can
be tested directly, I am happy to do that in this PR.