RHEED low-level features + embedding-vector query

[chris-price19]

Exposes two RHEED capabilities through the SDK, backed by atom-cloud PR #1840 (which adds the corresponding backend routes + response field).

Low-level RHEED features

client.get(data_ids, include_low_level_features=True) now surfaces the pipeline's region-level features (area_n, eccentricity_n, fwhm_0_n, profile skews/CVs, …) as extra columns on RHEEDVideoResult.timeseries_data. The backend nests them per-point under low_level_features; RHEEDProvider.to_dataframe flattens that into raw-named columns. Off by default; ignored for non-RHEED types.

Embedding-vector query

The raw Chronos timeseries embeddings (the inputs to similarity matching, vs. the derived curve from get_similarity_trajectory) are now reachable:

• client.get_rheed_embeddings(data_id, *, workflow, window_span, kind="window"|"prototype", offset, limit) → RHEEDEmbeddingResult — numpy (N, D) vectors + time axis / cluster sizes, with .to_dataframe().
• client.query_rheed_embeddings(data_id, *, workflow, window_span, kind, top_k) → tidy neighbors DataFrame ("find similar growths"), source excluded, sorted by similarity.

New RHEEDEmbeddingProvider (under atomscale/similarity/) mirrors the existing SimilarityTrajectoryProvider fetch/build split; RHEEDEmbeddingResult is exported from atomscale.results.

Tests

New pure unit tests (no network) for the provider/result conversions and the low-level flatten, plus creds-gated live integration tests. 26 passed, 2 skipped locally; full suite still collects (141) and test_core/test_align stay green.

Note: requires atom-cloud #1840 (the include_low_level_features flag + /embeddings/ routes) deployed for the live paths to return data; window_span must match an embedded span or results are empty.

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR exposes three new RHEED capabilities through the SDK: low-level per-region features flattened into the timeseries DataFrame when include_low_level_features=True, per-frame segmentation masks fetchable standalone via get_frame_masks or merged into the timeseries via include_masks=True, and a k-NN embedding neighbor query via query_rheed_embeddings. The implementation is well-structured with good test coverage (unit + creds-gated live) and correct handling of edge cases like sparse mask coverage, windowed frame scoping, and empty responses.

• Low-level features: _flatten_low_level_features uses json_normalize to expand per-point dicts into columns; collision protection prevents clobbering known metrics.
• Frame masks: get_frame_masks scopes the backend request to the frame range the (possibly windowed) timeseries actually spans, avoiding unnecessary data transfer; attach_frame_masks does a left join on Frame Number preserving the multi-level index.
• Embedding neighbor query: RHEEDEmbeddingProvider wraps the k-NN endpoint with a clean fetch/build split; decode_mask_rle is refactored into a shared helper now exported from atomscale.results.

Confidence Score: 5/5

• Safe to merge. The three new features are well-isolated, all new code paths are covered by unit tests, and the frame-scoping logic correctly limits mask fetches to the windowed series range.
• All new methods have clear validation, graceful empty-response handling, and thorough test coverage. The attach_frame_masks multi-index join and frame_number_bounds scoping logic are correct. No data loss or corruption paths were found.
• No files require special attention. The only open item is whether the backend guarantees sorted neighbor results — client-side sorting in neighbors_to_dataframe would make that contract explicit.

Important Files Changed

Filename	Overview
src/atomscale/similarity/embedding_provider.py	New provider for k-NN embedding neighbor queries; clean fetch/build split, robust empty-response handling, good column ordering logic.
src/atomscale/timeseries/rheed.py	Adds low-level feature flattening, frame mask attachment, and frame-range utilities. The json_normalize without max_level (noted in previous thread) and the overall logic are sound; mask join on MultiIndex works correctly.
src/atomscale/client.py	Adds query_rheed_embeddings, get_frame_masks, and include_masks support to get_rheed_timeseries. All validation, frame scoping, and sentinel-based to_frame logic are correct.
src/atomscale/results/rheed_image.py	Extracts decode_mask_rle into a standalone function to share between single-frame and per-frame mask endpoints; existing mask decode path now delegates to it without behavior change.
tests/test_rheed_timeseries.py	Good coverage for attach_frame_masks (sparse coverage, empty masks, no-frame-axis passthrough, re-attach idempotency), frame_number_bounds, and include_masks integration via monkeypatched _get.

Sequence Diagram

sequenceDiagram
    participant User
    participant Client
    participant RHEEDProvider
    participant RHEEDEmbeddingProvider
    participant Backend

    Note over User,Backend: get_rheed_timeseries (include_masks=True)
    User->>Client: "get_rheed_timeseries(data_id, include_low_level_features=True, include_masks=True)"
    Client->>RHEEDProvider: "fetch_raw(data_id, include_low_level_features=True, ...)"
    RHEEDProvider->>Backend: "GET rheed/timeseries/{data_id}/"
    Backend-->>RHEEDProvider: "{series_by_angle: [{series: [{low_level_features: {...}}, ...]}]}"
    RHEEDProvider->>RHEEDProvider: to_dataframe() → _flatten_low_level_features()
    RHEEDProvider-->>Client: ts_df (MultiIndex: Angle, Frame Number)
    Client->>RHEEDProvider: frame_number_bounds(ts_df) → (first, last)
    Client->>Backend: "GET rheed/images/{data_id}/frame_masks?from=first&to=last"
    Backend-->>Client: "[{frame_number, mask_rle, mask_height, mask_width}, ...]"
    Client->>RHEEDProvider: attach_frame_masks(ts_df, mask_rows)
    RHEEDProvider-->>Client: ts_df with mask columns (sparse NA for unmasked frames)
    Client-->>User: DataFrame

    Note over User,Backend: query_rheed_embeddings
    User->>Client: query_rheed_embeddings(data_id, workflow, window_span, kind, top_k)
    Client->>RHEEDEmbeddingProvider: fetch_neighbors_raw(client, data_id, workflow, ...)
    RHEEDEmbeddingProvider->>Backend: "GET similarity/{workflow}/{data_id}/embeddings/neighbors/"
    Backend-->>RHEEDEmbeddingProvider: "{neighbors: [{data_id, similarity, ...}, ...]}"
    RHEEDEmbeddingProvider->>RHEEDEmbeddingProvider: neighbors_to_dataframe(raw)
    RHEEDEmbeddingProvider-->>Client: DataFrame (ordered columns)
    Client-->>User: DataFrame

Loading

_{Reviews (2): Last reviewed commit: "Merge branch 'main' into claude/agitated..." | Re-trigger Greptile}

[chris-price19]

@greptile