Skip to content

feat(typespec): canonical dtype schema declarations (333) - #399

Merged
discreteds merged 7 commits into
developfrom
feature/333-canonical-dtype-schema-declarations
Oct 17, 2026
Merged

discreteds merged 7 commits into
developfrom
feature/333-canonical-dtype-schema-declarations

Conversation

@discreteds

@discreteds discreteds commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Behavior

  • Accept canonical dtype-only and compatible broad-type declarations; persist tagged enums and decimals.
  • Honor native overrides with mismatch/unverified warnings. Native objects retain parameters in owned in-memory snapshots and are recursively rejected by descriptor serialization.
  • Share effective targets across schema lowering, conform, inference, nested validation and ingress.
  • Preserve explicit JSON/native-duration distinctions and existing decimal conversion policy.

Verification

  • Full registered-backend quick suite: 32538 passed, 159 existing skips, 2490 xfailed, one schema-only inference failure. That in-repo regression was then repaired and verified by 353 assessment/inference tests; a subsequent selected-Narwhals native-carrier repair passed 69 source-shape/inference tests. The complete suite was not rerun after these final repairs.
  • Final focused conform/converter/ingress owners: 532 passed, 21 retained xfails; source lint passes.
  • All nine execution identities exercised for I8/U8/FP32 physical schemas and values: 23 passed, four nullable pandas cases remain xfailed under item252. Native-override nullable cases retain the same upstream failure.
  • Actual typed Parquet write/readback: 15 physical-schema/value cases, plus JSON Files resource-to-DAG I8 readback, nested empty/null-only materialization and native timestamp/validation/ingress smokes.
  • Native model/record ingress, wrong-owner rejection, decimal override precedence, categorical/timezone freeze, overflow and JSON/native-duration semantics exercised through actual public calls.
  • Independent correctness and complexity reviews completed; all reported in-repo defects repaired. No static type checks run.

Approved limitations

On 2026-10-17 the owner explicitly approved retaining pre-existing item252 (NumPy-backed pandas float/NaN to integer through Narwhals) as nonblocking for item333, like the previously approved temporal-unit limitation in item334.

Both issues remain open and unrepaired. Original failing inputs and narrow strict xfails remain; no workaround, automatic ingress normalization, or claim of passing nullable acceptance is introduced. This clears the item252 draft gate only. Verification evidence above is unchanged, and the approval does not authorize a merge or close item333 before merge.

@discreteds
discreteds marked this pull request as ready for review October 17, 2026 02:40
@discreteds

Copy link
Copy Markdown
Member Author

Merged develop (1e4166f) into this branch and pushed conflict resolution d431d24. Preserved item333 native override evidence alongside item311 detailed/native-first schema resolution. Fixed evolve fallback inference losing detailed source metadata; SQLite schema assertions now distinguish physical affinities from narrow typed egress without removing output-width checks. Verification: 11,613 passed, 10 skipped, 182 xfailed across TypeSpec, conform, relations, core dtypes, validation-plan and pydata suites with full backend scope. Ruff passed. Runtime smokes exercised override precedence, nanosecond preservation, nested narrow dtype Parquet round-trip, and evolve values/schema/drift consistency. Approved nonblocking backlog items 252 and 334 remain unchanged.

@discreteds
discreteds merged commit c1e8da5 into develop Oct 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant