Skip to content

Preserve custom FieldOptions extensions when loading schema metadata from JSON - #4540

Open
pengpeng-lu wants to merge 3 commits into
FoundationDB:mainfrom
pengpeng-lu:load_unknown_fields
Open

Preserve custom FieldOptions extensions when loading schema metadata from JSON#4540
pengpeng-lu wants to merge 3 commits into
FoundationDB:mainfrom
pengpeng-lu:load_unknown_fields

Conversation

@pengpeng-lu

@pengpeng-lu pengpeng-lu commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • CommandUtil.loadRecordMetaDataFromJson parses downloaded schema metadata JSON via
    JsonFormat.parser().ignoringUnknownFields(). JsonFormat (protobuf-java-util) cannot parse
    proto2 extensions under any configuration, so any custom FieldOptions extension in the
    JSON (e.g. a per-field marker used to distinguish multiple RecordTypeUnion generations of the
    same record type) was silently dropped during parsing.
  • Once dropped, two different generations of the same record type reduce to identical (empty)
    FieldOptions, and RecordLayerTable.Builder.addGeneration's dedup check throws
    "Duplicated options for different generations of Table X" as soon as a schema has more than
    one generation of the same record type in its RecordTypeUnion.
  • Adds CommandUtil.restoreFieldOptionExtensions (+ helpers): after the normal JsonFormat merge
    and dependency resolution, it walks the raw JSON tree, finds any FieldOptions extensions
    declared by the already-resolved proto dependencies, and copies their values back in via
    protobuf's reflective DynamicMessage/setField API, which — unlike JsonFormat — does
    support extensions.

Test

  • Added custom_field_option.proto, a small self-contained proto2 FieldOptions extension used
    only by this test.
  • Added field-options-extension-metadata.json + field-options-extension.yamsql: a
    RecordTypeUnion with two fields referencing the same record type, each carrying a distinct
    extension value; registered in YamlIntegrationTests as fieldOptionsExtensionTest.
  • Manually confirmed the regression is caught: temporarily disabling the fix reproduces
    "Duplicated options for different generations of Table MyRecord"; re-enabling it passes.

@pengpeng-lu pengpeng-lu added enhancement New feature or request testing improvement Change that improves our testing and removed enhancement New feature or request labels Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing improvement Change that improves our testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant