[BanyanDB] Data/Schema Import and Export

[mrproliu]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

BanyanDB already supports snapshots, backup/restore, and lifecycle management. However, it still lacks an operations-oriented, cluster-level data import/export workflow.

The design is here: https://github.com/apache/skywalking-banyandb/tree/4b3fc26c5e5a37d0e96212c2e6cd299b5744095e/docs/design/0.12.0/data-export-import

It adds bydbctl data export and bydbctl data import, built around two channels: a CSV channel that replays logical rows and works across differing topologies and versions, and a native channel that relays part bytes through the liaison for speed but requires the source and target to match. Both are driven by a plan file, both support a dry run that touches nothing, and both can resume after an interruption.

The steps below are ordered so that each one ends with something demonstrable, and the two E2E steps close a channel before the next one opens.

• 1. Extend databasev1.Node to expose supported file versions and timezone information.
• 2. Build bydbctl and implement the export Plan API to collect node information for dry run.
• 3. Implement CSV export.
• 4. Implement CSV import.
• 5. Complete CSV import/export E2E.
• 6. Implement the .bnc container format, with common read and write APIs.
• 7. Add property file-chunk support to the file-chunk protocol.
• 8. Implement native import/export.
• 9. Complete native import/export E2E.
• 10. Add resume support to both workflows.

Use case

Using to export or import data from banyanDB cluster.

Related issues

No response

Are you willing to submit a pull request to implement this on your own?

• Yes I am willing to submit a pull request on my own!

Code of Conduct

• I agree to follow this project's Code of Conduct