Feature requests

[rz-vastdata]

General

• better error messages and exceptions
• be IPython friendly (better docstrings, "hide" internal API)

Filters

• better expression & predicate API (e.g. https://ibis-project.org/tutorials/getting_started#common-operations)

Imports

• allow retrying imports (due to HTTP 503) + work nicely with our gatekeeper
• return the list of non-imported files (with an error message)

Export

• simple & performant Parquet export

Perf

• auto-tune splits/subsplits
• better load balancing
• export perf-related metrics from VAST

HA

• handle disconnections, to resume idempotent operations

Docs

• add DuckDB post-processing examples (e.g. for https://duckdb.org/docs/guides/python/sql_on_arrow.html)
• add a few asciinema.org screencasts for easier onboarding
• https://keepachangelog.com/ + https://towncrier.readthedocs.io/
• a Slack channel?

Other

• run ruff
• integrate with https://harlequin.sh/
• integrate with https://dask.org/

[rz-vastdata]

Questions/suggestion/comments are welcome :)