Skip to content

Merge components into unified structure #2072

Description

@bfoley12

Attachments, structures, and tables are all operated on quite similarly.

It would be ideal to begin moving them all to a shared "components" folder and MongoDB collection.

A few steps will be needed:

  1. Data storage
    • Storage of data should be moved out of MongoDB and into S3 via Parquet files (potentially a more advanced storage - delta tables, iceberg - but that is largely unnecessary at this point). MongoDB will then be in charge of handling metadata and maintaining references to the S3 objects
  2. Unify MongoDB collections
    • After moving unique storage information into S3, the components will hold metadata and an S3 reference. This will let us use a single "components" collection and key off of a field - "type" - to delineate between attachments, structures, and tables.
  3. Unify API server logic
    • The API server will have a simpler structure of code by combining the attachments, structures, and tables domains into a single "components" domain

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions