Skip to content

[FEAT] new pipeline building workflow #122

Description

@yongninglei

Proposed Workflow for Building New Pipelineworkflow in LaunchContainer (LC)

Recently, I have been following this workflow below to build new containers within LC. I am using prfpipeline as an example to illustrate the process.

1. LC structure for the new pipelines

  • Each big steps will have their own subfolders: prepare, run, qc etc
  • In the prepare subfolder, I coded the data cleaning steps in to different py scripts and they must follow the orders I was using a numbered structure (e.g., 01, 02, 03, etc.) for clarity and reproducibility.
  • For correct orphan sessions, or the part requires manual edit, I write into separate script:
    • This script performs the required cleaning, but it is not generic, it is some session specific.
    • It serves as documentation of how such cases are handled.

2. Code Structure and Utilities

  • For common functionalities such as:
    • printing/logging
    • generating subject lists
    • force overwrite options
  • I follow the existing LC conventions.
  • I build Reusable helper functions are implemented in lc/utils.py and imported where needed.

3. Development Workflow

  • Code under active development is placed in the tests/ directory.
  • Once the implementation is stable and validated, it is migrated into the main LC codebase.

4. Documentation

  • After integration into LC, we finalize:
    • workflow documentation
    • any special notes that needs to be acknowledged, for example prfanalyze folder strucure is hardcoded in the orignal prfanalyze-vista matlab code by Benson, so we have to respect that in the LC workflow

Open Question

Does this workflow align with the current LC development practices?
Any suggestions or improvements would be very welcome.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions