Proposed Workflow for Building New Pipelineworkflow in LaunchContainer (LC)
Recently, I have been following this workflow below to build new containers within LC. I am using prfpipeline as an example to illustrate the process.
1. LC structure for the new pipelines
- Each big steps will have their own subfolders: prepare, run, qc etc
- In the prepare subfolder, I coded the data cleaning steps in to different py scripts and they must follow the orders I was using a numbered structure (e.g.,
01, 02, 03, etc.) for clarity and reproducibility.
- For correct orphan sessions, or the part requires manual edit, I write into separate script:
- This script performs the required cleaning, but it is not generic, it is some session specific.
- It serves as documentation of how such cases are handled.
2. Code Structure and Utilities
- For common functionalities such as:
- printing/logging
- generating subject lists
- force overwrite options
- I follow the existing LC conventions.
- I build Reusable helper functions are implemented in
lc/utils.py and imported where needed.
3. Development Workflow
- Code under active development is placed in the
tests/ directory.
- Once the implementation is stable and validated, it is migrated into the main LC codebase.
4. Documentation
- After integration into LC, we finalize:
- workflow documentation
- any special notes that needs to be acknowledged, for example prfanalyze folder strucure is hardcoded in the orignal prfanalyze-vista matlab code by Benson, so we have to respect that in the LC workflow
Open Question
Does this workflow align with the current LC development practices?
Any suggestions or improvements would be very welcome.
Proposed Workflow for Building New Pipelineworkflow in LaunchContainer (LC)
Recently, I have been following this workflow below to build new containers within LC. I am using
prfpipelineas an example to illustrate the process.1. LC structure for the new pipelines
01,02,03, etc.) for clarity and reproducibility.2. Code Structure and Utilities
lc/utils.pyand imported where needed.3. Development Workflow
tests/directory.4. Documentation
Open Question
Does this workflow align with the current LC development practices?
Any suggestions or improvements would be very welcome.