Once a team is formed, we should meet and set the scope and guidelines for the benchmarking.
The focus for benchmark-data can be found in the white paper. Briefly, this team will aim to provide a framework for determining which pipelines can be recommended for which datasets.
There will likely be two focus points of this group:
1- identifying the "types" of pipelines we would like to start with
2- per pipeline, determinig the range of parameters to be tested
3- per pipeline, determining the outcomes to be tested
4- developing a repository for shared data to use for benchmarking, hopefully to be built into packages eventually.
Once a team is formed, we should meet and set the scope and guidelines for the benchmarking.
The focus for benchmark-data can be found in the white paper. Briefly, this team will aim to provide a framework for determining which pipelines can be recommended for which datasets.
There will likely be two focus points of this group:
1- identifying the "types" of pipelines we would like to start with
2- per pipeline, determinig the range of parameters to be tested
3- per pipeline, determining the outcomes to be tested
4- developing a repository for shared data to use for benchmarking, hopefully to be built into packages eventually.