Repository navigation
Clarify Production List and Submission rules - #361
Conversation
| introducing a storage system not currently on the Production List, | ||
| targeting an award, or having any questions regarding eligibility | ||
| should review their submission details with the IO500 Steering | ||
| Committee at least 3 months prior to the release of a new list. Early |
There was a problem hiding this comment.
This is a high bar - that would make any new contributor ineligible for SC this year, so it seems late to make this kind of rule change. We don't even get the CFS out 3 months in advance.
There was a problem hiding this comment.
I was thinking the same thing, but didn't have a better answer. How do we get them to engage early so we don't have to delist on problems? The other thing the 3 month standard would impose on us is code changes would be frozen then too. I'm not thrilled with that. We could make it work, but it is a change for us.
Maybe drop this item for now, but the onus is on us to process new submiters and high placement first giving the benefit of the doubt to the lower places. We still reserve the right to change a ranking after publishing the list.
We might want to encourage submissions at the start of the submission period to ensure we have adequate time to work through any new submission problems. Lack of responsiveness on their part during validation makes the entry potentially ineligible as it couldn't be validated in time. We need to give ourselves space to respond as well.
Any other ideas?
There was a problem hiding this comment.
How about "It is advisable...."?
There was a problem hiding this comment.
I don't think there is any expectation that code is frozen 3 months in advance of submission. It just means "your product has to exist for some time before it can be considered production". 3 months isn't going to make the difference between research and development. I do think that industry presence, customers, etc. are important, and TBH Sugon has a lot of that already.
There was a problem hiding this comment.
Definitely no intention of freezing code. "advisable to review their submission details" is all I'm saying here. The goal of this text is to just have them reach out in advance so we can give them early advice so we avoid any last minute disappointments. I added "It is advisable", instead of 'should', any other update to the text you can think of to indicate it is simply request for early engagement?
There was a problem hiding this comment.
I think we need to be clear we are talking about the metadata rather than the score. Maybe, "it is advisable to engage with the committee three months in advance with descriptive information other than the final submission score if the score is likely to place highly or win as well as all new entrants. This will give time for the committee to work with the submitters to potentially achieve the desired list categorization (i.e., entry on the production list)".
There was a problem hiding this comment.
Thanks for the input, I updated with some new text based on your suggestion,
" The Production List sets a very high bar for inclusion.
It is advisable to engage with the committee at least three months
in advance with descriptive information (no score required) of your submission to work
through any initial eligibility issues for the Production List.
Early engagement ensures the steering committee has sufficient time
to engage with submitters on their desired list categorization
(i.e., entry on the production list). This is especially
encouraged for submitters introducing a storage system not currently
on the Production List or believe their their submission score has the
potential to win an IO500 award."
Revised guidelines for engaging with the IO500 Steering Committee regarding submissions to the Production List.
Summary
Clarifies and improves the readability of the Production List and Submission rules in
the_lists.phpandrules_submission.php:the_lists.php):Production List Termssubheading.the_lists.php):rules_submission.php):