Prior to reading this document, please review the Author FAQ
Q. What are my responsibilities when reviewing a paper that has an Artifacts Description appendix, a Computational Results Analysis appendix, or both?
A. Reviewers are expected to read any appendix that is submitted with a paper. The information in these appendices is often included in any well-written paper. By collecting this information in a uniform way, we hope that the review process will be more effective without requiring a significant new burden on reviewers.
Q. Must I confirm that links in an appendix work?
A. Confirming links in an appendix are not required of every reviewer. We are asking for volunteers from the review committee who would be willing to validate links provided in appendices. Link validation is required as part of the ACM Artifacts Available badging process.
Q. Must I evaluate the code and results associated with a Computational Results Analysis appendix?
A. Code and results evaluations associated with a Computational Results Analysis appendix are not required of every reviewer. We are asking for volunteers from the review committee who would be willing to evaluate code and results associated with these appendices. Code and results evaluations are required as part of the ACM Artifacts Evaluation badging process.
Q. How may I use appendices as part of the review process?
A. The absence or poor quality of an appendix cannot be used to categorically disqualify a paper. Furthermore, inaccessibility of artifacts such as source code cannot be used to categorically disqualify a paper. However, the presence and quality of an appendix can be used as part of the review process. In particular, if two papers have similar ranking based on the manuscripts, the availability and quality of appendices can provide additional evaluation criteria.
