Submission + - The Recomputation Manifesto (software.ac.uk)
the discipline of Computer Science has never treated replication seriously, even though computers are very good at doing the same thing over and over again. Not only are experiments
rarely replicated, they are rarely even replicable in a meaningful way. Scientists are being
encouraged to make their source code available [13], but this is only a small step. Even in
the happy event that source code can be built and run successfully, running code is a long
way away from being able to replicate the experiment that code was used for.
I propose that the discipline of Computer Science must embrace replication of experiments
as standard practice. I propose that the only credible technique to make experiments truly
replicable is to provide copies of virtual machines in which the experiments are validated to
run. I propose that tools and repositories should be made available to make this happen. I
propose to be one of those who makes it happen.