Yes publishing is making public. I am assuming you mean peer-review...?
- Frank
from iPhone
@Duncan: It would be great if Publishing and Releasing were the same thing (for example, the Journal of Machine Learning Research has done some things to tie the two together), but a Journal article condenses the operational details leaving enough to understand and reproduce the experiment. I think what I was trying to say is that we shouldn't be satisfied with human-readable summaries. The job of communicating results should include the release of machine-readable data & code separate from the paper.
- Dan Gezelter
Love this: "“How can I possibly review this paper if I can’t see the code they were using?" -- I routinely suggest to authors that their paper would be stronger if it included raw data. I'm thinking about getting bolshier about it: "Four replicates, you say? 'Essentially identical' you say? OK, let's see the data."
- Bill Hooker
Even if you see the four replicates there is still the question of the replicates that were run but not included. There is nothing wrong with excluding replicates - it is just that leaving out the "undesirable" runs leaves out a huge aspect how a particular experiment works in practice, especially the ways it can fail.
- Jean-Claude Bradley
Graham pointed me to this thread; my response may be relevant? Cf FOIA request from anonymous colleague for funded grant applications - http://network.nature.com/people...
- Heather