Just got an email from them to be on the SAB for the Biology journal. Do they typically ask people without Ph.D.s (yet)? I like the concept but have concerns about legitimacy, etc... - Shirley Wu

If it were me, it would boil down to who else is on the SAB, and could I see myself publishing there. If it's not something I'd normally want to be associated with, then I don't think the cool title is worth it. my two cents... - Andrew Su

Shirley: erm... no, that doesn't sound typical. looking at the few members of the board who have joined up and are displayed on the site, they look to be all prof or assoc prof level (no names i recognise, but...), which seems pretty standard... - Joe Dunckley

It actually looks pretty good to me -- OJS, LOCKSS, CC-BY and a founding concept (publish all data) that I agree with. It's a definite plus in my book if they don't care about formal degrees and just want to get the best people. I'd like to know more about the business model/support framework, esp since there's no mention of author-side charges. I've registered as a reader/potential author/available reviewer. - Bill Hooker

If I could double-like this, I would. - Chris Miller

Let us know what you find out, Shirley. I'm a grad student who wouldn't mind reviewing either, if it's kosher. - Chris Miller

I replied with some questions, we'll see what they say - Shirley Wu

Quoting Feynman: "If you've made up your mind to test a theory, or you want to explain some idea, you should always decide to publish it whichever way it comes out. If we only publish results of a certain kind, we can make the argument look good. We must publish BOTH kinds of results." [http://wwwcdf.pd.infn.it/~loreti...] - Eric Jain

It will be interesting to see how this plays out. My main concern is that it is a lot of work to write a paper based on failed experiments and I don't know how many researchers would be willing to do that. I don't have a problem with sharing my failed results from within my lab notebook, which I have to maintain anyway. But if scientists do contribute I think it is fantastic. - Jean-Claude Bradley

Key questions are business model and what they are offering beyond what is already available. Also not confidence inspiring that two of the web pages provides for SAB members seem to lead to dead ends which don't have their name listed. Happy to give benefit of the doubt but presentation of these things is important. - Cameron Neylon

@Jean-Claude: I see two kinds of papers ending up in these journals: 1, "this didn't work" papers reporting pretty much only failed expts; and 2, regular papers that simply include extra information on the stuff that failed, as well as the stuff that didn't. Even when the failed stuff is useful information, trad. journals usually tell you to leave it out. So this model seems to be a sort of compromise between ONS and normal science -- a "gateway drug" for ONS, perhaps? - Bill Hooker

@Bill: A bit of an aside, I'm currently reviewing a paper for Neuropharmacology, a journal which is a tad out of my usual reading. Scanning the author instructions, I was struck by this line:Results. The results should be fully illustrated. Negative findings should also be noted to avoid unnecessary replication by others. - NatBlair

Bill - I appreciate the motivation and the utility if they get submissions - I am just wondering if they can get enough people to put in the effort. It will be interesting information about the scientific community either way. - Jean-Claude Bradley

Haven't heard back from them yet. The morning after I sent my response with questions, however, I got a verbatim copy of their original email sent to me again by the same person. So I don't know what's going on but like Cameron said, not exactly confidence inspiring - Shirley Wu

Does anyone have an opinion on the Journal of Negative Results? - Mr. Gunn

@Jean-Claude - I think your concern is well founded. We get good feedback for JNR-EEB (http://www.jnr-eeb.org/index...), but few submissions. - Bob O'Hara

I got a reply from the section editor that contacted me. They do want Ph.D. scientists, and he didn't know I was still a graduate student (not that hard to click on the "about me" page of my blog, especially if you mention that you've seen my blog and that's why you're contacting me...). Anyway, the other things he mentioned were that there are no author fees (in addition to no access fees), and everything is done in a volunteer fashion because the people involved want to improve science. - Shirley Wu

I'd still be concerned about whether it's possible to sustain high quality without a business model but then again, that's what this open stuff is all about, right? So it would be great if it worked long enough to get something behind it to maintain it. Of course, J-C's concern is also a big one - who will write and submit papers to it? - Shirley Wu

I'd be very worried - getting papers is a big enough problem - but to do it with no business model just isn't sustainable. The business model can be volunteer but then you've got to explicitly worry about how to support and retain your volunteers. But in general I think I'm coming to JC's point. No-one has the time to write full papers on material that isn't up to the grade for existing journals - I don't think they are going to start just because the journals are there. The barrier has to be much lower. - Cameron Neylon

People who start writing their papers before doing the actual work and keep them in sync with the latest progress should be glad to be able to submit them somewhere (even if a few more days of effort are required to clean up the paper and answer reviewer questions etc)? This approach also seems more effective than spending weeks after the fact going through unreadable lab notebooks and struggling to remember what it is you had done! I can see that the motivation for going through all of this is low if the experiment didn't work out. Yet I suspect this is how most people work? - Eric Jain

Which one of the two Eric? I think the latter is more common...but even with a good record the hassle of going through a peer review process is a big disincentive - Cameron Neylon

Kind of journal of negative data. More likely to fail, or at least what people claim as "all results" will be the tip of iceberg. - Dean Johns

"They do want Ph.D. scientists" -- that grates my cheese. Suppose you read Shirley's blog and think "smart person, want her on board my project" -- well, what has changed when you find out she's a grad student? This feeble reliance on a piece of paper is why universities are less and less focused on actual education, and more and more on certification -- a product, bought and sold. [/rant] - Bill Hooker

I certainly agree with you, Bill, but I think there was some validity to what they said; they went on to ask me what my plans were after graduating, whether i was going to remain in the sciences. I would assume that they wouldn't be recruiting folks with Ph.D.s in mol bio that were now working as management consultants on wall street, for example. But they didn't exactly do a great job of vetting if they didn't know I was still a student. - Shirley Wu

@Cam: I think this is basically another "journal of negative results", not so much about material that isn't up to scratch. Eric's comment about people who keep notes in sync with benchwork makes me think that journals that are willing to take negative results are likely to be a boon to anyone who keeps an open or semi-open notebook. (I disagree that peer review is a hassle or a disincentive. Sure, some reviewers are jerks, but overall the process is fun.) - Bill Hooker

I've never looked at the journal of negative results. But it strikes me as quite weird to think about peer-reviewed negative results. I think it's sufficient to just publish your notebook and an informal summary ("we were hoping this would happen, but instead this happened and we don't feel like publishing it in a peer-reviewed journal. Hope these results are helpful!") It's tough to see what peer-review would add w/o asking the researchers to do extra experiments? - Steve Koch

I'm also unclear on whether there is a definition of "negative result." Does the term originate from pharma stuff? I.e., drugs that didn't work? In other fields is "negative result" a synonym for "less-interesting result?" That's sort of what I was thinking and why peer review would seem weird. - Steve Koch

Steve, I think there does need to be some degree or review, or they could end up with a bunch of "Bigfoot was not discovered in Alaska" kinds of submissions. Negative, in the sense of isn't something that will support a grant application, is how I imagine it to be used. - Mr. Gunn

Mr. Gunn, yeah I agree with that. I was really saying I don't see the point of having a journal like that. It seems like a waste of peer-review resources. Self-publishing seems to make more sense to me...but I'm probably not thinking about the right examples of negative results. - Steve Koch

What the journal adds over self-publishing is editing, review, and discovery. Pretty much the same as other journals. I see what you mean about a waste of resources, though. Some people consider Friendfeed to be a waste of resources, too, so there's that. - Mr. Gunn

This is all fascinating. Mr. Gunn says, "What the journal adds over self-publishing is editing, review, and discovery. Pretty much the same as other journals..." But it is the same as other journals? This one would be specifically devoted to secondary findings—that would set it apart, wouldn’t it? If this journal were well run (which doesn’t seem to be the case, based on Shirley’s accounts), wouldn’t there be real value? Sometimes when I read about drug discovery efforts in amyotrophic lateral sclerosis, it seems that there are fascinating bits in stray sentences in papers that report that yet another once promising drug didn’t work in humans that some brilliant scientist could run with. Might that not be of some value? Would it have to be full papers? Couldn’t it be snippets of info? - Hope Leman

Hope, my comments were directed towards Steve's question of what value publishing negative results in a journal would have over simply self-publishing them. Of course the journal would be set apart from others in terms of content. - Mr. Gunn

I can't make much comment about how organized they are other than that they didn't know I wasn't a Ph.D. despite having been to my blog. Even if they do have a committed and organized core of people, the bigger concern is whether they can sustain a purely volunteer effort. If they can't, they need an actual business model, and the arguments here (and elsewhere: http://tinyurl.com/c2b3xm) are of the mind that a full-fledged journal of negative results isn't cost effective. - Shirley Wu

Thank you for your patience, Mr. Gunn. Thank you for the link to Cameron's post on the topic. That is an outstanding, asute bit of analysis. - Hope Leman

Another possible use for the open journal system model would be using it for well-done student publication -- another venue of practice before hitting the bigger journals -- individual schools/labs could put OJS to use, students learn not simply to write for publication but also how to act as reviewers -- provides a neat continuing reinforcement of how to read science (stats, design, etc) -- basically a more finished open notebook product -- also useful for initiating new people into the lab - Mickey Schafer

Hey all -- I got an email invite from All Results Journal:Physics today for the scientific advisory board. I reread my comments above and still agree with my own skepticism. On the other hand, wouldn't mind supporting it to see if it can fly. Did any of you end up getting invovled? Jean-Claude, I see your name on the SAB for ARJ:Chem ... any opinions? - Steve Koch

(As an aside: is this thread sufficiently public that the majority of reasonable people wouldn't mind if I linked it to the editor who sent me the invite? I think definitely yes, but given the hoopla that ensued several weeks ago, figured I'd ask first.) - Steve Koch

I'm still skeptical - I don't see evidence of awareness of the magnitude of the job they're taking on. But then anything that starts from the assumption "publication should basically be free" sets my alarm bells ringing. Happy to see this made public but then I would be I guess :-) - Cameron Neylon

It'd be good if it did take a role as a dumpster for drug research -- stuff failing at any stage might be usefull from discovery on. - Chris from twhirl

Pity is that it's not what I thought at first, which is a journal that publishes just data sets with structured annotation (rather than a full paper being needed, shortcutting the idea of the paper-writing robot) to really get people to clear out their lockers. Just a tab-format form answering the kinds of questions listed in projects linked to by MBBI (ref. http://mibbi.org/index...) which would in passing make any data set so-registered directly citable and therefore useful for passing credit back to the generator on use -- recycling. Can someone please start that journal (incidentally, the journal wouldn't hold the data -- they'd have to go to a public database -- hopefully in time all databases will be issuing DOIs rather than accessions anyway and there's a handy tool to get the metadata into tab that even looks nice at http://isatab.sf.net/isacrea...). - Chris from twhirl

Call it ' Kudos Convertors -- "Free you data. Boost your citations*." ' [* Subject to funders counting it]. How about some kind of data copyright statement while we're at it, to assist with enforcement. - Chris from twhirl

Yeah, I agree, Chris. It'd be much better w/o need for formal paper and "rigorous peer review" as they state now. Editorial or peer review should just make sure that necessary info has been included. Wonder if SAB members can change any of these decisions before they go "live?" - Steve Koch