 |
|
| FriendFeed is the easiest way to share online. Learn more » |
Jonathan Eisen › Likes
Stumbled across Morgan Langille & Jonathan Eisen's BioTorrents site http://www.biotorrents.net/. Looks like it could be extremely useful but, how do you make sure your system is safe while hosting a torrent? I'm sad to admit it but, I know basically zilch about torrents. Any suggestions on what to do?
Clark Kent, Jonathan Eisen and Steve Koch liked this
I've been asked for the data I've posted on YouTube multiple times and it seems like a torrent would be a good idea since the data is ~100MB.
- Andy Maloney
There's no safety issue - treat a torrent the same as you would any other file you download from the internet - know your source. As far as hosting data, a torrent is no better than any other method if you're the only one seeding. The hard part is getting other people to save the data and leave the torrent app open. It's also a challenge at many institutions where torrent still == piracy in their mind.
- Chris Miller
Thanks Chris. I should edit my above comment to note that the file is more like 100GB. This means popular services like Dropbox are out of the question. Maybe PLoS ONE would host it for me? Any one tried to get them to host gigantic files?
- Andy Maloney
They did say today was the day Skynet gained consciousness: http://is.gd/GTd2bY(via @ejacqui)
Jonathan Eisen liked this
BioTorrents: Drosophila Gut Microbiome Scripts and Data Files (Papers)- CC By
Endre Sebestyen, Egon Willighagen, Kubke and Jonathan Eisen liked this
RT @wilbanks: ISCB Statement on Open Access to S&T Research Literature is epic Win. CC BY, PD data. http://www.iscb.org/iscb-po...
Scott Edmunds, Mr. Gunn and Jonathan Eisen liked this
#sci011 Open Notebook Science: pushing data from bench to web service – Jean-Claude Bradley, Carl Boettiger and Antony Williams http://scienceonline2011.com/watch-l...
Björn Brembs, Steve Koch, Peter Miller and 3 other people liked this
Open Notebook science is about moving from the culture of trust to one of truth
- Kubke
Google spreadsheets: They have an API that programmers can link with [I didnt know this]
- Kubke
Social Chemical labs are linked for labs working on same compounds - You can then see how research in different groups overlap - Need to maximised discoverability
- Kubke
Open means different things to different people - Eg. proprietary or OSS systems -
- Kubke
"this conference is not necessarily representative of what is going on out there"
- Kubke
Go Carl.
- Graham Steel
trying to address in this session: How do you go beyond the notebook?
- Kubke
By using social tools for electronic lab notebook then you can start building a 'social lab notebook'
- Kubke
Open notebooks let people look at the 'real variability' of the data, when what you publish is a 'representative example'.
- Kubke
"This all sounds awesome, I wanna get home and do this right away" (from the audience)
- Daniel Mietchen
Part of the problem is that you get applause for the efforts but no participation (Antony Williams)
- Kubke
deleting is changing history - Not using data for analysis, readers should be able to challenge why certain data points were not used
- Kubke
What's the license of the #scio11 live stream, or screenshots thereof?
- Daniel Mietchen
Good question Daniel!
- Kubke
Both Dems and Reps think that tax cuts are good. Take a look at this graphic and tell me which plan you prefer? - http://www.washingtonpost.com/wp-dyn...
Jonathan Eisen liked this
Watch me in about an hour…This Week In Science with Dr.Kiki Sanford - http://blog.coturnix.org/2010...
Björn Brembs, Jonathan Eisen and Graham Steel liked this
did anyone watch? Was it OK?
- Bora Zivkovic
I subscribe to their podcast, I'll let you know when I've listened to it.
- Björn Brembs
RT @SPARC_NA: SPARC Congratulates the wonderful Phil Bourne of PLoS - winner of the 2010 Jim Gray eScience Award! http://blogs.msdn.com/b...
Jonathan Eisen liked this
It Has Not Escaped Our Notice #343 - http://cromercrox.blogspot.com/2010...
 |
Graham Steel and Jonathan Eisen liked this
Jonathan Eisen, Daniel Mietchen, Cameron Neylon and 9 other people liked this
What tipped him over the edge?
- Peter Binfield
from iPhone
Sadly, the impact factor i think, but I think really feel it because I just sat there and talked to him about the journal. He was clearly interested and mentioned he was noticing the high quality of the articles there and that his colleagues were also starting to publish in there. But its interesting. In conversation today he told me he was rewriting the manuscript to make it a little bit more general because of the breadth of PLoS One. He felt he wanted to make it more accessible to the broader audience.
- Kevin Z
from email
Warning: Misusing the journal impact factor can damage your science! - http://cameronneylon.net/blog...
Mickey Kosloff, Heather, 'Mummi' Thorisson and 25 other people liked this
I had a bit of a rant at a Science Online London panel session on Saturday.As usual when discussing scientific publishing the dreaded issue of the Journal Impact Factor came up. While everyone complains about metrics I've found that people in general seem remarkably passive when it comes to challenging their use. Channeling Björn Brembs more than anything else I said something approximately like the following.
- Cameron Neylon
"...as professional measurers and analysts of the world we should be embarrassed to use JIFs to measure people and papers. It is quite simply bad science." Hear, hear!
- Bill Hooker
Totally OT - I would gladly pay to watch Aussies, Bill and Cameron play tennis (other games may be applicable) and all the proceeds be donated to a worthy cause. #anyonefortennis?
- Graham Steel
Ironically enough, just yesterday I filled in a form required for an application for a professorship, where they wanted to know how many papers I had in which IF journals. Should I be interviewed, this particular position would be so important for me, that for the first time ever, I would probably not say anything about this embarrassing use of the IF, which would normally disqualify them as employers immediately.
- Björn Brembs
Björn, was that Göteborg? I was trying to find the details, but they are apparently using that as part of the officiel job application process... did not find those details yet, though...
- Egon Willighagen
Now, and that makes it even more embarrassing, it was here in Berlin. When I interviewed in Uppsala I did not see any of this nonsense. If I get the professorship, you can be sure there'll be a lecture or two about IFs. And there will be figures with forms from certain universities...
- Björn Brembs
This whole sad discussion reminds me: why isn't there a tool available, that allows people to construct their own citation list??? I've been doing this by hand for years now: http://bjoern.brembs.net/citatio...
- Björn Brembs
Bjoern, have you tried PoP http://www.harzing.com/pop.htm?
- Bill Hooker
Yes, I use it, but it only pulls from GS which is neither as user friendly nor as 'accurate' as WoS/Scopus. So I use all three, de-duplicate by hand, copy and paste into an HTML editor and format (also by hand). I don't know of any other way to do this. To stay on topic: I think this is currently the best way to replace IF counts when evaluating people: use actual citations.
- Björn Brembs
In Poland JIF is used every single time scientists are evaluated (whether it's a grant or a new position). Also, quite often a lecturer on a science seminar is introduced with mentioning her/his "total IF points". We have also "ministry points" (from Ministry of Science). These are awarded in the same manner as IF - per publication (points are also awarded for writing a syllabus for a...
more...
- Pawel Szczesny
@Björn We're working on something with the intention of delivering this and PM-R has been arguing a lot recently for open citations and open metrics.
- Cameron Neylon
I would have thought that using JIF in a job application process would open an organization up to being sued...
- Cameron Neylon
@Pawel - that was kind of the basis of the prestige vs outcomes riff that I most recently wrote about in the interview with Michael. It's a perfectly reasonable decision for a country, particularly a small country to go for prestige as a way of making a mark. But they shouldn't expect that to lead to either a viable, stable, or particularly valuable research community. If you want those things then you need to optimise for them (which is harder to measure obviously, but most important things are)
- Cameron Neylon
I was thinking the other day about changing my cv and instead of listing 'my publications' start listing the papers that cite my papers (first order) and those that cite those first order papers (second order)) (or some quantification of that sort based on 'order'). A visualization of it could be fun to do too. Then I start wondering whether I should wait until I am out of my continuation period ....
- Kubke
@Kubke... agreed... if your research published in a low ranking journal but used significantly in Nature X publications, what JIF should you fairly take... should we perhaps make a black list of universities where JIFs are used? it seems that SHOUTING is the only way to get things changed these days... :(
- Egon Willighagen
@Egon :) I am on the advisory board for creative commons Aotearoa New Zealand, and one thing that came up is that 'opening up' requires a serious change in assessment policies. One example: Lets say someone gets 1000 citations on nature preceedings (not peer reviewed) shouldn't that count more than zero or 1 citation on a 'peer reviewed' nature? Should we move from 'peer reviewed' to 'peer accepted'?
- Kubke
Tres interesting, Kubke. >>> "Should we move from 'peer reviewed' to 'peer accepted'?"
- Graham Steel
@Graham or 'peer uptake'
- Kubke
And depending on who your peers are, we could have top peer, instead of top tier.
- Noel O'Boyle
What if the citing papers all cite the paper to dismiss it, or because it was shown to be fraudulent? You'd need either a citation typology or he possibility to retract papers from the record, the latter being difficult in non-peer-reviewed archives.
- Björn Brembs
@Cameron: Looking forward to that tool!
- Björn Brembs
@Björn It's not so much the tool. That's pretty trivial. It's getting hold of the data that is the problem.... but that's what the project is about.
- Cameron Neylon
Similar issue here as what Bjorn mentioned in the beginning: about to start a tenure-track, and one of the items on my checklist to be eligible for tenure in 5 yrs is "x papers/yr in a journal with IF >= y". Which obviously completely bypasses my open-source work... But at this point in my career there is nothing much that I can do.
- Jan Aerts
I think there are two things I would say to that. One is don't assume that tenure process in 5yrs will look like tenure today. Things are shifting, slowly admittedly, and perhaps too slowly but they are shifting. "Impact" and demonstrated income potential will be very important, both of which your prominence in the Open Source community will help with. Secondly, yes you need some good...
more...
- Cameron Neylon
@Björn wrt. citation typology: here's a recent paper on this very topic: Shotton. CiTO, the Citation Typing Ontology. Journal of Biomedical Semantics 2010, 1(Suppl 1):S6 http://dx.doi.org/10... "..ontology for describing the nature of reference citations in scientific research articles and other scholarly works, both to other such publications and also to Web information resources, and for publishing these descriptions on the Semantic Web. .."
- 'Mummi' Thorisson
@Mummi: nice! This sort of technology needs to be developed and incorporated in citation analyses are to progress.
- Björn Brembs
Separating the aspirations and the instruments of Open Research - http://cameronneylon.net/default...
Michael Wolfle, Mr. Gunn, Matthew Todd and 6 other people liked this
I have been trying to draft a collaboration agreement to support a research project where the aspiration is to be as open as possible and indeed this was written into the grant. This necessitates trying to pin down exactly what the project will do to release publications, data, software, and other outputs into the wild. This has lead me to wonder where it would be more effetive to articulate the fundamental attitudes or aims of what we are trying to achieve. Rather than trying to create tickbox lists we talk about what our priorities are, and perhaps how they should be tensioned together. For me this is about maximising potential re-use of research outputs.
- Cameron Neylon
Principles are slippery buggers, as one of the authors of the Panton Principles will know better than I. Remembering how hard you had to work to make the PPs sufficient to their purpose but not overly restrictive, I think a tickbox list of best practices is an easier intermediate goal, at least.
- Bill Hooker
Perhaps but it might be easier to get some ticklists together if the overall direction is agreed. Partly putting this out to see whether my thoughts are convergent with others or completely off beam...
- Cameron Neylon
from twhirl
I do very much like your compass statement -- "In the choices I make about how and when to communicate my research I will, to the best of my ability and considering the resources available, prioritise above all other considerations, the ability of myself and others to access, replicate, re-use and build on, the products of that research." (I'd delete "and considering the resources available" since that doesn't seem to me to do any extra work above and beyond "best of my ability".)
- Bill Hooker
I wanted to draw a distinction between what I _could_ do and what I might choose not to prioritise because of limited resources. I can see that they can be read the same way but I guess this is part of my obsession with efficient tensioning and markets that I keep wanting to draw that point out...
- Cameron Neylon
@Jean-Claude In a sense that should fit in under process. The process of writing a paper is possibly just as informative and useful as the processes that went into creating the data and ideas that are in the paper
- Cameron Neylon
Cameron - co-incidentally Bjorn is writing a paper openly right now too http://ff.im/pJjTI I hope this becomes a popular trend - it certainly speeds up knowledge sharing
- Jean-Claude Bradley
Analysis: Half of published URLs are dysfunctional a decade later - http://larsjuhljensen.wordpress.com/2010...
Daniel Mietchen, Pierre Lindenbaum, Daniel Swan and 17 other people liked this
As a small aside when setting up a local mirror of Medline, I extracted 15,915 URLs that were mentioned in the abstracts. Checking them revealed that 12,354 of them (78%) were functional, which may not seem that bad. However, plotting the percentage of dysfunctional URLs as a function of publication year reveals a less pleasant [...]
- Lars Juhl Jensen
Interesting analysis--and needs to be compared to the old estimate that the half-life of a URL, in general, is 90 days. Ten years is first-rate by comparison.
- Walt Crawford
Oh neat - thanks Andrew, I'll add a reference to it.
- Lars Juhl Jensen
@Mr. Gunn - I did a small subsequent check in response to a comment on the blog, and for 4 of 10 randomly chosen cases I was not able to locate the content elsewhere. So it is also content disappearing. I agree that the URL moving without a proper redirection is more of a nuisance than an actual problem.
- Lars Juhl Jensen
@Duncan, thanks I hadn't seen that one before but it reminded me of the original paper by Jonathan Wren from 2004 on the topic :-)
- Lars Juhl Jensen
It can just strengthened my opinion about why we should abolish the supplementary material sections of papers...
- Julien Colomb
To me it rather shows that we need the publishers (or university libraries) to take responsibility for the supplementary information. If it just lives on the authors' website, it will eventually disappear.
- Lars Juhl Jensen
Extract the email address from each abstract with nonfunctional url-s and send them a note. At least you can compare the # of bad url-s with the # of bad email addresses.
- Hedi Hegyi
Another Benefit of submitting to PLoS ONE: less time wasted on cover letter: no need to wax poetic about scope, novelty, potential impact. #plosone
'Mummi' Thorisson, Damian, Claudia Koltzenburg and 6 other people liked this
That's right, but unfortunately some authors still write poems there! Truth is, I almost never read cover letter. I jump right to the abstract, figures, then text...
- Ramy Karam Aziz
Poems in the cover letter, those I actually would read ;) Good to hear about relative unimportance otherwise, though.
- Heather Piwowar
That's PLoS ONE. But I think that in other journals that have an initial "screening" step, many initial rejections are made solely based on cover letters
- Ramy Karam Aziz
Interesting to see the published review process for this paper. - http://www.nature.com/emboj...
Jonathan Eisen, Claudia Koltzenburg, Björn Brembs and 4 other people liked this
Referee completely dismisses the work because they don't "improve" upon native kinesin. I find the paper an amazing piece of work with all kinds of new insights. I wonder if Ref#3 would have sent that terse review if his name were attached to it.
- Steve Koch
from Bookmarklet
Also illustrates the potential role of peer review in improving a paper (wish we could see the original ms). I sure wish more journals would do this, and encourage (but not require) reviewers to sign their names.
- Bill Hooker
fascinating - the author(s) really got a lot of mentoring out of it too. Is this typical?
- Christina Pikas
@Bill, you could try emailing the authors to ask for an old copy...who knows, may get lucky? I am still struggling with the current version, so I'm not going to ask ;) I think it's very nice work, as far as I can tell so far.
- Steve Koch
@Christina, I'm not sure. In my very limited experience, the editors seemed to be very hands-off. I've had some incredibly helpful referees, and am convinced that most of my papers were better because of it. Was it worth their time? I don't know. ... you made me remember how helpful grant reviewers have been to me, in terms of mentoring. I've had many many rejected grants where the referees obviously spent a bunch of time trying to tell me how to make it better.
- Steve Koch
Probably lots of gems in http://www.google.de/search... but unfortunately, pdfs at nature are not allowed to be crawled: http://www.nature.com/robots... .
- Daniel Mietchen
This editor is just amazing. It is clear that he spent a lot of time assessing the quality of the manuscript and of the referees' comments. And he is very supportive.... I've never received such a detailed letter from an editor... I've been more often disappointed about the lack of involvement from editors in the review process. I always felt they had never read the reviews but only taken into account the recommendations (Publish, major/minor revision, reject) from the referees.
- JJ
I'm with Steve and JJ, this editor has gone above and beyond. I'm a conscientious reviewer/editor but I don't go that far.
- Bill Hooker
Mendeley and the ecology of science - http://www.hyperorg.com/blogger...
Björn Brembs, Antony Williams, Victor / Mendeley Team and 6 other people liked this
"Today I interviewed Victor Henning, the co-founder of Mendeley, a very popular desktop app for researchers that indexes your PDFs, provides social reading and researching tools, and aggregates anonymized information about what researchers are actually reading and annotating. There’s a lot packed into Mendeley, all designed to help researchers find out what they need to know, primarily though social means."
- AJCann
from Bookmarklet
This is really stupid on my part, but I never thought about anonymization on Mendeley before. The first time I ever showed a scientist a social bookmarking application, their eyes lit up. When I asked what they liked, they said "This is great. I can see what everyone else is reading but they can't see what I'm reading". I tried to explain that it doesn't work like that. Except that #Mendeley sort of does. Doh.
- AJCann
"I can see what everyone else is reading but they can't see what I'm reading" -- I don't get it. What is appealing about that?
- Bill Hooker
Oh, I geddit. Ugh.
- Bill Hooker
BMJ Open: accessible medical research - http://blogs.bmj.com/bmjopen...
Science 2.0, Alexey, Jonathan Eisen and Jean-Claude Bradley liked this
"BMJ Open is an open access journal for general medical research. Using a continuous publication model the journal will provide rapid publication for research from any medical discipline or therapeutic area. Not only will the journal publish traditional full research reports, including small or low-impact studies, but we intend to shed light on all stages of the research process by publishing study protocols, pilot studies and pre-protocols. The journal will also place great emphasis on the importance of data sharing; raw data will be linked to at its repository or hosted online as supplementary material wherever possible. This emphasis on transparency will continue with research protocols and reviewers’ comments being published alongside final papers. Authors will need to provide clear statements of their studies’ strengths and limitations. Peer review will be open, and the criteria for acceptance will be that the research was conducted in a transparent and ethical way. Naturally, all articles will need to comply with the BMJ Group’s high standards of research and publication ethics."
- AJCann
from Bookmarklet
But if it is OA why does it say "Individuals may register for a free 30 day online trial to all content"
- Jean-Claude Bradley
Where does it say that?
- AJCann
Just clicked on the first article http://ard.bmj.com/content...
- Jean-Claude Bradley
I don't think that's it - the open journal hasn't launched yet.
- AJCann
Search and clustering orders of magnitude faster than BLAST. - http://www.citeulike.org/user...
Science 2.0, Bas Jansen, Darek Kedra and 2 other people liked this
Bioinformatics (Oxford, England) (12 August 2010), btq461. MOTIVATION: Biological sequence data is accumulating rapidly, motivating the development of improved high-throughput methods for sequence classification. RESULTS: UBLAST and USEARCH are new algorithms enabling sensitive local and global search of large sequence databases at exceptionally high speeds. They are often orders of magnitude faster than BLAST in practical applications, though sensitivity to distant protein relationships is lower. UCLUST is a new clustering method that exploits USEARCH to assign sequences to clusters. UCLUST offers several advantages over the widely-used program CD-HIT, including higher speed, lower memory use, improved sensitivity, clustering at lower identities and classification of much larger datasets. AVAILABILITY: Binaries are available at no charge for non-commercial use at http://www.drive5.com/usearch. CONTACT: robert@drive5.com. Robert Edgar
- Simon Cockell
P ≠ NP and the future of peer review - http://cameronneylon.net/blog...
Steve Koch, Iñaki Arrieta Baro, Jim Till and 32 other people liked this
The online maths community has lit up with excitement as a document, claiming to prove one of the major outstanding theorems in maths has been circulated. In response an online peer review process has swung into action that is very similar to the kind of post-publication peer review that many of us have advocated. Is this a one off, a special case? Or does it point the way towards successfully using the web to find a way of doing peer review effectively and efficiently?
- Cameron Neylon
Cameron - you mention the NaH discussions. The actual outcome from the blogosphere was much more interesting (and useful) than true or false. Instead of linking to these discussions the journal just retracted it without explanation, which isn't very useful for advancing knowledge.
- Jean-Claude Bradley
JC, that's a good point. Not only was it quicker and more comprehensive, but also much more useful and informative.
- Cameron Neylon
from Android
JC++
- Egon Willighagen
OMG, *shivering*, theoretical computer science exam memories are coming back, can this be true? Honestly just reaching that proof level is already unbelievable, and if the gaps could be fixed, it would be just mindblowing. So, "Vinay Deolalikar you rock!", and @community "That might keep you, the smartest brains, busy for a while".
- joergkurtwegner
Besides, please help voting for non-deletion on WP! http://en.wikipedia.org/wiki...
- joergkurtwegner
I vote special case -- it's such an important problem that post-pub peer review works great because everyone is interested.
- Bill Hooker
I certainly agree that it is a special case. I guess with some of these other examples I'm becoming hopeful that its not _so_ special that we won't start to see more of this going on - indeed there has been extremely little commentary that I've seen that seems remotely surprised by the process that seems to be being followed by the community. It's the fact that most seem to be assuming that this is a natural process that I find most interesting in some ways...
- Cameron Neylon
Does anyone have a PDF copy, proof was removed from his web site? http://www.hpl.hp.com/persona...
- joergkurtwegner
I don't think it's one of a kind, but I think that it is extraordinary or out of the ordinary. Probably <<10% of papers would warrant this kind of attention
- Christina Pikas
If we agree that this is a special case, it seems to me we are seeing a lot more "special cases" than there used to be, making this kind of review a practical possibility, not just a theoretical idea.
- Bonnie Swoger
One issue I haven't seen being given a lot of attention by bloggers is the issue of copyright. I'm assuming Vinay Deolalikar will have copyright over the TeX manuscript and PDF pre-prints, but maybe HP will have some sort of claim here? What happens if there's a bidding war amongst journals? Have scribd and the other hosting sites been issued with take down notices? Could the issue of...
more...
- Dan Hagon
Further to the above, it's also possible that a handful of super-special special cases will whet the public appetite for this kind of process, so that we start to see it carried out for more and more "routine" sorts of discoveries. It would be nice even to get to the point where "extraordinary claims require extraordinary evidence" (say, the top 10 or 20% of...
more...
- Bill Hooker
Upon reading, "Most journals with commenting systems still get relatively few comments on the average paper" my immediate comment was, "The average paper is rarely worth reading, let alone reviewing." If people can't get at least several friendly colleagues to review their work, it tells us something about the scope and the quality, right there. You get to this point toward the end of the essay. Thank you for the story!
- Maria Droujkova
every movement needs a figurehead, so why not take this example to figure well in Open Science
- Claudia Koltzenburg
@ Maria: "If people can't get at least several friendly colleagues to review their work, it tells us something about the scope and the quality, right there." Not necessarily. Obviously, currently out of fashion research will attract less interest. But little interest doesn't necessarily imply lack of scope or quality - it could simply be ahead of its time. So, unfortunately, this reasoning only works one way.
- Björn Brembs
+1 Claudia - Agreed, everything is in there, science, controversy, copyright issues. I would prefer a better life science example, but some maths+informatics example will do as well. Though, lets be honest the IP situations and relevance are totally different in those fields. So, we should never assume "one movement", its not that simple.
- joergkurtwegner
@joergkurtwegner good point, it might still work in general, though, if the most general points (relevant for interdisciplinary perspectives, I mean) could be picked up for every example, btw there's quite a few useful bytes for this in http://michaelnielsen.org/blog...
- Claudia Koltzenburg
@Björn "it could simply be ahead of its time" yepp, fair point!
- Claudia Koltzenburg
I'm not sure that it matters. One of the points of post-publication peer review is that it doesn't matter when it happens. Something might be harder to discover when it becomes relevant five years down the track than it would otherwise be but once it is discovered and reviewed that will improve. Much better than it not being published at all!
- Cameron Neylon
hm, certainly at the stage of encouraging someone to get it out there, the "ahead of its time" factor is important, maybe crucial - imagine I wish to seek critical feedback from my most immediate peers who I trust for a first tentative round, what if I do not have them because of the "ahead of its time" factor? --- Once it IS up there: yes, I agree with your point @Cameron
- Claudia Koltzenburg
@the-"ahead of its time"-factor: maybe like the inventors of double-blind peer review we might have to discuss anonymous authorship for Open Science post-publication review scenarios - anyone seen ideas & discussions on this one?
- Claudia Koltzenburg
wondering if you guys have seen <a href="http://cordis.europa.eu/ictresu...">this article</a> about the european research effort "liquidpubication"? features the sentiments, "don't print it, post it" and "don't review it, use it". the latter especially is a really interesting point, although perhaps easier in computer...
more...
- jessykate
perhaps a form of "using it" could also be in showing it's wrong. that's interesting because it gets at the question of incentive, and also of quality. if i review a paper and reject it (especially in an anonymous peer review system), what professional credit (incentive) do i get? especially in an open peer review system where i probably don't even get my name on a list of editors. but...
more...
- jessykate
@jessykate, re @liquidpub see http://ff.im/oW73i
- Claudia Koltzenburg
very interesting point about ''using it" @jessykate - let's look at what disciplines may share or maybe not: what is a use case in a given field, and what is the "it", respectively?
- Claudia Koltzenburg
Claudia - agree this will be a difficulty, but if someone really can't get feedback from _anyone_ on work then I think they've got bigger problems than being ahead of their time. No-one works alone any more - if nothing else you should be able to get local colleagues to take a look. @jessykate My view is that using it is inherently a form of review, as is citation, or any other...
more...
- Cameron Neylon
Cameron, partly agree, which disciplines do you have in mind?
- Claudia Koltzenburg
@Claudia - Fair enough, thinking mainly of experimental sciences. This is clearly less true of maths and theoretical physics, where people do seem to largely work alone. But there there seems to be a more positive culture of review both prior to submitting pre-prints and around the pre-prints before formal publication. And of course it remains the case that the lone non-professional...
more...
- Cameron Neylon
The post was picked up on Techdirt: http://techdirt.com/article... Some comments there interesting although much of it similar to what we've seen before.
- Cameron Neylon
new blog post: supplemental data or no? http://scientopia.org/blogs...
Daniel Mietchen, Benjamin Tseng, Kubke and 5 other people liked this
thanks for this blog post, cannot answer your questions, though, but feel like adding a new one: why not put data first? and put up the text as "supplementary material"? no, am not being 100% serious, what I mean is this: why privilege one format over another at all? they're all bytes after all, and if they are relevant to what the author(s) want to tell the world - let's have'em all. And in the open, of course.
- Claudia Koltzenburg
this is definitely part of my reaction. I guess I've only thought about the consumer's side of supplemental materials, not the producers.
- Christina Pikas
http://ff.im/p7twO "When articles become fully online and consist mainly of the data, all of this nonsense will go away :-)"
- Claudia Koltzenburg
http://ff.im/pbbdP Fwd: fresh bytes re Supplementary material / Supplementary materials / Supplemental material: bring author perspectives and LIS perspectives together!
- Claudia Koltzenburg
Are there any journals/repositories which currently allow authors to put up data for the world to see? (i.e. a real replacement for supplemental materials/emailing a non-responsive lab)
- Benjamin Tseng
e.g. CTT was designed as a composite creature right from the start: open access journal/ open subject repository (Cellular Therapy and Transplantation), for a short glimpse of where we are on the Open Data journey, see point 7. here: http://okfnpad.org/science...
- Claudia Koltzenburg
I Hate Your Paper - The Scientist - Many say the peer review system is broken. Here’s how some journals are trying to fix it - http://www.the-scientist.com/2010...
Steve Koch, 'Mummi' Thorisson, Lit and 26 other people liked this
“When it comes to journals and publications, I’m highly skeptical that [the peer review] process adds much value at all,” adds Richard Smith, former editor of the British Medical Journal, who has written extensively about peer review. “In fact, it detracts value because it wastes a lot of time of a lot of people,” he says. “There’s lots of evidence of the downside of peer review, and very limited evidence of the upside.
- Alexey
from Bookmarklet
Wonder about studies that have measured the relative quality of various review processes?
- Mike Chelen
"The full podcast, originally created for the Journal of Participatory Medicine, is hosted here." http://www.patientpower.info/JoPM... (Podcast with transcripts) -- Peer Review and Reputation Systems: A Discussion -- 1. Defining the Problems and Issues with Peer Review Today -- 2. Light Versus Heavy Peer Review -- 3. Transparency in Peer Review -- 4. Wikipedia-Style Peer Review…and Rating/Reputation Systems -- 5. Crowdsourcing Research/Peer Review -- 6. Building a Community
- Claudia Koltzenburg
Hmm, I don't know if peer-review improves papers but it has so far often improved *my*papers...
- Björn Brembs
what's your take, Björn, does review quality depend on anonymity? should we say blind or non-blind should be up to the reviewer?
- Claudia Koltzenburg
btw, have not seen this idea voiced before ;-)
- Claudia Koltzenburg
I am with Bjoern here, I see improvement of papers on peer review (mine and others'). Both on substantial issues as much as smaller things. But i also don't think that peer review is a bulletproof system.
- Kubke
I think the point of peer review is not to improve papers but to prevent the publication of truly shitty ones. We really should judge the system on the true negatives. Kind of like democracy - democracy is great, not because it helps you choose great leaders, but because it helps you get rid of shitty ones.
- Bosco Ho
if publications were wiki documents then... +1
- Claudia Koltzenburg
Yes, like Kevin Kelly proposed (wikiscience): http://www.edge.org/3rd_cul... Additionally micro-contributions like comments and corrections should be measured and added to the overall reputation of a person.
- Konrad Förstner
I'm going to offer counter examples. I've had several papers that were damaged by peer review in my opinion and several more which were severely delayed to publication and not significantly improved by the process, leading to potential opportunity costs (one paper which took nearly two years to get published subsequently has got around 50-80 citations depending on what you count). but...
more...
- Cameron Neylon
Bosco, I disagree. Firstly the evidence suggests that most of the shitty papers get published anyway eventually and secondly, like I argued here (http://cameronneylon.net/blog...) actually the more rubbish is out there, the easier it will be to find what you want.
- Cameron Neylon
Bosco. While Cameron may be right on arguing against the papers part of your argument, I think the democracy part is a true gem......
- Nils Reinton
Yep Nils, I don't think search is going to do much to improve our parliamentarians unfortunately...
- Cameron Neylon
In both cases, though, bringing in transparency has lots of potential to improve the system. By the way, I listened to the podcast last night (not knowing about this thread then) and took my notes at http://ff.im/oVHcb .
- Daniel Mietchen
Not sure I've done this before, but -- I disagree with Cameron here. The assumption that more data = better search just doesn't convince me at all. I've seen a lot of papers that were basically trash, but that would appear as hits in any search that would also find good work on the same topic. I think the figure that gets bandied about is that only 30% of papers fail to find a home...
more...
- Bill Hooker
There's a step missing there. My argument was more that we need more data to build better search systems to enable better search. Putting out more stuff with today's Google won't help but it could help build tomorrows was the idea. In terms of the 70/30 figure the thing we don't have any real data on whether papers are improved, delayed, or made worse - most scientists say it improves...
more...
- Cameron Neylon
Well, we've had this part of the conversation too -- if reviewing were seen as part of a researcher's job and being good at it were properly rewarded, then studies like the BMJ one might not find such rampant slack-arsery. I guess part of this is failure of imagination compounded by lack of knowledge on my part -- I don't see how any search algorithm is going to be able to distinguish...
more...
- Bill Hooker
Imagine two papers, reporting the same experiments but in paper A the obvious control was omitted in each case, whereas paper B included proper controls. A and B may even draw the same conclusions! Both are going to be hits on any search I can imagine, and then you have to read 'em to know that A is rubbish. Scale that up, and you pretty quickly get beyond the point where you can read...
more...
- Bill Hooker
I begin to think that what we need most is more data...
- Bill Hooker
I don't really buy the opening crystallography example as an argument that peer review is broken. Okay, one reviewer suggested something unnecessary. There should have been more than one reviewer, and the editor (or his/her minions) should have reviewed the reviewer's comments, either omitting any blatantly ridiculous ones in correspondence back to the author or easily accepting the author's note that it did not need to be addressed.
- Rachel Walden
Bill, agree that more data is required to draw any sensible conclusions but in response to your point about paper A and B, as far as I'm aware we have absolutely no credible evidence that this is _not_ the case and some examples of dreadful peer review at the top of the pile. So should we be spending billions a year on something we have no evidence does any good?
- Cameron Neylon
In any case, my argument would be that unless you have both papers A and B then there is no way anyone can develop tools to distinguish between them, whether they be social or technical. Also surely the argument must be that both support the same conclusion and it is only by reading them together that you get the fullest possible picture...even without the control paper A strengthens paper B's case.
- Cameron Neylon
Bill: Why should the concept of search be limited to keywords? Consider Friendfeed for example, where results are highlighted by number of Likes. If anyone considers one paper better than the other, it would appear higher in the search results.
- Mike Chelen
Mike, may be a matter of semantics but when you get into "likes" etc, to me that's post-publication review -- in other words, a filter. I love the idea, but a glance at PLoS journals (and other experiments) will show that it hasn't taken off: people just don't interact with the research literature (yet?) in a way that makes social filtering effective.
- Bill Hooker
Cam, if A really is going to get published without the controls (70% of the time), then I'd have to conclude that peer review is a waste of time and money. I'm finding that hard to believe, but as you point out -- I have no evidence. I've made fun of other people for falling into the trap where "I can't imagine it" == "it's not possible", and now it seems I am hoist on my own petard......
more...
- Bill Hooker
A different way to consider peer review: it's as much a psychological barrier as a real one. So, consider paper A -- it may be shite but the authors thought it was good enough to pass peer review. If they didn't even have to consider that hurdle, what might they be pushing out? Is there a mountain of substandard work that would be dumped into the knowledge base but for fear of the Peer...
more...
- Bill Hooker
That argument I'd agree with more. One of the things I've not seen a discussion of is the extent to which the citation advantage in high IF journals is a result of author selection bias (i.e. authors are probably pretty well placed to say which of their papers will be most successful and send them to 'appropriate' journals on the basis of that). There's a flip side to this tho. What...
more...
- Cameron Neylon
from twhirl
see also http://journalology.blogspot.com/2010... (e.g., "CNS" disease explained) thx to http://ff.im/oZdka
- Claudia Koltzenburg
For those of you with institutional repositories, do you limit submissions to only peer-reviewed work?
Jonathan Eisen, Jean-Claude Bradley and Claudia Koltzenburg liked this
Nope - that wld limit deposits to maybe ten a year. We do lots of gray lit, non-peer reviewed material
- Sarah
Thanks Sarah. That's what my director wants to do and I'm 100% against limiting submissions in that way.
- Meredith
No, we accept pretty much anything they're willing to give us. Or even to tell us about and we go hunt for a copy for ourselves. It's not like there's any need to be selective for fear of running out of storage space or something.
- Deborah Fitchett
Our institutional repository is an important part of archiving all the files and documents for our lab notebooks - it would be really disappointing if they changed their policy and stopped helping us in that way http://onsbooks.wikispaces.com/ONS+Sol...
- Jean-Claude Bradley
I don't think my Director is really aware of what's going on at other institutions, so, I'm pretty sure that once she is, she'll look more favorably upon accepting other forms of intellectual output from the University.
- Meredith
Cameron Neylon on practical steps toward open science - http://michaelnielsen.org/blog...
Steve Koch, John Dupuis, Heather Piwowar and 10 other people liked this
Astonishing Landscape Photographs of Our Planet Earth - http://blog.insicdesigns.com/2010...
 |
Jonathan Eisen, Ed Millard and MafiaX liked this
Open Data citation advantage - http://www.pawelszczesny.org/2010...
Bill Anderson, Endre Sebestyen, Claudia Koltzenburg and 5 other people liked this
Hi Pawel. The argument has indeed been started ;) http://www.plosone.org/article... (see also "will you be scooped or will you be famous" http://www.prio.no/Researc... ) fwiw, I think the citation argument can be useful in some circles, for a limited time. But I agree that it should not be the only argument, nor is...
more...
- Heather Piwowar
Oh, how cool, thank you Heather. I've missed that one (and failed again to notice that there are important things that weren't published last week ;) ). Interesting point about field-specificity. Do you have the code available? You could crowdsource reanalysing citation advantage in other areas (I can dig in genomics).
- Pawel Szczesny
Agreed, extending to other areas would/could be very interesting. The hard part is getting the baseline "datasets created" articles, then annotating them for whether data shared or not. We have a start in evolutionary biology thanks to DataONE intern Sarah Judson http://www.openwetware.org/wiki... and I've got 11k gene expression...
more...
- Heather Piwowar
btw Pawel, now I'm curious.... you think the citation advantage argument is counterproductive? Or you want to explore it? Or both??? I could understand both. People contain multitudes :)
- Heather Piwowar
While I don't think the argument is absolutely counterproductive, I might shoot the lecturer after OD advocacy talk that mentions _only_ citation advantage (such talks happen in OA advocacy). But seriously I'm interested in exploring the argument however not because of advocacy, but rather I'm curious about field-specific nuances and sharing culture/sharing policies. Thanks for the links - I'll have a look.
- Pawel Szczesny
Gathering citation data is pretty easy from Scopus through PubMed IDs... let me know if you want more info on that or on my data and code. Three more refs on sharing policies, fyi, are DataONE intern Nic Weber in evo bio: http://www.openwetware.org/wiki... my 2008 ELPUB paper on microarray policies, and Kate McCain's 1995 paper on Mandating Sharing. We are due for an across-the-board look, it is true!
- Heather Piwowar
Review of Open Science Summit 2010, #OSS2010 - http://betascience.blogspot.com/2010...
Dan Hagon, Carl Boettiger, Mike Chelen and 9 other people liked this
Interesting observations about some practical aspects of making a conference successful. Not having breaks is a tough one.
- Martin Fenner
Not having breaks is a tough one. For one thing, it forces people to take bathroom or refreshment breaks during the session and once you have people leaving the room during the sessions it becomes "okay" to leave. Before you know it people will leave the room for almost any reason and the audience starts to check out a bit - once the focus is broken it is really lost. We spent *a lot* of time discussing schedule and breaks and psychology of group focus when we were planning Sage Congress.
- Lisa Green
Lisa, that makes a lot of sense. Not having breaks not only reduces the overall focus of sessions. The discussions in small groups between sessions for me are a very important reason to go to meetings.
- Martin Fenner
The not having breaks was really really unfortunate. It not only creates some direct challenges but it means that people don't have time to talk. I was the one who kind of forced the break when there was supposed to be a panel discussion on day 1 . I know the panel would have been interesting. I did not think they would cancel the panel when I insisted on a break. But they were going to...
more...
- Jonathan Eisen
Jonathan - so you are the culprit who canceled the panel I was on ! :)
- Jean-Claude Bradley
from Android
Sorry - did not realize it would be fully canceled. But a meeting w/o interaction might as well be a blog post.
- Jonathan Eisen
I don't think it was Jonathan's fault they cancelled it. And it was Crazy to make a schedule that went from six hours without a break so I understand why he spoke up. Who could have guessed that the guy would cancel the panel? That was CRAZY and I certainly would not have predicted it would happen if someone asked for a break.
- Lisa Green
Strongly agree on Martin's point "The discussions in small groups between sessions for me are a very important reason to go to meetings" and Jonathan's point "But a meeting w/o interaction might as well be a blog post."
- Lisa Green
Essentially the panel cancellation was my fault. There was a combination of people being exhausted as Jon points out but I think it wasn't clear that some of us were going to have to leave soon after. In retrospect I should probably have suggested that we do it after 30 minutes or in a smaller scale informal way.
- Cameron Neylon
from twhirl
Cameron - I think the fault is with the original schedule more than anything else. Lack of breaks is to blame and scheduling SciFoo people in the afternoon is also to blame. Your group could have been scheduled at 11am, giving you more of a time buffer for getting to Mountain View for SciFoo.
- Lisa Green
I'm just teasing a bit - but it was weird how that played out. Even if there are no questions it is useful to get a recording after taking the time to travel
- Jean-Claude Bradley
from Android
A separate point is the I-House auditorium is better suited for entertainment than presentations requiring focused attention. I know there are summer sessions going on, but I would still think a proper venue with tiered seating might have been available. Not to detract from the overall fact that having such a conference was great, and I enjoyed the talks I saw.
- Ruchira S. Datta
from Android
As someone who had stayed inside all day here in Oregon in order to hear Cameron, Jean-Claude, Victoria Stodden et al via the live steam it was a major bummer it was dropped. I had told all kinds of librarian professional groups about that session and the fact that it was dropped was quite distressing.
- Hope Leman
Pharma embraces open source models : Nature Biotechnology : Nature Publishing Group - http://www.nature.com/nbt...
Pawel Szczesny, Mickey Kosloff, Walter Jessen and 9 other people liked this
Surprise, surprise, and did I not just say that ... listening can be a wonderful thing ;-) http://ff.im/o9R4d And I just finished reading "open innovation" http://twitter.com/joergku... and Lindegaard noticed it http://twitter.com/joergku...
- joergkurtwegner
Yeah, the time between the time something starts and the message actually gets through to the bigger audience does not really to have changed with the introduction of the internet :(
- Egon Willighagen
Interesting analysis. Companies getting involved is good (speed) and bad (different priorities to academic partners. Includes this quote about the problem of scaling large projects: "Managing the complexity of collaborations that occur when large numbers of institutions and individuals collaborate creates another twist in the road toward open participation. This is particularly...
more...
- Matthew Todd
@Matthew Todd Do you have a link to Jonathon Cummings study ? I couldn't track it down via google. My own experience is that large scale *research* collaborations are difficult to manage and more importantly take forever (lumbering is the right word).
- Greg Tyrelle
No, sorry, never seen it. Large CAN mean lumbering, depending on how you do it.
- Matthew Todd
My fault missed the double quotes, thought you were referring to it rather than the article.
- Greg Tyrelle
There are different studies on communication efficiency and group size, the best investigations are probably available in the software engineering area, since code development contributions can be measured easily (lets not get into the quality question, which is a another topic). Anyway, two notable readings might be "Drug discovery: new models for industry–academic partnerships"...
more...
- joergkurtwegner
@joergkurtwegner not my area, but great links. I was intrigued that someone had done a study on quantifying academic collaborations in terms of publication output. Just confirms some anecdotal evidence. Don't know what metric you'd use for academic-industry collaborations...
- Greg Tyrelle
Professionally our methods of transmitting and reviewing the results of research are generations old and by now are totally inadequate for their purpose. - http://larsjuhljensen.tumblr.com/post...
Greg Tyrelle, Berci Mesko, MD, Björn Brembs and 5 other people liked this
... and that is still the case.
- Dan Hagon
65 years and little has changed...
- Björn Brembs
Friendly bacteria protect flies from sterilising worms http://blogs.discovermagazine.com/notrock...
Ruchira S. Datta and Jonathan Eisen liked this
Phyloinformatics in the age of Wikipedia - http://www.slideshare.net/rdmpage...
 |
Paul Gardner, Michael Kuhn, Andrew Su and Jonathan Eisen liked this
nice view of the positive and not-so-positive side of working in Wikipedia... (Rod, did you notice that the main obstructionist to your proposal recently got blocked for 72h for his/her obstructionism?)
- Andrew Su
Other ways to read this feed:
