 |
|
| FriendFeed is the easiest way to share online. Learn more » |
Neil Ernst › Comments
Notation And Thinking - http://rjlipton.wordpress.com/2010...
APL - for those who find Perl too easy to maintain.
- Neil Ernst
It says a lot about who I am in my 35th year that I find this funny and anxiety-provoking.
- Neil Ernst
Thought for the day: We need better tools to distinguish between the grey literature and the brown literature.
Chris Granade, suelibrarian, AJCann and 2 other people liked this
yeah, I'm missing out on the color references too. Is this a UK thing? Grey lady? Brown only conjures one thing for me.
- Neil Ernst
Neil, that one thing is exactly what I'm referring to...
- Cameron Neylon
from twhirl
probably requires a little context - this was occuring to me in the middle of an angsty discussion about the value and necessity of pre-publication peer review and the primacy of the traditionally peer reviewed literature. I was thinking of a Venn diagram - you've got the "white" traditionally PR'd literature, the non peer reviewed "grey" literature, but the point was there is a lot of crap in both of these categories.
- Cameron Neylon
like i was just saying... http://twitter.com/steinsk...
- Joe Dunckley
Online commenting on journal articles at least point out the 'bad' papers. It is a start, isnt it?
- Kubke
Peer review: What is it good for? - http://cameronneylon.net/blog...
!lker yoldas. )°(, Meryn Stol, Wobbler and 29 other people liked this
It hasn’t been a real good week for peer review. In the same week that the Lancet fully retract the original Wakefield MMR article (while keeping the retraction behind a login screen – way to go there on public understanding of science), the main stream media went to town on the report of 14 stem cell scientists writing an open letter making the claim that peer review in that area was being dominated by a small group of people blocking the publication of innovative work. ...
- Cameron Neylon
Yes, but one bright note: the Lancet retraction made news in at least two outlets: I heard on NPR report on it and also a general radio news report that mentioned it.
- Mickey Schafer
It was actually quite thoroughly covered in the UK media, so in that sense good. There is a question given the issues now raised as to why it ever got through peer review in the first place though.
- Cameron Neylon
If it doesn't work in the journal format, it works even less in conference format (primary publication venue for computer science). Example: from submission to notification is 6 weeks, in which time the reviewers have to process over 20 papers each. I can't see how this allows for any meaningful comments. See http://cacm.acm.org/magazin...
- Neil Ernst
And yes, I am bitter :)
- Neil Ernst
Peer review is a complex issue and this is the best "in a nutshell" overview I've seen to date. Nice one.
- Bill Hooker
Love this part, Cameron: "But there is perhaps an even more important procedural issue around peer review. Whatever value it might have we largely throw away. Few journals make referee’s reports available, virtually none track the changes made in response to referee’s comments enabling a reader to make their own judgement as to whether a paper was improved or made worse. Referees get no...
more...
- Steve Koch
Two points: 1) Why doesn't computer science use its distance from the journal system to leapfrog further away from it, e.g. to wiki-based publication models? http://cacm.acm.org/magazin...
- Daniel Mietchen
2) There are some journals which do the whole review process in public, notably those published by the European Geoscience Union. http://bit.ly/axjZeo
- Daniel Mietchen
The BioMedCentral Journals publish the pre-publication history of a paper, including the reviewers names: http://www.biomedcentral.com/info...
- Martin Fenner
I wrote about the value of peer review a few months back: http://network.nature.com/people...
- Martin Fenner
The best quote: "Whatever value [peer-review] might have we largely throw away." This is probably also why previous studies have had such a difficulty finding a measurable benefit in peer-review.Many of your suggestions have been implemented by one journal or another, but nowhere with the goal in mind to make the best out of the process for science. IMHO this is the most innovative suggestion in your post.
- Björn Brembs
Martin, yes sorry I missed your post somehow originally, possibly in a post-Scifoo haze. The comments there are really thoughtful and apposite to my post as well. Particularly about potential unintended side effects and multiple rounds of peer review.
- Cameron Neylon
Daniel, re: computer science its an interesting question, and a comment on the post raises the issue that the problems we're raising are much worse in a conference paper system due to time constraints. But everything I've seen around CS has been suggestions to move more towards a journal system as this is typical of "mature" subjects. Again its about wanting the cache that the magic...
more...
- Cameron Neylon
Cameron, what I took away from Lance Fortnow's CACM piece was that the current state of affairs in CS does not provide reasonable publication outlets for a large body of solid science that isn't graded as "the best" or for longer, in-depth papers. Open access is one answer to providing discoverable publication. But there's a conflict between the growth in scientific research output and...
more...
- Bill Anderson
@Bill I don't think peer review can scale and it didn't scale well. In my opinion a good consequence of fully open and attributed peer review would be a huge reduction in the number of peer-reviewed publications submitted and published. Then, on average peer review value would not be wasted like it is now as Cameron points out.
- Steve Koch
Bill, yes I can understand the motivation for the need for providing more venues for publication. Conventional (or some variant on) peer might provide a partial solution, particularly for more in depth stuff, but I guess I'm with Daniel in wondering whether as a discipline CS is better placed than most to do something different. Firstly because there is already better engagement with...
more...
- Cameron Neylon
High-energy physics has longed used Arxiv to deposit papers without peer review. But for reasons that somebody involved in the field can explain better (e.g. Enrico Balli), the community still wants/needs peer-reviewed journals.
- Martin Fenner
My understanding is that this is largely an external validation issue. So people in some theoretical physics disciplines rarely bother to go through peer review because they don't need external validation from people who don't "get" the Arxiv. But it would be good to get the perspective of someone on the inside on that.
- Cameron Neylon
I thought about the scaling issue as well. For example, a typical top-level conference has ~300 submissions and 40-50 reviewers filtering to 40 accepted papers. If we 'allowed' all the submissions, arguably there are now too many to keep up with. But if I read the proceedings now, I only find 4-5 papers of interest. So without the gatekeepers, I would probably now see 20-30 papers of...
more...
- Neil Ernst
@ Cameron, authors cope by less frequently submitting low-quality manuscripts, referees by providing careful (and often signed) reviews. To get a feeling on how it works, just take a look at any recently published discussion or final paper. Mine are at http://www.biogeosciences.net/title_a... .
- Daniel Mietchen
After they have switched from CC-BY-NC-SA to CC-BY some years ago, I see only one problem with their current system: They do not have a journal with a scope extending significantly beyond the geosciences (fair enough, given that they are a geoscientific society), which is a pity for work that does not fit that scope, since there is no real alternative out there which does the review in...
more...
- Daniel Mietchen
Another interesting aspect of their publishing - not related to peer review - is that they use http://sref.org/ instead of http://doi.org/ .
- Daniel Mietchen
Interesting. Ulrisch Proschl has left some more detailed descriptions about the EGU approach at the blog post as well.
- Cameron Neylon
Another one on the "peer review week" theme: http://dx.doi.org/10... .
- Daniel Mietchen
At Neil, above: Couldn't the gatekeeping function (somewhat necessary if space is limited at the conference) be fulfilled equally well by doing the review in public, in combination with a karma system?
- Daniel Mietchen
For an opportunity to put peer review to a test, please see https://www.hypios.com/valenti... . Background: http://ways.org/en... and https://www.hypios.com/valenti... .
- Daniel Mietchen
A pointer to a remix of Cameron's original post, re-focusing it on peer review of research grants rather than of manuscripts: http://ff.im/hsSJO .
- Daniel Mietchen
How to write a scientific abstract in six easy steps | Serendipity - http://www.easterbrook.ca/steve...
I'd say if you stray from these you ought to have a good reason.
- Neil Ernst
How to write a scientific abstract in six easy steps | Serendipity - http://www.easterbrook.ca/steve...
I'd say if you stray from these you ought to have a good reason.
- Neil Ernst
During lunch at #scio10 ScienceOnline2010, heard from Bill and Pawel about and idea for publications being a single figure ... interested in trying this out via PLoS ONE.
Bruno C. Vellutini, Daniel Mietchen, Dan Hagon and 4 other people liked this
I don't know to whom to attribute the original idea. But I like it. (Don't know details of what was discussed, would love to hear.) I have some old data from graduate school and postdoc that I would like to use to try out this idea. Basically interesting results that are possibly valuable to a few people around the globe, and which it disappoints me that they're buried out of view. From my perspective, I don't see why I couldn't just make these into publishable figures and submit these to PLoS ONE and see what happens? The only negative I can think of (so far) is that people may think I'm just trying to pad my resume. But I can think of a lot of positives if it works (including padding my resume, ha ha :) ).
- Steve Koch
Steve, you could experiment with Nature Precedings as well
- Deepak Singh
Yes, and they handle versioning as well, right? That's a great idea in combination with PLoS ONE for handling the review and final publication. (note added later: whoops! I thought you were commenting about the open peer review idea)
- Steve Koch
I don't get it. You want to publish figures? Not a research study or story, just some old data you have lying around?
- Walter Jessen
No, a figure that tells a story. Maybe an incomplete story, but a story. For example, I took data that showed site-specific binding of EcoRV to it's recognition site in the absence of Magnesium in about 2001. At the time it was a big debate as to whether EcoRV had site-specificity in absence of Mg++. Probably has been solved in the meantime. But would have been an easy story to tell in one figure.
- Steve Koch
Another example is that I took DIC microscope images of kinesin that had been sitting at +4C in storage buffer for a month or so. I saw distinct beads-on-a-string aggregates. The kinesin also becomes inactive at +4C, which is very annoying. Are these beads-on-a-string aggregates related to the amyloidosis protofibril thingies that I don't know much about? Would this give someone a clue...
more...
- Steve Koch
Hopefully someone else who heard the original conversation can chime in and correct me. The more general idea in my head is: A lot of meaningful data is suppressed because researchers and / or referees are waiting until it fits into a more traditional publication (3+ pages, abstract, introduction, methods, results, discussion, known impact, etc.). The follow-on experiments don't happen...
more...
- Steve Koch
That's really pushing the limits for peer-reviewed research. I could see you publishing stuff like that at Nature Precedings, but I'm not so sure about something *that* incomplete getting published in PLoS ONE.
- Walter Jessen
Isn't there a journal that focuses on data that never made it into published research? Or maybe that's the journal that allows you to publish experiments that didn't work ...
- Walter Jessen
Yeah I hear you. On the other hand, as an Academic Editor, it'd be exceedingly easy to review, compared with elaborate 8-figure papers. The idea, I think is that the elaborate, 8-figure paper would be written later, possibly by other people pulling together figures from multiple groups. But you can't write that 8-figure paper with other people's figures unless they've already been...
more...
- Steve Koch
I didn't think figures could be recycled in the way you propose. Once something's published, it can't be used again. Does Nature Precedings allow an author to "release" data to the public domain so that others could use it?
- Walter Jessen
I think we've heard that idea from Fabiana. As far as I understood, it's pretty much along the lines what you've just described here. To me it sounds like a peer-reviewed notebook - each sufficiently large portion of work is assessed, published and then citable. The last thing is probably most important - if your even incomplete work is published, you cannot be scooped anymore.
- Pawel Szczesny
BMC Research Notes is supposed to be a venue for putting out stuff that doesn't fit into a conventional story. PLoS ONE might not be so good because aside from all the radical stuff the papers themselves explicitly need to be "publishable"
- Cameron Neylon
Good point, Cameron -- Bill Hooker had told me about that as well, but I'd forgotten. I wasn't clear enough in my initial rambling, but I am talking about stuff that I think is "publishable" and would be approved by peer-review. But maybe it's not worth messing with what's working really well at PLoS ONE. Unless they had a special designation for research notes.
- Steve Koch
I think the problem at PLoS ONE for this kind of thing is that we are actually a quite conventional and conservative journal. So I think you might fall at the first hurdle because a single figure doesn't fit the conventional template. But I think its a great idea.
- Cameron Neylon
BMCRN publishes very small papers (called "notes" for a reason!), I don't remember any single-figure ones but it seems entirely do-able to me. As a lowly AE, I cannot say how BMC Res Notes would respond -- but I could run the idea past the Editor in Chief if you like.
- Bill Hooker
Forgot to add -- I think Pawel is right, we got the idea from @kubke.
- Bill Hooker
You mean a journal like Acta Crystallographica? They publish molecular structures (i think ...) However peer review seems a problem: http://lemeshko.blogspot.com/2010...
- Neil Ernst
Hopefully can hear more from @kubke about the original idea that I probably butchered! Yeah, I guess it doesn't have to be a single figure, but really the smallest publishable unit aka "publon." I do think peer review is important. @Neil, I don't let fraud worries influence my thinking too much. Trust is necessary for any system to work (credit Volker Vogt for teaching me this). I don't...
more...
- Steve Koch
See also: http://friendfeed.com/axiomso...
- Bill Hooker
Dunno how I missed this one but have been thinking along similar lines ( http://ff.im/fHHB9 ) and would very much like to see a real example.
- Daniel Mietchen
I posted on the other thread too: I really liked that post, Daniel!
- Steve Koch
Magazine Preview - How Christian Were the Founders? - NYTimes.com - http://www.nytimes.com/2010...
At least Texas discusses its curricula openly -- I have no idea how it is done in Canada, but I doubt it is via public hearings.
- Neil Ernst
Magazine Preview - How Christian Were the Founders? - NYTimes.com - http://www.nytimes.com/2010...
At least Texas discusses its curricula openly -- I have no idea how it is done in Canada, but I doubt it is via public hearings.
- Neil Ernst
British Library Helps Lock Down More Knowledge - Community - ComputerworldUK - http://www.computerworlduk.com/communi...
What libraries/librarians *shouldn't be*
- Neil Ernst
British Library Helps Lock Down More Knowledge - Community - ComputerworldUK - http://www.computerworlduk.com/communi...
What libraries/librarians *shouldn't be*
- Neil Ernst
My Extension Wish: Workflow in Wave - http://googlewavedev.blogspot.com/2010...
Peter Miller, Steve Koch, Daniel Mietchen and Dan Hagon liked this
These kind of lightweight workflows also have a significant potential to work well in research applications. Less need for heavy weight control but a need to push data through a specific set of processing boxes. Particularly where those processes are available as web services. The best bit - once you've done remove the input data and you've got a shareable workflow - leave the data in and you've got a shareable instantiation of that workflow.
- Cameron Neylon
How often would you say you re-use workflows vs. create new ones? Are there certain 'boxes' you keep using?
- Neil Ernst
This Car Runs on Code : Discovery News - http://news.discovery.com/tech...
The Toyota recall may be a harbinger of things to come in the auto industry, where vehicle electronics are getting more and more complex.
- Neil Ernst
This Car Runs on Code : Discovery News - http://news.discovery.com/tech...
The Toyota recall may be a harbinger of things to come in the auto industry, where vehicle electronics are getting more and more complex.
- Neil Ernst
Pen and paper rule! | Daniel W. Drezner - http://drezner.foreignpolicy.com/posts...
Commentary on why so many people persist with pen(cil) and paper, rather than the computer.
- Neil Ernst
Pen and paper rule! | Daniel W. Drezner - http://drezner.foreignpolicy.com/posts...
Commentary on why so many people persist with pen(cil) and paper, rather than the computer.
- Neil Ernst
Victoria Stodden - http://blog.stodden.net/2010...
Nice highlighting of the difference in scientific audiences. Now we are targeting the general public, whereas previously it was 'learned societies (of men)"
- Neil Ernst
Victoria Stodden - http://blog.stodden.net/2010...
Nice highlighting of the difference in scientific audiences. Now we are targeting the general public, whereas previously it was 'learned societies (of men)"
- Neil Ernst
@lucas_gonze: Yes, partly. Buildings can/should be instrumented. The data can help us evolve existing structures as well as design new ones.
Seems like most of the lessons learned are applied to new buildings, not so much evolving old ones. And it's not just materials but process of building as well.
- Neil Ernst
do you think INSIDE the box?
there is no box ... the cake is a lie.
- Neil Ernst
Obama’s 2011 Budget Proposal, Department by Department - NYTimes.com - http://www.nytimes.com/interac...
Don't need a Canadian version since we just get told the budget, there's no debate.
- Neil Ernst
Obama’s 2011 Budget Proposal, Department by Department - NYTimes.com - http://www.nytimes.com/interac...
Don't need a Canadian version since we just get told the budget, there's no debate.
- Neil Ernst
Are we taking supercomputing code seriously? - at ZDNet.co.uk - http://resources.zdnet.co.uk/article...
via Steve Easterbrook. I wonder if it's time to return to a division between the experimentalists and the theorists
- Neil Ernst
Are we taking supercomputing code seriously? - at ZDNet.co.uk - http://resources.zdnet.co.uk/article...
via Steve Easterbrook. I wonder if it's time to return to a division between the experimentalists and the theorists
- Neil Ernst
Finally reading Stewart Brand's wonderful How Buildings Learn. Wondering who can write How Companies Learn.
Cameron Neylon and Richard Akerman liked this
A great book with, I think, some good lessons for long-term software development.
- Neil Ernst
Amazed by the number of people jumping onto the #ipad vendor lock in ... people are sheep, happy about taken care off... whatever that means
Not sure that's entirely fair. People are happy having the stuff they don't much care about taken care of. Vendor lockin won't matter for a lot of people in the short term.
- Cameron Neylon
Sadly, though, I think vendor lock-in will matter a lot to almost everyone in the long run, even if it matters only in a subtle and hard to perceive way.
- Chris Granade
Oh yes, but history tells us people aren't much good at thinking in the long run.
- Cameron Neylon
I guess it depends what lock-in you are worried about. It has a web browser, so it can view web pages, and whatever you can display on them (increasingly complex apps!). Personally I got tired of debugging kernel code to figure out how to get my laptop to sleep properly, so I accepted the lock-in penalty (slow Java6 upgrade, for example).
- Neil Ernst
Let's compare to MS tablets, which use IE, and lock-in. Or to Kindle... Apple uses an open ePub format for books, one of the best open-source browsers, pushes HTML, CSS, and JavaScript standards (esp. for web apps). QuickTime is even a standardized format. You can still add web apps as an icon equal to compiled code. No, it's not Android, but it's better than MS. Personally, I'm not sure I'll buy one. But they're pushing the envelope, and that's what I like. No more WIMP interface. That's interesting.
- Geoffrey Hutchison
Geoffrey, let's not play the "X is better than Y" game, when there's discrete, objective things that need to happen for something to be open that none of the big players are doing. Android isn't where they need to be on openness either. Symbian is perhaps the closest, but they're not really that active in the US market. I know the pragmatists out there are going to say we have to take...
more...
- Mr. Gunn
Bin Laden warns of the dangers of Climate Change? Really? Must be a slow day on the Jihad newscycle... http://www.theglobeandmail.com/news....
This ought to really convince people there is no global conspiracy ...
- Neil Ernst
JoVE: Journal of Visualized Experiments - Biological Experiments and Protocols on Video - http://www.jove.com/index...
Jean-Claude Bradley liked this
A journal that publishes video blogs on life science experiments. Helpful for replication in experiments where there is a high degree of technical skill.
- Neil Ernst
Architect Arthur Erickson - http://blog.wanken.com/2010...
Have you ever been to SFU or York? There's a reason they despise Erickson in Burnaby.
- Neil Ernst
I need some personal workflow/productivity engineering. And even more than that, I need self-discipline.
Hey, look at that shiny thing over there!
- Neil Ernst
lol :)
- Michalis Famelis
Other ways to read this feed:
