Sign in or Join FriendFeed
FriendFeed is the easiest way to share online. Learn more »

Alex Holcombe › Comments

Alex Holcombe
How do we bring more debating into science? Maybe get medieval on it, like a quodlibet http://wp.me/sh4jF-630
"Beginning in the thirteenth century, quodlibets were a part of the academic program of the theology and philosophy faculties of universities… The first of the two days of the quodlibet was a day of debate presided over by a master who proposed a question of his own for discussion." I'm looking for more information on these... - Alex Holcombe
Matthew Todd
SciFoo (and how about some more Science Unconferences) - http://intermolecular.wordpress.com/2010...
"So many talks at the ACS would have been more interesting if the speaker had simply given out paper copies of their latest paper and given us 10 minutes to read it in silence then 10 minutes to talk about it." - Alex Holcombe
Hey Matt - why not come to the Blue Obelisk meet up next time at the ACS? Keep an eye on the BO mailing list just before conference time. - Noel O'Boyle
Alex++ (a preprint would do fine too...) - Egon Willighagen
I'd certainly be up for a chemistry-themed/populated unconference. Let me know about any plans (and help required). - Anna Croft
Matthew Todd
Ultraviolet light reveals how ancient Greek statues really looked http://io9.com/5616498... Oh dear! via http://twitter.com/#!/WillNoel
in most of those recreations, the color is super-saturated. Can they really determine the saturation and lightness? I doubt it- those are probably arbitrary choices, and misleading to make everything saturated, right? I was under the (possibly-mistaken) impression that startlingly pure and saturated paints were difficult or impossible to come by back then.. - Alex Holcombe
Matthew Todd
Australian Code for the Responsible Conduct of Research. Guidelines on what to do with data. Will be referencing this in grant applications for data management. http://www.nhmrc.gov.au/publica...
From our eResearch showcase at Usyd, emphasis was placed on the re-use of data, particularly by Andrew Treloar (http://twitter.com/#!/atreloar) from ANDS. He said data needs to be managed, connected, findable and reusable. Advantages are efficiency, validation, integrity, value for money (government wants this) and self-interest. - Matthew Todd
Treloar gave the interesting example that more papers are published from reuse of Hubble data than from data initially collected from the instrument, i.e. there was arguably more value from reuse. References for this (thanks Andrew) are http://archive.stsci.edu/hst... using methodology at http://archive.stsci.edu/hst... - Matthew Todd
published in 2007. I wonder if there are any updates - Alex Holcombe
Rich Apodaca
"Why is it that some lab groups and managers will spend countless days travelling to schools around the globe to give talks to small audiences, but won’t devote any time to posting the same information on their Web sites that are accessible to millions of people?" - Rich Apodaca
Because there is little or no formal credit within institutions and grant awarding bodies, unlike for conference presentations & elite seminars. - AJCann
What AJ said: every talk I gave is one line in my resume. - Björn Brembs
Sad but true. People seem impressed when I give a talk/sit on a panel. No-one gives a damn when a blog post gets 2500 page views (that's a lot for me...) - Cameron Neylon
HTML is hard; let's have a meeting. - Bosco Ho
It's sad but the scientific community reminds me of traditional media, etc. Holding on to something whose heyday was in the past - Deepak Singh
+++ Deepak - Bill Hooker
I agree with @Cameron, the old school metric of "authority" still value invited ( and uninvited) talks more than blog readership - Hari
Generally true, but it depends who you are invited to talk to. Keynotes at international conferences are certainly credited in UK RAE/REF, as is media work (less so). - AJCann
+1 Neil. Most points for publications (n x IF) then way down the list would be invited talks, then way down again would be outreach if you ever mentioned it at all. There is no metric for impact associated with the creation of web content that is highly visited. You're starting to see statements like "was one of the top ten most downloaded papers in the journal for the period March 1-April 24." Outreach counts a little in promotion decisions, but there are far fewer of these than grant proposals. - Matthew Todd
[The original post was actually mainly about web pages, but there is a comment there about posting seminars on the web too. Becoming difficult to justify not doing this. But merits of real vs. virtual talks/lectures is a complex topic.] - Matthew Todd
@Neil: I agree about crossing anybody's mind, but if we had the incentives in place, it would cross their minds :-) Also, I have had places where you would have to list national and international invited talks if you applied there for a job. - Björn Brembs
Yes, terrible that ppl still using so much carbon for a few hours of face-time, but for networking, there's still nothing like a real handshake. And sadly, at least for promotions and fellowship apps in Australia, I think that although invited talks are well behind publications, they are taken as a substantial indication that one is one of the top 'experts' in a field. And perhaps the best way to get invited to more talks is to give an invited talk; audience members will later remember and recommend you. - Alex Holcombe
Alex Holcombe
Make evidence charts, not review papers: http://wp.me/ph4jF-5S . Soft-launching http://www.evidencechart.com , and hoping for some feedback
Great. Amusing that you have the Holy Bible as evidence in the toy chart. - Matthew Todd
Yes! it's an example from the 16th century, there weren't many refereed papers available :) http://www.evidencechart.com/charts... - Alex Holcombe
the most surprising thing I read about the heliocentric/geocentric debate is that in Galileo's time, supposedly noone brought up parsimony as a factor. So apparently people didn't consider all those epicycles of geocentrism to be a point against geocentrism. Even tho that's what I always assumed http://alexholcombe.wordpress.com/2008... - Alex Holcombe
Minor niggle - but the site gives the impression on the landing page that you need to login before you can see anything which is slightly offputting for someone just wanting to look around. Would be good to see the list of charts from the landing page so you can explore what's been done already. - Cameron Neylon
thanks Cameron. And unfortunately there's a lot more niggly user-interface imperfections beyond that one ;) we'll use your suggestion and others to improve it little by little - Alex Holcombe
Just thinking off the top of my head, for scientific issues, would it be interesting to ask the question, what papers are not included on this list? Again trying to tackle the selection bias you mention in the post... - Cameron Neylon
Since when do review papers tout theory? Maybe there is a class of them that do -- and perhaps this varies across scientific disciplines -- but the majority of reviews I read and allow students to use (and teach to write!) are NOT arguments supporting a theory -- they aren't even particularly biased. They do offer a personal perspective through selection of articles, organization of... more... - Mickey Schafer
"It allows one to create systematic reviews of a topic, without having to write many thousands of words, and without having to weave all the studies together with a narrative unified by a single theory." -- Alex, how have you created a systematic review if all you've done is categorized data? Or, simply gathered all the articles into one place? What's the basis for understanding one... more... - Mickey Schafer
Alex, totally agree with you on the experience reading reviews as it relates to expertise. But I think the conclusion that reviews are thus biased is incorrect; expertise so changes perspective that relatively little one reads as an expert feels "unbiased" -- the science has to be phenomenally well done for us to be convinced. I read language stuff done by neuroscientists sometimes and find it frustratingly distant from what happens in fields of linguistics. But the bias in this case is mine. - Mickey Schafer
I absolutely agree that the requirement for the narrative format in review articles is as constraining as it is for communicating experimental results. Lets hope more people start experimenting with more modular formats such as this and share them on the open web. - Jean-Claude Bradley
From a discourse linguistic perspective, Jean-Claude, I find that statement really interesting. Maybe b/c I come from a humanities background way back when, I do not experience reviews as narratives, nor do I teach them that way. But I very much agree that the IMRD is narrative in feel b/c an idealized sequence is implied that may never have happened (and I tell this to students as... more... - Mickey Schafer
Mickey - I think that reviews are expected to be narratives in the sense that there is a "story" to be told. Reviewers will expect an historical perspective, key people involved, an exhaustive review of different aspects of the topic, etc. That is why it is annoying to read reviews in one's own field because we always find missing components. However just sharing information in a modular way as is shown here can be done because there are no such expectations. - Jean-Claude Bradley
Jean-Claude, it's the contention that one is "just sharing information in a modular way" that I'm disagreeing with -- not b/c it's a bad idea, but b/c it is not value-neutral. No chart is without a bias; it cannot exist without terms defining where entries go, so by definition, is not "merely" sharing. That would be the exchange of raw data; charts are a form of filtering. As for... more... - Mickey Schafer
this is an interesting organizational tool. I agree with Mickey - non-scientists may not find it useful and might event mock such an attempt to "pidgeonhole" ideas or see the format as a way to structure an argument to achieve a desired result. As a librarian with a lot of colleagues in the humanties and social sciences I am routinely surprised by how different I think about concepts or... more... - Elizabeth Brown
Mickey- I believe that any format will involve its own biases, so I didn't mean to imply that evidence charts are always better than reviews; just different in sometimes-useful ways. Any form of discourse has its own norms and perhaps even implicit assumptions. My own experience of writing articles is that to make it 'work' with the word limit provided and to get the 'story' to slot nicely into the reader's head, I am pushed to ignore certain difficulties and even to sweep things under the rug. - Alex Holcombe
Hi, Alex -- yep! I discuss these struggles with students all the time, and show them how it can change the interpretation from a reading point, too. The thesis students in particular struggle with this as PIs push them to condense 2 years of research into 10 pages or less. One student nearly cried 2 years ago when a single graph covering 1/2 page was the result of 2 years of life with... more... - Mickey Schafer
Hi, Alex. I've just spent the afternoon reading several reviews written in various areas of biology. I think I must amend at least some of my previous comment: I read several papers that were clearly making arguments. This is quite different from what I usually read in medicine where a review is more, well, review-like. Could have something to do with the place I was reading (the... more... - Mickey Schafer
Mickey yes, now that you mention it, it's also my impression that medical reviews tend to be more "just the facts, ma'am". I haven't been expert enough in a medical topic to judge whether they might somehow be just as biased, but being more subtle about it. - Alex Holcombe
Alex -- I am not qualified as a medical expert, but from a linguistic standpoint, I think the bias is there, but represented in the selection of materials rather than in how language is used. Frankly, I found much of the stuff I read today refreshing insofar as authors weren't hiding their agendas; I don't mind bias when the writer says "Look, I think this is what we should be doing and... more... - Mickey Schafer
Mickey- this is interesting; I have seen a lot of tables in papers, but rarely or never one with each column representing a theory and each row evidence. In a systematic review for example, each row might be a study and each column a methodological trait of the study, with other columns representing the results. If you could send me the references, I'm very interested to have a look at the ones you're referring to. - Alex Holcombe
I will need to go back to web site and see if I can locate a good example -- the "Trends" journals published by cellpress seem to require "boxes" of information -- usually 3-4 per 8 page pub. A few of those were charts. I'm not at a VPN enabled computer at the moment, so this may need to wait until Monday. - Mickey Schafer
This is awesome. I wish I knew it existed when I pushed my students to make this: http://pharmacomicrobiomics.com/papers... - Ramy Karam Aziz
Sure it is awesome, i like this idea, new vision and interesting! @ Dr. Ramy as u know when one do things from the ground up he learns more:), so it is still good experiment for me^_^ - len hat
Alex, here is a link to one paper that seemed to be using an evidence chart of some kind -- see Table 1, "Evolutionary hypotheses..." -- http://bit.ly/d15jVQ (sorry for the pdf -- wasn't sure if you had sub or not, and just seemed faster). - Mickey Schafer
@ Ramy, that is a nice education tool -- I have my undergrads do something similar, but use a Q/A format and have them answer in a bulleted list. Of course, my students then have to write those bulleted lists into prose, which is the point of the class for me -- it's also a great way to help students avoid unintentional plagiarism. Whether lists or charts, the information is distinct from the paper and students can write with clearer heads - Mickey Schafer
Alex Holcombe
Discussion of fraud by scientific researchers in China. Presented as threatening China's economic ascent! scientific journals apparently more important to the world than I realized http://www.nytimes.com/2010...
"a third of the 6,000 scientists at six of the nation’s top institutions admitted they had engaged in plagiarism or the outright fabrication of research data. In another study of 32,000 scientists last summer by the China Association for Science and Technology, over 55 percent said they knew someone guilty of academic fraud." - Alex Holcombe
"The software, called CrossCheck, rejected nearly a third of all submissions on suspicion that the content was pirated from previously published research." - Alex Holcombe
Cameron Neylon
Online mathematics finally takes off … - http://gowers.wordpress.com/2010...
Welcome to the globalisation of education...and subsequent destruction of our current western educational institutions... - Cameron Neylon
It's outsourcing, rather than innovation. - Matthew Todd
Yep the real crisis hits when its no longer innovation that hurts you, it just implementation... - Cameron Neylon
Seems to me to be a reasonable approach for two cases; (1) occasional stimulation of the learning process by changing the mode of instruction (this can be achieved otherwise, e.g. by visiting a company and taking a guided tour of its mathematical underpinnings); (2) regular stimulation for those whose mathematical abilities are well beyond average (in either direction) - not easy if these pupils are to be in the same room with the rest of the class. - Daniel Mietchen
They've gone with cheap (but highly educated) labour, but upmarket versions can't be far behind. Such as: I want my child to have an online tutor from Singapore (or country X), because their students get the best international scores in math (or subject Y). Similar trends could happen in higher education..? - Alex Holcombe
Alex Holcombe
Is scientific communication today in accord with Feynman's basic vision of scientific integrity? http://alexholcombe.wordpress.com/2010...
"If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it…" -Feynman, 1974. I'm still waiting for this to be the norm! - Alex Holcombe
Some more quotes from the cargo cult lecture ( http://yost.com/misc... ) linked from the post: "We obviously have made no progress—lots of theory, but no progress— in decreasing the amount of crime by the method that we use to handle criminals." (also fits with a recent commentary on economics: http://biz.thestar.com.my/news... ) - Daniel Mietchen
"For example, I was a little surprised when I was talking to a friend who was going to go on the radio. He does work on cosmology and astronomy, and he wondered how he would explain what the applications of this work were. “Well,” I said, “there aren’t any.” He said, “Yes, but then we won’t get support for more research of this kind.” I think that’s kind of dishonest. If you’re... more... - Daniel Mietchen
"I often talked to the people in the psychology department. One of the students told me she wanted to do an experiment that went something like this—it had been found by others that under certain circumstances, X, rats did something, A. She was curious as to whether, if she changed the circumstances to Y, they would still do A. So her proposal was to do the experiment under... more... - Daniel Mietchen
"All experiments in psychology are not of this type, however. For example, there have been many experiments running rats through all kinds of mazes, and so on—with little clear result. But in 1937 a man named Young did a very interesting one. He had a long corridor with doors all along one side where the rats came in, and doors along the other side where the food was. He wanted to see... more... - Daniel Mietchen
"Nowadays there’s a certain danger of the same thing happening, even in the famous (?) field of physics. I was shocked to hear of an experiment done at the big accelerator at the National Accelerator Laboratory, where a person used deuterium [also known as “heavy hydrogen”]. In order to compare his heavy hydrogen results to what might happen with light hydrogen” he had to use data from... more... - Daniel Mietchen
Alex Holcombe
The Science Collaboration Framework (SCF) is a reusable, semantically-aware toolkit for building on-line communities http://sciencecollaboration.org/
What does that mean? "Reusable"? - Matthew Todd
hm, good question. Sounds like it could refer to it being open-source, but if that was the case why not just say 'open source' since people know more what that means - Alex Holcombe
Matthew Todd
YouTube - Simulation of millisecond protein folding: NTL9 (from Folding@home) - http://www.youtube.com/watch...
YouTube - Simulation of millisecond protein folding: NTL9 (from Folding@home)
Play
Interesting choice of music and amusing slow-mo replay of C-terminus folding. Interesting also that Stanford is able to copyright this, given it was a distributed computing effort. Perhaps the copyright just applies to the movie production. - Matthew Todd from Bookmarklet
Matthew++ - Egon Willighagen
how about linking to some biochemical thing created by the 2010 MacArthur-winning australian animator? - Alex Holcombe
Alex Holcombe
PLoS thinking about requiring authors to write a 'data availability statement'. How to word it to avoid the pitfalls? http://alexholcombe.wordpress.com/2010... Hoping to duplicate some fraction of Cameron's crowdsourcing success
pitfall 1: authors would have to write statement at time of submission. Most won't have tried yet to deposit their data anywhere. So they may write something inconsidered that won't actually be true come publication time. Indeed, many will say the data will be available but then they won't follow through. - Alex Holcombe
pitfall 2: Policing. PLoS would feel some responsibility to police whether the authors' data is available, an additional burden on PLoS - Alex Holcombe
pitfall 3: PLoS will look less kindly at authors who don't make their data available and don't have a legitimate excuse like human data privacy issues. But may be impossible to apply a uniform standard- expectations differ across fields, most reviewers still won't expect any data sharing, etc. If data availability is a factor but not completely consistently, is that fair? - Alex Holcombe
Have you look at the NSF plan? Maybe you could use some of that for ideas? http://www.nsf.gov/news... - Joe - Systems Analyst
Keep an eye on what BMC Res Notes is doing (http://blogs.openaccesscentral.com/blogs..., http://blogs.openaccesscentral.com/blogs...). I'm not sure just how political it all gets, but P.ONE and BMCRN have a lot in common and it would be great to see them work together on this sort of thing (data standards, standard requirements for data availability, and so on). - Bill Hooker
are there expectations that data will be shared locally? I don't think this will be possible for all authors. - Elizabeth Brown
Thanks much for these pointers and comments. Elizabeth, I'm not sure I know what you mean by 'shared locally'. I was thinking they'd have to share it globally, which includes locally :) but probably I'm misunderstanding - Alex Holcombe
I think Elizabeth means taking responsibility for hosting the data locally. And I think the answer is no, not necessarily, it could go to Supp Mateials, it could go to BMC Research Notes, or to an appropriate data repo or IR, or some local server. Actually they can just make it available in response to email requests and satisfy existing policy. The aim here is to get people to think a... more... - Cameron Neylon
Yes, I meant hosting locally. Some authors (esp. at small colleges and orgs) may not have a local IR or server space. I agree the email option is a good way to satisfy that though. - Elizabeth Brown
regarding Policing ... after the authors addressed the reviewers comments but before it is formally accepted and published is around the time where editors have the most leverage. this is when the editor should make sure that the data is available in supplementary info or databases. If it is hard for the editors to evaluate this then include in the feedback you ask from your reviewers to detail if the authors should provide any extra data to comply with your policy - Pedro Beltrao
re: Elizabeth's suggestion regarding email as a dissemination mechanism. There is quite a lot of evidence in the literature about how poorly sharing-upon-request works in practice that an effective policy should steer clear of that route entirely. PLoS should encourage and enable authors to archive the data before publication so that the article can contain a permanent link to the data, and authors no longer have the burden or responsibility of mediating data reuse once the article is published. - Todd Vision
You know, I've never even known a case where 1 scientist requested to see another's data. The climate is such that I'd probably feel like to jerk doing so, unless it was a friend. And I never come across data made freely available either. And yet I feel people would devour the data if they had the chance. Definitely one of the difficulties has to be how to decide which data to share -... more... - Jason Snyder
About email distribution, as Todd says it hasn't worked well in the past because most authors don't reply to requests or refuse them (there was a recent PLoS ONE paper showing this). Jason: I've asked for others' data on two or three occasions. I know what you mean though- I would have felt like a jerk doing so if I wasn't used to being such a callous person already :) If I remember... more... - Alex Holcombe
Agree that email is a bad mechanism in general terms and tends to lead to low compliance, but the point is that authors _can_ satisfy the current policy by just replying to an email. On the other hand, if they say, 'available by email' then arguably the editor could request it by email and stick it in a comment... - Cameron Neylon
Cameron Neylon
Ok crowd-sourced proposal time. Topic: Research Metrics. Aim: Develop and deploy credible metrics frameworks that encourage "open" behaviours. Value: $30k. Go.
More precisely: I have an agenda, some ideas, and a possible line on about this amount of money. Am interested in any ideas on how most effectively to create change with about this resource and anyone who is interested in taking part (with the caveat that this is not a whole lot of money if it gets split a lot of ways). - Cameron Neylon
Particularly interested in bringing existing projects/initiatives together to maximise effect. I'm aware of a range of things going at the moment but would appreciate pointers to any others. - Cameron Neylon
Quite. More downstream funding might be possible if there was demonstrated value out of the "pilot" or whatever it might be. There are a bunch of JISC funded projects that are sort of, but not really, in this area at the moment (they are focussed on citation) so that's another place to start. - Cameron Neylon
Do we have any existing examples of metrics that encourage open behaviors? - Mr. Gunn
Well I would argue that re-use metrics (downloads, data citation) push in the right direction. I couldn't point to hard evidence to support that tho. - Cameron Neylon
Perhaps the pilot could be to gather such evidence? - Mr. Gunn
Thomson-Reuters shares went down hard in the last 26 minutes. - pn
@D Peter Binfield has talked about Google Analytics style tools provided by a third party to do cross publisher ALMs. Something I always thought was an interesting idea. It could at least provide a consistent way of dealing with that kind of gaming? - Cameron Neylon
@Mr Gunn: It would be difficult to do that prospectively in the time frame of a small grant. Can you think of clean examples which could be looked retrospectively? Key issue is actually persuading funders to adopt something of course. - Cameron Neylon
@Paulo: Don't worry, my next idea will take out Pepsi... - Cameron Neylon
No idea how you'd do it but what I'd really like is to understand _why_ someone downloaded something. Was it for the figures, a specific piece of data, or for the beauty of the whole work... - Cameron Neylon
It's that or some sort of single use citation token as Claudia (I think) suggested at one point. Either would likely lead to bias anyway... - Cameron Neylon
Dorothea - Have you looked at the PIRUS2 and MESUR projects? They are looking measuring usage statistics from mulitple locations (PIRUS) and assesing quality of content (MESUR). They already have done a lot of preliminary work. I would suggest usage metrics as they both argue usage happens earlier in the research cycle than citations. Is there a way to look at any open behavior at the grant application stage, like this thread? That's even earlier in the research process. - Elizabeth Brown
Cameron, if I could think of a way to do something like that retrospectively, I'd already be doing it ;-) Agreed that usage metrics are the way to go, though, to move upstream from citations. - Mr. Gunn
Interested that no-one yet has explicitly brought up the issue of data that metrics are based on... - Cameron Neylon
@Mr Gunn, yes but how far upstream do you think it is possible/sensible to go? - Cameron Neylon
Hate to say it, but I think your funding is short a few zeroes. If this were something you could do for $30K, PLoS or BMC or even Thomson Reuters would already have done it. But so as not to be all negative all the time: how about expanding Heather Piwowar's work on data re-use? The aim would be to compare the rewards (citations, collaborations, etc) reaped by open, re-usable datasets... more... - Bill Hooker
It's alright. I'm not proposing to tie up all the loose ends for this kind of sum. Although I would say that in many cases PLoS/BMC don't have that kind of spare cash just sitting about to do something different. Question to my mind is how to make the most difference with that kind of amount...not necessarily to solve the problem. - Cameron Neylon from twhirl
The question with this kind of thing is always how best to take a small amount and multiply it with existing funds that other people already have... - Cameron Neylon
In that light, I think my dataset idea looks a little better: you might be able to tie in to what Heather is doing now. The real trick would be to get a few *more* related efforts coming together under the same roof. - Bill Hooker
Certainly I'd be pretty happy with building a reference dataset of data citations and/or uses. - Cameron Neylon
I agree that a) this is a very important issue, and b) $30k is not a lot of money. I would use the money to talk to the stakeholders (researchers, institutions, funders, publishers, vendors) - and that would include Thomson Reuters - and write a report that summarizes the key issues and comes up with a set of recommendations. On a very small scale I have started to do just that (Cameron knows about this). - Martin Fenner
Agree with the talk to stakeholders. Less sure about the writing of reports...but that's maybe a personal bias...as a focus for discussion yes, but as an end in itself? But definitely agree that engaging the relevant people is critical. There is no point building/writing about anything that doesn't serve the needs of those whose metrics will be taken seriously by researchers... - Cameron Neylon
Reports indeed have a bad track record. Maybe principles and action points? - Martin Fenner
A set of agreed principles would be a very interesting outcome if it were feasible. Action points seems easier but then actually turning them into action becomes the challenge... - Cameron Neylon
About citations of data and other open science outcomes like code, can current citation metrics make a separate count for numbers of citations of data and code as opposed to articles? Is there some agreed meta-data field that indicates something is data rather than an article? If someone wants to reward open science, then we need to be able to count and aggregate open-sciency citations. - Alex Holcombe
Yes, progress being made on citing data but citing data within a publication rather than the publication itself is some way off. And citing code is still pretty rare (partly because "publishing" code is still pretty rare). Martin and Mummi did a nice double act at Science Online London looking at the use of ORCID to mix and match data and paper citations. - Cameron Neylon
I would suggest to talk the David Shotton (Oxford) on CiTO, and get citation types into the equation... and develop a simply H-index-like thing that is weighted for citation types... type: usesMethodIn -> large positive for H'-index... if type: refutes -> negative (or really small positive) on H'-index... - Egon Willighagen
To my mind the best thing would be a pilot where you pay for (part of) a person or student to piggyback on an established database with appropriate data sets (at EBI, for example, this would be something like PRIDE or ArrayExpress) and track downloads then provide example chunks of end-of-grant-style reports of impact and write a paper or three on it (one on mechanics, one with submitters, one forward-look), etc. - Chris from twhirl
This could then make for a nice proof-of-concept thing to drag in some real money. - Chris from twhirl
I think it's reasonable to go up to the level of the individual experiment, but not sure how possible it is. I really do like the idea of looking at the qualities of a citation, rather than just the A cites B model. - Mr. Gunn from YouFeed
interesting. thinking. fwiw, I have a research proposal drafted and submitted, proposing to lay groundwork on patterns of data citation and reuse tracking. Might could be synergistic? Submitted for what is supposed to be a double-blind review process, so in an attempt to make it ungoogleable I'll just put a link here: https://docs.google.com/documen... - Heather Piwowar
I also have a large set (11k) of "data creation" papers whose citations we could follow, examining for data reuse patterns and citation benefit of sharing. Totally agree on CiTO. Ideally would supplement with data creation papers in other fields/datatypes too, to illustrate differences across fields and datatypes. Not easy, but have some methods. Also, could work with GEO and... more... - Heather Piwowar
That is a great proposal. Wish I were reviewing it... - Chris from Android
Made me think of something that might be complementary: to interview a set of data producers who have submitted somewhere about their attempts to make their data traceable, to claim credit in subsequent funding proposals, and how they think that played out. And ideally, to then revisit those applications with the funders to whom they were submitted. - Chris from Android
I think anyone submitting a proposal like those above should look at the MESUR project in particular- they've already looked at usage stats and compared this to article citations to estimate the quality of work. Some of the questions above have already been answered. My question is, can a list of open behaviors be clearly identified? I think that might be more challenging than the metrics, since most of the tools are evolving so quickly. - Elizabeth Brown
MESUR definitely high on my list (as is PIRUS now) and will also talk to Carl Bergstrom as well. Also liking Chris and Heather's comments. - Cameron Neylon from twhirl
I would like to extend something like http://wikidashboard.appspot.com/ (plus http://orcid.org/ - once workable) to contexts like http://www.science3point0.com/coasped... . Might solve the on-wiki part of the problem. - Daniel Mietchen
Are you crowd-sourcing tips/pointers/suggestions for your own proposal or are you asking people to contribute directly to a 'community' proposal (via google docs for example), with the intention that people who contribute to the grant get a slice of the funding ? - Greg Tyrelle
Greg, A bit of both. I have my own ideas and want to make sure I haven't missed anything obvious but also am looking to try and form a network of people to take this forward in some form. I'll move onto an online document in the next few days to which people are welcome to contribute with the idea of taking part. Bottom line is that at this level there won't be money for much beyond maybe travel/meetings and one FTE for a couple of months. - Cameron Neylon
My basic idea is meeting with stakeholders followed by hacksession "sandpit" out of which the group decides to fund someone already in post to be extended to take most promising aspects a bit further. Open minded as to whether that aspect might be liberated data, some sort of mashup, or a protocol/framework, or a document. - Cameron Neylon
Would love to participate. Sounds interesting. - 'Mummi' Thorisson
Throwing my $0.02 worth into the pool: how about considering also attribution and metrics for less granular contributions than papers/datasets: e.g. output of curator activities, or 'nano-publications' mined from the literature? (http://www.nbic.nl/uploads... http://laikaspoetnik.wordpress.com/2010... ) - 'Mummi' Thorisson
Would certainly very much like to do that in the longer term. Keep it as a longer term target that any thing that gets built needs to be open to? - Cameron Neylon
Agreed. Try and keep the concept of 'contribution' that one gets credit for (and can ideally be formally cited) quite open-ended. - 'Mummi' Thorisson
First GDoc version: https://docs.google.com/documen... It's light on actual substance at the moment but that is kind of deliberate. - Cameron Neylon
Can't edit. :) - Andrew Lang
Ok, fixed the editing problem I think. I ticked the wrong box! :-/ - Cameron Neylon
What D said; I can't think of any substantive changes to suggest. Success will, of course, depend on picking the right people to be at the initial workshop... - Bill Hooker
Absolutely - so phase II - who wants to be involved, who should be involved? Already some good suggestions above. This will necessarily be UK centric but hopefully not entirely so. - Cameron Neylon
Just made some edits. I find two points missing: (1) If funders were to allow for public peer review of (at least some) proposals, the re-use of such proposals and reviews would provide a good use case, and probably be of interest to the funders. (2) As already mentioned above, wikis have solved much of the problem already (... more... - Daniel Mietchen
Agreed. But I would say this proposal is positioned to ask funders what they want first. Not for us to tell them what they should do. On the timeframe of this grant persuading any UK funder to do public peer review is a non-starter. In terms of Wikis, yes as a framework there is a lot there, but as you say the problem is you'd have to bring that content in because getting people to use them is difficult. But as you say the COASPED project would be a good example of the kind of thing that could be done. - Cameron Neylon
So tldr: yes they're good points but they're kind of deliberately not the focus because the intention is to be open ended as to what will come out the other end. - Cameron Neylon
Fair enough. I would like to be involved, but if it has to be via the UK, perhaps @science3point0 would be the best way to do that - would this counts as a "Infrastructure Organization"? - Daniel Mietchen
Sorry for arriving late to the conversation, I've been dipping out, but I'd like to reaffirm Daniels sentiments and would happily offer S3.0 be It as a hosting service to the proof of principle or to act as an infrastructure organisation, either in it's current format or in a tailor made format for those involved, closed group etc - science3point0 from iPhone
Alex Holcombe
Fwd: Where are all the scientific debates? They’re disturbingly hard to find - http://alexholcombe.wordpress.com/2010... (via http://friendfeed.com/cameron...)
Reminds me the ill-fated Debates in Neuroscience journal: http://www.springer.com/biomed... - never saw the light of Pubmed but could have spiced thing up. - Jason Snyder
Wow I didn't know about Debates in Neurosci. Anybody know anything about the reason for its demise?! - Alex Holcombe
Bill Hooker
"Readersourcing.org is an independent, third-party, non-profit, academic/scientific endeavour, aimed at quality rating of both scholarly literature and scholars." - Bill Hooker from Bookmarklet
I hope this succeeds. But how will it get critical mass? See also thirdreviewer.com, journalfire. Amassing sufficient comments may require novel functionality or a replacement for existing workflow, as in Connotea, Mendeley etc. - Alex Holcombe
literature comment aggregation sites are popping up like "facebook for scientists" were two years ago. Do any of them have a "secret sauce" that will pull people in, or do they all think they're the only one with the idea? - Mr. Gunn
Wow. And here I thought I've already read every paper/author on the subject of improving peer review/scholarly communication. Yet, I'm somewhat ashamed to admit I've never heard of this guy (behind this site) before. - Wobbler
Alex Holcombe
norm of shoddiness and slacking in Italian academia http://www.sociology.ox.ac.uk/documen...
"A classic case to illustrate this question is the timing of appointments: we agree to meet at 10 when we both know that we will both be late. The question is how late? How can we coordinate so that we both arrive, say, at 10:15? There is an infinite series of minutes by which one can be late. If each fears to arrive earlier than the other how is one to decide at which minute to show up? In principle this could lead the LL equilibrium to unravel, and no one ever to show up. So what, if anything, keeps it from doing just that?" - Alex Holcombe
Alex Holcombe
Rhetoric of science, as a branch of inquiry, does not look at scientific (natural science) texts as a transparent means of conveying knowledge, but rather it looks at these texts as exhibiting persuasive structures. From http://en.wikipedia.org/wiki...
I believe science relies overly on journal articles and the like. Need new forms of communication, possibly non-prose. E.g. evidencecharts (evidencechart.com) that I'm working on - Alex Holcombe
Yes, it is true that those of us considering science as a branch of inquiry believe it to have persuasive structures. We are often surprised at the vehemence of scientists who we believe mistakenly claim themselves driven by data (none of which can be separate from the human perspective so is tainted from the beginning by a species-delimited perspective). It is akin, we sometimes think, to evangelists claiming non-believers will find themselves condemned to hell for all eternity. As if eternity existed. - Mickey Schafer
Michael Nielsen
Ivory Tower Reconsidered - http://www.colby.edu/colby...
"How long will you need to find your truest, most productive niche? ... however long it takes, it will be time well spent. I am reminded of a friend from the early 1970s, Edward Witten. I liked Ed, but felt sorry for him, too, because, for all his potential, he lacked focus. He had been a history major in college, and a linguistics minor. On graduating... he concluded that, as rewarding as these fields [were], he was not... cut out to make a living at them. He decided that what he was really meant to do was study economics... he applied to graduate school... after only a semester, he dropped out of the program... Ed became an aide to Senator George McGovern... [&] wrote articles for... the Nation and the New Republic. After some months, Ed realized that politics was not for him... [what] he was really suited to study [was] mathematics. So he applied to graduate school [... and then dropped out]. He realized...he was really meant to... study physics" - Michael Nielsen
Fifth time lucky... - Michael Nielsen
and they say you have to start early at math and physics, to be a theoretical superstar - Alex Holcombe
Alex Holcombe
Victory against political interference in science http://news.sciencemag.org/science...
climate change scientist Mann: " a victory not just for me and the university, but for all scientists who live in fear that they may be subject to a politically-motivated witch hunt when their research findings prove inconvenient to powerful vested interests.” - Alex Holcombe
Alex Holcombe
excellent counterpoints in review of book that proposes to reform universities by reducing the research they do: http://www.nytimes.com/2010...
"authors’ deepest scorn is reserved for the claim that good teaching depends on research, and their most extreme proposal is that universities drastically reduce the amount of research they support, by “spinning off” medical schools and research centers, discontinuing paid sabbaticals and abolishing the current system of promotion and tenure, a system that tends to reward research productivity more than effective teaching. " - Alex Holcombe
"Have they spoken with undergraduates who have enjoyed the privilege of assisting a top investigator in an active, federally financed laboratory? In my own anecdotal experience, the best of those students, far from shutting themselves away in a narrow specialization, are very likely spending their time outside the lab in life-expanding service activities that, again, were quite beyond the ken of undergraduates in earlier generations." - Alex Holcombe
if I didn't know that at many iSchools there aren't any full time faculty capable of teaching cataloging or reference - but anyway - Christina Pikas
The handing of teaching wholesale to grads and adjuncts is not as common outside the US. In fact its highly discouraged in the UK - people have been disciplined for getting their postdocs to give their lectures. I can't help but feel that the good experiences of being exposed to research within education do not depend on those researchers being teachers. And the case for research led... more... - Cameron Neylon
I take on several undergraduate or masters level students into research projects each year and I go nowhere near any teaching - don't even work for a teaching institution. When my institution offers summer projects for undergrads (or even school kids) we get inundated so there are mechanisms that exist. I've no doubt there might be better ones as well. The research project is a pretty... more... - Cameron Neylon
I guess overall I question whether having an unhappy researcher in a lecture theatre with 200 students talking about something that they may not be that interested in is "exposure to researchers" in any positive way. The real issues lie in how valued teaching is (or is not) and how much effort people are willing to put into it. - Cameron Neylon
my research lab has a bunch of interns from all levels from high school through grad school (and then a very very few post docs - we're not popular with them bcs we aren't a good stepping stone to academe). i hadn't thought of them in this case. The problem is that these positions are very few and far between. Many students don't get selected or can't because they can't afford to live here and mom and dad are in a rural area - Christina Pikas
True that many researchers can't/won't teach well. Maybe some universities are moving to take teaching more seriously, but isn't a bigger priority finding new streams of research income? Will that trend simply accelerate as online learning matures and learning from lectures becomes more inefficient? - Alex Holcombe
Dorothea, another way that undergrads learn about research is in the classes that I teach -- discipline-specific writing classes where students read the primary literature, get that "meta" view of what a discipline looks like, and in a few classes, also conduct mini-original research projects (which I do in 3 of the 6 classes I teach). In some cases, my classes also align with methods classes, particularly in sociology and psychology. - Mickey Schafer
Another thing to consider when placing undergrads in research labs is mentorship -- I'm shocked by how diverse the mentorship experience is, with some students emerging from 2 years of research with knowledge/skills equivalent to 1 year of grad school and publications already in place -- and others leaving with little but resentment for time wasted. - Mickey Schafer
Alex Holcombe
The Hauser affair is a teachable moment for explaining why open science and open data are needed (e.g. http://alexholcombe.wordpress.com/2010...) but not everybody is interpreting it this way. Some say discovery of possible Hauser misconduct means science already self-corrects http://www.psychologytoday.com/blog...
Please use this teachable moment. To teach! - Alex Holcombe
Anyone know where the retraction note is? As of now at least, it seems nowhere near the original article at http://dx.doi.org/10... . - Daniel Mietchen
Sheesh, so still have this problem of not linking the retraction notice to the original?!... Furthermore I think one of the news reports noted that the paper that Hauser subsequently said he failed to replicate continues to be cited on its own.. - Alex Holcombe
The closest I found for the retraction note was "Cognition editor Gerry Altmann confirmed that a forthcoming issue will include a retraction of a 2002 study that found that cotton-top tamarin monkeys are able to learn patterns." at http://harvardmagazine.com/harvard... . - Daniel Mietchen
Alex Holcombe
Fwd: The Hauser affair is a teachable moment for explaining why open science and open data are needed (e.g. http://alexholcombe.wordpress.com/2010...) but not everybody is interpreting it this way. Some say discovery of possible Hauser misconduct means science already self-corrects...
here's one (that I disagree with) saying Hauser misconduct discovery is because science consistently self-corrects: http://www.psychologytoday.com/blog... - Alex Holcombe
Alex Holcombe
Bill Gates says in 5 yrs the best education will come from the web http://techcrunch.com/2010...
I hope he's right; could mean in 5 yrs I will be doing something more valuable with my teaching time than delivering lectures. As better lectures than mine will be available on the web. - Alex Holcombe
One wonders. Did you see the talk I posted, which has a similar message? http://ff.im/oPjOx - Matthew Todd
and "640K ought to be enough for anybody" ... - marcin
Mat: have watched it now- the Jarvis talk is best thing I've seen on this so far http://ff.im/oPjOx . Marcin: Wasn't it 64K? We're so many orders of magnitude beyond that now, that it's hard to remember.. - Alex Holcombe
Still conflicted on this subject. I like the challenge the web brings - that if you are going to give a lecture you have to make it something that can't be obtained by a recording. It has to be a performance, with audience interaction and spontaneity. This makes a lecture much harder to give, but at least it means there's a point in your standing there. The same criteria apply to the... more... - Matthew Todd
I think rather the best instruction will use both, perhaps with instructors more free to engage more directly with students than in the traditional lecture. I doubt that the lectures themselves have ever been the crucial part of the educational experience, so replacing universities requires more than replacing lectures. Many intriguing ideas on incorporating web and classroom discussed at the ISTE 2010 conference: http://center.uoregon.edu/ISTE... - Carl Boettiger
Michael Nielsen
Warren Buffett: "More than 99% of my wealth will go to philanthropy during my lifetime or at death. Measured by dollars, this commitment is large. In a comparative sense, though, many individuals give more to others every day." - Michael Nielsen
Buffett's letter is rather remarkable, and worth a read. Several of the others are also pretty interesting. - Michael Nielsen
"I've worked in an economy that rewards someone who saves the lives of others on a battlefield with a medal, rewards a great teacher with thank-you notes from parents, but rewards those who can detect the mispricing of securities with sums reaching into the billions. In short, fate's distribution of long straws is wildly capricious."-Buffett - Alex Holcombe
Mickey Schafer
For those of you running labs or supervising others, do you have training routines in place to smooth the transition of new people?
this is a big issue because it can be time-consuming. I try to put much of what they need to know on the lab wiki (http://openwetware.org/wiki...). Want to move away from reliance on oral transmission of knowledge (which should have gone out of style 2000 yrs ago). Also the wiki potentially benefits others around the world. - Alex Holcombe
Important question. There are a few online resources that the people in my lab use, but most of their instruction will be on their topic and hands-on, so it has to be shown. However, for some tasks I have videos, e.g. a JoVE video. - Björn Brembs
The lab I work in recently created a small induction manual for new students joining the lab - it's a bit like Kathy Barkers "At the Bench: A laboratory navigator", but with specifics that apply to our lab. A copy is kept on our internal wiki (not wiki-fied unfortunately - it's a Word doc), and I think it helps reduce the training load by answering a lot of the common queries about protocols and safety etc. - Andrew Perry
As Alex says keeping lab notebooks and other info on a wiki helps tremendously - Jean-Claude Bradley
Thanks so much, everyone! I'm in the position of training new faculty and my supervisor is encouraging me to use the CMS to produce course templates. It also has wiki function that I was planning to use, too. - Mickey Schafer
Alex Holcombe
New version announced of automated lab notebook for computational studies - Sumatra http://neuralensemble.blogspot.com/2010...
it's python code - Alex Holcombe
Michael Nielsen
How many journal articles have been published (ever)? « O'Really? - http://duncan.hull.name/2010...
"50 million" - Alex Holcombe
Next question: where can I download them? Oh right ... I can't ... ;-) - Lars Juhl Jensen
It should be possible, though, to get an estimate how many of them are available via Green or Gold OA, or even under an Open license. My guess would be on the order of 5 million, 2 million, 1 million. - Daniel Mietchen
Michael Nielsen
Alice Bell: Storm the Royal Society? - http://doctoralicebell.blogspot.com/2010...
"Shudson argues our notion of an informed citizen is anachronistically rooted in the context of the last “information revolution”, that of the early 20th century. Then, idea that a voter should learn as much as possible is base on a time where access to information was opening up (mass-media, literacy rates, emancipation), but was nowhere near as open. Now we simply have more data than we can deal with: we are promised with everything, but in reality we can only manage a bit. Should citizens “follow everything about everything?”. Are those who don’t delinquent? “Or, in contrast, could they be judged exemplary if they know a lot about one thing and serve as sentries patrolling a segment (but not all) of the public interest’s perimeter?”" - Michael Nielsen
All good questions. And the blog post has several more. I agree that building public trust in science, and more generally in how we know what we know, is something to be tackled locally, in small groups, and sometimes, in quiet conversations among friends. - Bill Anderson
If only we could get everybody on board with random sampling of the citizenry, we could run democracies with focus groups or the like rather than up/down elections for candidates - Alex Holcombe
'The Evolving Self' by Mihaly Csikszentmihalyi contains quite similar ideas, although in the context of evolution of the whole society. - Pawel Szczesny
Matthew Todd
Elsevier impact factors for chem journals. PLoS One would be fourth, beaten only by review journals. I find this amazing.
Screen shot 2010-06-25 at 2.36.14 PM.png
I like how this will help upend people's conceptions of IFs. Must really confuse authors who worry about IFs! - Alex Holcombe
Where is Angewandte Chemie and JACS? - Egon Willighagen
Not Elsevier. - Matthew Todd from Android
Ah, sorry! - Egon Willighagen
Matthew Todd
Mendeley import from RIS connotea library brings with it correct URLs containing DOIs, but the DOIs themselves are not recognised. 95% of author names are blank. Ugh. Surely some way of doing an automatic batch look-up of the DOIs to fill in all these blanks?
I gave up on Mendeley when it didn't import my Endnote-formatted exported library correctly. Someone at Mendeley said they'd get back to me about it but they never did. - Alex Holcombe
Matthew, RIS import has multiple issues, some having to do with the RIS format itself. I recommend converting the RIS file into Bibtex and importing that. If you've have supplied tags in connotea, your tags will end up in the wrong field in the bibtex, so rename the field that contains your tags to "mendeley-tags", then import the modified Bibtex. After that, import the PDFs. There is... more... - Mr. Gunn
Alex - Did you import Endnote XML (not .enl)? Try that and if it's still not working properly, let me know. - Mr. Gunn
Other ways to read this feed:Feed readerFacebook