"Amid all the problems that have been highlighted with impact factors based on citations, would it work just to have a number of humans rate a paper?" - Maxine from Bookmarklet

Rating, in the sense of giving a numerical score to, never seems like it gets much use at the sites where I've seen it employed. Assigning attributes makes more sense to me. No matter how you slice it, though, the relative importance of a work is never going to become known until years after. A more short-term rating system would only show popularity, not true usefulness/influence. - Mr. Gunn

Isn't this what we've just abandoned in the UK RAE as far too expensive? I agree with Mr Gunn though - getting any rating is pretty meaningless - looking at the underlying data for what is important to you at that moment in time is more useful - a "metric for the moment" approach? - Cameron Neylon

See my comment under the post at the link! - Maxine

ratings would be worse! think about the click fraud going on for search engine optimization -- at least with citations, there's some small barrier: you have to actually write a paper. - Christina Pikas

Yeah, the gaming of ratings (particularly by malicious interests) was one of the things I raised in my ICSTI 2007 presentation on peer review. http://scilib.typepad.com/science... - Richard Akerman

It is completely impractical. He suggests no weighting (for credentials), paying people to mark (every paper ever published?) etc. Nothing practical at all. - Maxine