torsdagen den 26 april 2001 11:45 Eliezer S. Yudkowsky wrote:
> Amara Graps wrote:
> > I learned last Fall from an article in Physics Today (1) that several
> > subfields of physics almost completely bypass the paper journals and rely
> > on the Los Alamos Preprint Server (http://xxx.lanl.gov).
>
> I'd love nothing better than to see The Literature go online, and I think
> the existing peer-review mechanism has serious problems, but this still
> makes me uneasy. It only works as long as the vast majority of
> contributors are already highly competent people - like Usenet in the days
> when only scientists and programmers used the Internet. In essence,
> they're taking the peer-reviewed authors and putting them straight to
> LANL, and indeed this will work fine, until the day when thousands upon
> untold thousands of non-peer-reviewed authors crash the system and the
> whole thing goes the way of Usenet.
Yes. There are already some crackpot papers in lanl, and more are sure to
follow. I have noted that more crackpot papers appear as PDF and HTML than
LaTeX, so the format gives a slight hint, but there are of course many papers
that slip through that primitive filter. In addition, there are of course
plenty of bad papers around.
> Not only does this system need a
> reputation manager, but I also want to see the reputation-manager problem
> *thoroughly* solved, tested, and debugged for, say, the Extropian mailing
> list, before we try replacing the peer review system. I am fully in favor
> of the Open Archive initiative, but not if this is the consequence.
What are the basic problems with reputation management? Some problems right
off the cuff:
* Need to get a critical mass of people involved with the system - competing
standards weaken the systems quite a bit, and the interface has to be so
simple even a Ph.D. can use it.
* The problem of mutually supporting cliques
* Linking reputation to factual results and not just fashions or politics
* Making reputations matter
I have thought about some kind of "reputation futures" or other
investment-like reputation system. You can invest reputation by endorsing a
paper, and if the paper becomes well cited you gain reputation while if it is
shot down you lose reputation.
This archive was generated by hypermail 2b30 : Mon May 28 2001 - 09:59:56 MDT