Re: Open Peer review on Internet

Eric Watt Forste (arkuat@factory.net)
Tue, 24 Sep 1996 18:21:54 -0700


At 5:16 PM 9/24/96, Robin Hanson wrote:
>The problem with this proposal is, what do you put on your vita so
>readers of it can quickly assess the quality of the publication?
>With an ordinary journal the journal itself signals quality.
>Sure the vita reader could go read all the reviews, but they usually
>won't have time to.

I don't have any brilliant ideas about this in the short-term, but it's
starting to look pretty obvious to me that sooner or later we are going to
have a recognized profession of credential-creators. People whose ability
to judge work within a given field is fully recognized by practitioners of
that field and is generally recognized by people outside that field who
need to consume information produced by workers in that field.

This role has traditionally been played by institutions (schools, colleges,
journals, etc.) But as specialization accelerates, fields are coming into
existence for which there are only a few people capable of judging work
done in that field (over and above just *doing* work in that field). People
who work in these specialties have to figure out who these "de facto"
credential grantors are as their work proceeds... unless they feel no need
whatsoever for external validation of the quality of their own intellectual
work. There are lots of fields of work proceeding as we speak for which no
journal exists. They are managing to set their own standards somehow.
Eventually the practices they are evolving will develop into full-fledged
institutions.

Maybe someday there will be a few loosely recognized "professional
organizations" for the profession of credential-granting. The best analogy
I can think of for this future development that I'm thinking about is the
voluntary accrediting institutions that accredit schools in the United
States. It would be the same kind of thing, only they'd be accrediting
individual specialists, who would in turn endorse the work of people
working in that particular field. It's these endorsements by individual
credential-creators (probably with their digital signature) that you'd put
on your vita. Endorsement would probably be for specific pieces of work
done over specific periods of time, not carte-blanche endorsements of the
researcher as a person.

The potentials for abuse are stupendous. Here is a field ripe for those who
are interested in designing alternative institutions. As far as I can tell,
it'll be a matter of getting the design of the protocols right and leaving
room for the future improvement of those protocols. What do you think,
Robin? You've got a lot more expertise in this matter than I do.

(It's pretty obvious that my thinking on this is heavily influenced by the
"web of trust" ideas in the PGP documentation and related texts. But I'd
expect that of most of the other people on this list, even now.)

Eric Watt Forste <arkuat@pobox.com> http://www.c2.org/~arkuat/