Averaging
A statistical textbook [1] contains this gem of wisdom:
Only a fool would conclude that a painting that was judged as excellent by one person and contemptible by another ought therefore to be classified as mediocre.
Common sense indeed; but does the procedure not recall how the typical conference program committee works, with averages obligingly computed by the supporting web-based systems?
Reference
[1] David C. Howell: Statistical Methods for Psychology, sixth edition, Thomson-Wadsworth, 2007
Eiffel Software
Home page at ETH
Informatics Europe
LASER summer school
Propulsion Academy
Actually, in my experience, competent PCs do not accept papers based on averages but on who stands up for them. High and low scores don’t cancel each other but rather generate discussion. See: http://scg.unibe.ch/download/champion/index.html#PATTERN5
Indeed, that’s what the good PCs do (certainly the ones in which I participate). I wish they were the norm.
Thanks for the reference.