Nastiness in computer science
(This article was originally published in the CACM blog.)
Are we malevolent grumps? Nothing personal, but as a community computer scientists sometimes seem to succumb to negativism.
They admit it themselves. A common complaint in the profession (at least in academia) is that instead of taking a cue from our colleagues in more cogently organized fields such as physics, who band together for funds, promotion, and recognition, we are incurably fractious. In committees, for example, we damage everyone’s chances by badmouthing colleagues with approaches other than ours. At least this is a widely perceived view (“Circling the wagons and shooting inward,” as Greg Andrews put it in a recent discussion). Is it accurate?
One statistic that I have heard cited is that in 1-to-5 evaluations of projects submitted to the U.S. National Science Foundation the average grade of computer science projects is one full point lower than the average for other disciplines. This is secondhand information, however, and I would be interested to know if readers with direct knowledge of the situation can confirm or disprove it.
More such examples can be found in the material from a recent keynote by Jeffrey Naughton, full of fascinating insights (see his Powerpoint slides 
). Naughton, a database expert, mentions that only one paper out of 350 submissions to SIGMOD 2010 received a unanimous “accept” from its referees, and only four had an average accept recommendation. As he writes, “either we all suck or something is broken!”
Much of the other evidence I have seen and heard is anecdotal, but persistent enough to make one wonder if there is something special with us. I am reminded of a committee for a generously funded CS award some time ago, where we came close to not giving the prize at all because we only had “good” proposals, and none that a committee member was willing to die for. The committee did come to its senses, and afterwards several members wondered aloud what was the reason for this perfectionism that almost made us waste a great opportunity to reward successful initiatives and promote the discipline.
We come across such cases so often—the research project review that gratuitously but lethally states that you have “less than a 10% chance” of reaching your goals, the killer argument “I didn’t hear anything that surprised me” after a candidate’s talk—that we consider such nastiness normal without asking any more whether it is ethical or helpful. (The “surprise” comment is particularly vicious. Its real purpose is to make its author look smart and knowledgeable about the ways of the world, since he is so hard to surprise; and few people are ready to contradict it: Who wants to admit that he is naïve enough to have been surprised?)
A particular source of evidence is refereeing, as in the SIGMOD example. I keep wondering at the sheer nastiness of referees in CS venues.
We should note that the large number of rejected submissions is not by itself the problem. Naughton complains that researchers spend their entire careers being graded, as if passing exams again and again. Well, I too like acceptance better than rejection, but we have to consider the reality: with acceptance rates in the 8%-20% range at good conferences, much refereeing is bound to be negative. Nor can we angelically hope for higher acceptance rates overall; research is a competitive business, and we are evaluated at every step of our careers, whether we like it or not. One could argue that most papers submitted to ICSE and ESEC are pretty reasonable contributions to software engineering, and hence that these conferences should accept four out of five submissions; but the only practical consequence would be that some other venue would soon replace ICSE and ESEC as the publication place that matters in software engineering. In reality, rejection remains a frequent occurrence even for established authors.
Rejecting a paper, however, is not the same thing as insulting the author under the convenient cover of anonymity.
The particular combination of incompetence and arrogance that characterizes much of what Naughton calls “bad refereeing” always stings when you are on the receiving end, although after a while it can be retrospectively funny; one day I will publish some of my own inventory, collected over the years. As a preview, here are two comments on the first paper I wrote on Eiffel, rejected in 1987 by the IEEE Transactions on Software Engineering (it was later published, thanks to a more enlightened editor, Robert Glass, in the Journal of Systems and Software, 8, 1988, pp. 199-246 ). The IEEE rejection was on the basis of such review gems as:
- I think time will show that inheritance (section 1.5.3) is a terrible idea.
- Systems that do automatic garbage collection and prevent the designer from doing his own memory management are not good systems for industrial-strength software engineering.
One of the reviewers also wrote: “But of course, the bulk of the paper is contained in Part 2, where we are given code fragments showing how well things can be done in Eiffel. I only read 2.1 arrays. After that I could not bring myself to waste the time to read the others.” This is sheer boorishness passing itself off as refereeing. I wonder if editors in other, more established disciplines tolerate such attitudes. I also have the impression that in non-CS journals the editor has more personal leverage. How can the editor of IEEE-TSE have based his decision on such a biased an unprofessional review? Quis custodiet ipsoes custodes?
“More established disciplines”: Indeed, the usual excuse is that we are still a young field, suffering from adolescent aggressiveness. If so, it may be, as Lance Fortnow has argued in a more general context, “time for computer science to grow up.” After some 60 or 70 years we are not so young any more.
What is your experience? Is the grass greener elsewhere? Are we just like everyone else, or do we truly have a nastiness problem in computer science?
Eiffel Software
Home page at ETH
Informatics Europe
LASER summer school
Propulsion Academy
I think part of it is the subject matter vs. the amount of work one generally has to do, or in other words, the people who get to review these things are doing so because they have no other work to do, i.e. they probably don’t understand 90% of what they’re looking at, so unless it’s written in a particularly engaging way, they’ll skip most of it and base their opinion solely upon how interested they were in reading it (which, for the more technically-oriented submissions, will be not at all). Computer scientists, and programmers in particular, tend to be nasty by default because most of the time they occupy the sort of mental level not accessible to the large majority of the population, and thus they tend to look down on pretty much everyone else. Now imagine you’re someone like that who, in reality, isn’t that bright and doesn’t understand most of the ideas percolating around the mental strata you’re supposed to be occupying: I think at that point it becomes reasonably easy to understand their attitude.
[…] Bertrand Meyer has another possible explanation – it is because computer scientists are too rude about one another. This extract says it […]
One of the problems refereeing papers in CS is the reproduction of results. This is illustrated by the quotes given which are no more than opinions. Many in CS work in the discipline of information: its nature; its application; structuring it; processing it. Endless arguments are heard on which language is ‘best’, which operating system is most secure – I
often wonder how many ever use these facilities to solve problems.
In the Natural Sciences people experiment and compare with theory and reproduction of results rather than opinion is essential. The boorishness of comment is thus reduced although jealousy and ambition is detectable.
I am aware that for some in CS, application to solving problems in the natural world is not seen as quite so respectable. However for myself working for many years in pattern recognition it has been a real joy, although I have not as a result published in CS and not been subject to the negativity articulated in this article!
Look at it this way, imagine these were entries in the field of Physics? How would such boorish answers be accepted by the community? No doubt there we be quite harsh words. The academia in Physics are the rock-stars of their field. Those who are experts in the field consider their academic background absolutely essential. However, too many computer programmers have no computer-science background at all, and if they do, it’s most irrelevant because of the type of work that they do creating line-of-business applications. Methods and systems are continuously reinvented because they don’t have to the background to know what has been done in the past, and future generations grow up using these systems, half of which ignore and start their own fork of work.