Friday, April 22, 2011

Strengths and Weaknesses

When you review a paper for a conference or journal, many of the reviewing forms request that the reviewer outline the paper's strengths and weaknesses.

Recently, I read a review that went something like this:
"Weaknesses:  Several of the main findings presented in this paper are merely confirmation of previous work."
I wish this was one of those publication venues where you can review the reviewers, because I would have written back:
"Weaknesses: Reviewer 2 has gotten so thristy for novelty they have forgotten one of the hallmarks of science: replicability."
It's not just Computer Science that is plagued by this problem, certainly it's cropped up elsewhere. But our discipline does have a tradition of getting a little too obsessed with the novelty of an idea that they forget the value of reproducing previous findings.