A critique of critique

I won't deny it: I want my stuff to be widely read. As an academic, 100% of my output is text (including diagrams & equations), and having a readership alleviates many existential dilemmas involved in spending my life writing down black marks on white paper. There are a lot of measures of having an expanding readership, and I hit another milestone last night: I got my first crackpot critic.

One of Lawrence Lessig's book on intellectual property cites G for providing a few numbers, but the numbers are from private correspondence, so I didn't have the methods behind them. I wrote to G asking him for the info, and he wrote back with numbers on an entirely different topic (and again, no methods) and a note in the way of `by the way, Lessig's positions are totally idiotic; write back and I'll tell you why'.

I was a bit uneasy, since I could find no academic or legal credentials on the guy and still had no methods, but I used one of his numbers in my book anyway. Yesterday, I wrote G to thank him and mentioned that my manuscript was finished and he should have a look at it. He did, and called the part he read "nonsense". He continued, "I have a paper on my Web site [gives URL] - read it and rewrite your book."

I skimmed it. It included 90 court cases and opinions, and his commentary of each of them. He praised those that agreed with his beliefs and directly slammed those that disagreed, calling them "illegitimate", "bad", or "bogus", and offered minimal or no explanation for why they were illegitimate (which is to be expected given that he's covering so much ground, I guess). But the list comforted me, because it confirmed that this was just his style toward all things.

How to critique As a grad student, I got to see the full range from good to bad writing critique. Most of the problems stem from the fact that nobody has the time or inclination to give a careful, complete reading of everything that passes their desk. This is just a fact of life, and there's no real point in me complaining about it, but there is an immense range in how people deal with the problem of critiquing a half-read paper.

The ones who were good at critique were honest about how much time they'd spent on my stuff. One had a strict policy of reading closely until he got to the first item he didn't understand, and then he'd stop immediately. We had a hundred short meetings instead of one or two long ones and it was all very frustrating, but my papers got better, and in retrospect I'm glad I suffered through it.

The good ones also acknowledged that they could be wrong. Instead of `You don't address this one counterargument', they would say `I don't see how you address this one counterargument'. The difference had a big emotional impact on me, because 80% of the time I did address it, but after the person stopped reading. The debates with people who used the first wording were tricky because I had to pretend that they had read all of my arguments and then come up with a polite way to express them without just pointing to page seventeen. All so socially awkward.

When I started submitting to journals, the problem was exacerbated. As I've noted before, the peer review system has fundamental flaws, and famous people who have lots of stuff published also believe this. Simply put, the reviewer has no incentive to carefully read a paper beyond squishy sort of goodwill. Being an anonymous review, they have no reputation on the line if they totally screw up. Math requires an especial amount of care and focus, and I am told that the refereeing quality in math journals is correspondingly especially crappy. Since my papers are primarily mathematical I faced this problem directly, and wound up with two types of reviewer: those who will reject based on the first, most trivial excuse (so they don't have to read it carefully), and those who skim it and acquiesce that it seems OK to them (so they don't have to read it carefully).

Another sign of the good and bad critique was what people did when they just plain disagreed. The best ones could work out exactly what their arguments against were, and strained to be as detailed as possible. Second best but still honest were the ones who said `I'm just not convinced,' and then left me to puzzle out why they weren't happy. It was often a worldview issue which was sort of fundamentally unresolvable---which gives us another branch in critique quality: those who can follow work based on a premise which they hate but which they know a large number of people agree with, and those who can't. Finally, you get the annoying ones would just give half-assed reasons to reject it so they wouldn't have to think about the fundamental assumptions at all.

And then there's the boilerplate stuff of good critique: don't be ad hominem, don't be rude even if you don't know the person personally, be as specific as possible, make some effort to suggest how the person could fix your complaints. But to boil it all down, I feel that the good critiquer is a person who believes that they are in a cooperative venture with the author in a search for some sort of truth, and asks the same question the author would ask: how can this become a better paper?

Naturally, my crackpot critiquer fails almost every single test above (he doesn't know me, so no ad hominem attacks). G is entirely convinced of his infallibility, and is thus perfectly comfortable calling Supreme Court rulings which were precedent for a decade "illogical" and "illegitimate" without feeling much need to explain why. He prefers to express his disdain for opposing viewpoints ("Finally, I will hit anyone in the head who says [the opinion I disagree with] - with a thick bat.") instead of explaining why he thinks those viewpoints are wrong. And, of course, he tells me I didn't address arguments which I spent the majority of the book addressing.

In short, G is the perfect belligerent critiquer, with zero interest in seeking out truth via good debate and lots of interest in pushing forward his uncritiqueable personal beliefs. I'm sort of happy to have merited his disdain, and can't wait for his review to come online (he promised to write me a negative review in his newsletter if I didn't revise the book to agree with him). I just hope that after that he'll go away.