Attention conservation notice: Of interest only if you have been following the mess about William Saletan's series of articles in Slate, saying that liberals are just as bad as creationists for refusing to accept the scientific evidence that black people are just inherently dumber than white people. Of course, it may not be interesting even if you have been following that. (For background, see here, here, here or here; I don't feel like rewarding Saletan's rubbish with a link, but it shouldn't take you but a moment to find it, if you want.)
Saletan has written an epilogue, titled "Regrets", to his series, which is a very curious piece of work indeed. Here's the end of it in its entirety (except for the links):
In researching this subject, I focused on published data and relied on peer review and rebuttals to expose any relevant issue. As a result, I missed something I could have picked up from a simple glance at Wikipedia.For the past five years, J. Philippe Rushton has been president of the Pioneer Fund, an organization dedicated to "the scientific study of heredity and human differences." During this time, the fund has awarded at least \$70,000 to the New Century Foundation. To get a flavor of what New Century stands for, check out its publications on crime ("Everyone knows that blacks are dangerous") and heresy ("Unless whites shake off the teachings of racial orthodoxy they will cease to be a distinct people"). New Century publishes a magazine called American Renaissance, which preaches segregation. Rushton routinely speaks at its conferences. I was negligent in failing to research and report this. I'm sorry. I owe you better than that.
In my first post about this, I said that there were two possible interpretations of Saletan's actions: that he didn't know that the ideas he was spreading were crap, or that he did, but spread them anyway to advance an agenda. Saying that the second interpretation was more charitable wasn't just a joke. Sadly, this partial mea culpa supports the first interpretation, that of incompetence. To put it in "shorter William Saletan" form, what he is saying is: I am shocked — shocked! — to discover that the people who devote their careers to providing supposedly-scientific backing for racist ideas are, in fact, flaming racists. And he does seem to be shocked, though it is hard (as Yglesias says) to see why, logically, he should strain out those gnats he displays for our horrified inspection while swallowing the camel of group inferiority (and telling his readers that camel is really great and the coming thing). This indicates a level of incompetence as a reporter and researcher that is really quite stunning — as Brad DeLong says, this seems like a trained incapacity.
But let me back up a minute to the bit about relying on "peer review and rebuttals to expose any relevant issue". There are two problems here.
One has to do with the fact that, as I said, it is really very easy to find the rebuttals showing that Rushton's papers, in particular, are a tragic waste of precious trees and disk-space. For example, in the very same issue of the very same journal as the paper by Rushton and Jensen which was one of Saletan's main sources, Richard Nisbett, one of the more important psychologists of our time, takes his turn banging his head against this particular wall. Or, again, if Saletan had been at all curious about the issue of head sizes, which seems to have impressed him so much, it would have taken about five minutes with Google Scholar to find a demonstration that this is crap. So I really have no idea what Saletan means when he claimed he relied on published rebuttals — did he think they would just crawl into his lap and sit there, meowing to be read? If I had to guess, I'd say that the most likely explanation of Saletan's writings is that he spent a few minutes with a search engine looking for hits on racial differences in intelligence, took the first few blogs and papers he found that way as The Emerging Scientific Consensus, and then stopped. But detailed inquiry into just how he managed to screw up so badly seems unprofitable.
The other problem with his supposed reliance on peer review is that he seems confused about how that institution works. I won't rehash what I've already said about it, but only remark that passing peer review is better understood as saying a paper is not obviously wrong, not obviously redundant and not obviously boring, rather than as saying it's correct, innovative and important. Even this misses a deeper problem, a possible failure mode of the scientific community. A journal's peer review is only as good as the peers it uses as reviewers. If everyone, or almost everyone, who referees for some journal is in the grip of the same mistake, then they will not catch it in papers they review, and the journal will propagate it. In fact, since journals usually recruit new referees from their published authors or people recommended by old referees, mistakes and delusions can become endemic and self-confirming in epistemic communities associated with particular journals. To give a concrete example, the community using Physica A is pretty uniformly (and demonstrably) mistaken about how to tell when something is a power-law distribution, so what that journal publishes about power laws is unreliable, and those who derive their training and information from that journal go on to propagate the errors. It would be easy to find even more extreme examples from the physical and mathematical sciences (especially, I must say, among journals published by Elsevier), but it would take too long to explain why they are wrong.
Put simply, the problem is that any group of quack scholars with a shared delusion can put together a journal, dub each other peer reviewers, and go on their cheerful way by endorsing each others' work for their journal. (One of the ways you can tell that intelligent design creationism is a propaganda front and not a real, if stupid, scholarly movement is that their effort to put together just such a journal was never more than half-assed, and it's moribund for some time now.) This isn't even always a bad thing, since sometimes people who seem like quacks are in fact right, and doing things like starting their own journals gives them a chance to get their act together and assemble a convincing case. But all of this does mean that the peer-review filter is a very weak and accepting one, especially on controversial topics. It does not seem unreasonable of me to ask that those who set themselves up as science reporters grasp this.
(I hope no one will mis-interpret me as saying that peer review is worthless — I think some form of it is essential, it's just not enough — or that I'm endorsing some silly social-constructionist view that science is just the views of the winners of the scientific community's internal political squabbles. If I thought that, I'd not be pursuing a scientific career, but rather making much more money, and reading many fewer boring papers and writing many fewer boring grant applications, on a sub-tropical island. Science is systematic and cumulative inquiry into what the world is like and how it works, and by and large one that succeeds in producing increasingly reliable and refined knowledge about the world. This is marvelous and inspiring, but it's still a social process implemented by East African Plains Apes [and some of their tools], and it's wise to be realistic about the implications of this fact.)
Let me close with a quotation (via Jessa Crispin) from William Langewiesche, which conveys, better than I could, something of what I was trying to say about the responsibilities of journalists:
"You have this precious, incredibly privileged thing," he said, "which is the reader's attention for a little while. And you can make the slightest misstep and the reader will put you down. People will say that the reader lives in a busy world. But that's not the reason why. The reason is that the writer blows it, and loses the reader's trust."
Saletan has blown it very badly indeed.
Update, 2 December: I should have said in the first place that my discussion of peer review and hysteresis is heavily indebted to the very illuminating chapter on "Market Failures in the Economy of Science" in the dissertation of Dr. Nienke Oomes. I hope that my mentioning this, even belatedly, might prod Nienke into publishing it, or at least putting it online.
Update, 4 December: Stephen Metcalf, Slate's critic-at-large, goes some way towards redeeming its honor.
Manual trackback: The Geomblog; 3 Quarks Daily; Crooked Timber; Quantum of Wantum; Egregious Moderation; Noli Irritare Leones; The Mahatma X Files; Brendan Nyhan; Rash Matters; The Useless Tree; Nanopolitan; IQ Review; Existence is Wonderful; OpenLeft; Earthli; Jewcy; Science After Sunclipse
Posted at November 30, 2007 13:35 | permanent link