Bayes's Theorem and Diagnostic Testing

The Bactra Review Error and the Growth of Experimental Knowledge

Suppose we're screening people for a disease --- say tuberculosis to be definite, and to abbreviate it as T --- with a test which gives either a positive or a negative result (A and B, respectively). Suppose further that the test is quite accurate, in the sense that, say, it will give a positive result 95% of the time when tuberculosis is present, i.e. p(A|T) = 0.95. What is the probability that a person who tests positive has tuberculosis? The naive answer, given by a truly shocking proportion of medical students and even doctors, is 95%; but this is wrong. What we want to know is p(T|A), and Bayes's theorem tells us it is p(A|T)p(T)/p(A). So, in addition to knowing p(A|T), which is 0.95, we need to know p(T) and p(A), the probability of having tuberculosis, and p(A), the probability of testing positive. The last is clearly the probability of testing positive if one has the disease, plus the probability of testing positive if one does not, i.e. p(A|T)p(T) + p(A|NT)p(NT), where NT stands for ``not tubercular.'' (As system administrators know, NT in no way implies health.) Suppose the test is unlikely to give a positive result if the disease is absent, p(A|NT) = 0.05, and the disease is, fortunately, quite rare, p(T) = 0.001, or one tenth of one percent. (This means p(NT) = 0.999, of course.) Then p(A) = (0.95)(0.001) + (0.05)(0.999) = 0.0509, and p(T|A) = (0.95)(0.001)/0.0509 = 0.019, which is to say the probability that testing positive means actually being tubercular is less than one in fifty.

I have been deliberately ambiguous in my language here, as to whether or not this probability applies to an individual patient, or to the class of patients who test positive. Bayesians routinely phrase such examples in a way which prejudges the issue in favor of the probability applying to the person currently at the clinic in front of us, and insinuate that the only way such a probability makes sense is if it represents a degree-of-rational-belief on our part. But the reasoning works impeccably if we're just concerned with frequencies in a statistical ensemble: here, the population of patients.