Ethical and Political Issues in Data Mining, Especially Unfairness in Automated Decision Making

30 Apr 2021 14:58

I won't be explaining data mining here. But I will say that I think "ethical and political issues in data mining" is a lot more accurate and reasonable name for what people are really worried about than "algorithmic fairness". This opinion is partly because I don't think algorithms are really at the core of a lot of the justified and widely-shared concerns. The formal notions of "algorithmic fairness" could also be applied to human decision makers. (It would be very interesting to see whether, say, unaided human loan officers are closer to false positive parity than credit-scoring algorithms; maybe someone's done this experiment.) Indeed, if those formal notions are good ones, we probably ought to be applying them to human decision-makers. That doesn't mean those who design automated decision-making systems shouldn't pay attention, but it does tell me that the real issue here isn't the use of algorithms.

Or, again: it's (probably!) a fact that in contemporary English, the word "doctor" is more likely to refer to a man than to a woman, and vice versa for "nurse". If a text-mining model picks up this actual correlation and uses it (for instance in an analogy-completion tassk), it is accurately reflecting facts about how English is used in our society. It seems obvious to me that those facts are explained by untold generations of sexism. Whether and when we want language models to use such facts would seem to depend on their uses, as well as on ethical and political choices about what kind of world we'd like to see. (There are, after all, plenty of people who approve of a world where doctors are more likely to be men and nurses women.) It would also seem to require sociological knowledge, or at least theories, about how modifying the output of text-mining systems might, or might not, contribute to changing society. If the combination of political values choices with reliance on necessarily-speculative theories about the remote, cumulative impacts of technical choices on social structure seems like a recipe for disputes, well, I think you wouldn't be wrong, and I wouldn't blame you for wanting to ignore the issue and get back to making the damn things work. But the issue will not ignore you.

(I also dislike talk of "regulating artificial intelligence", not least because artificial intelligence, in the sense people like to think of it, "is the technology of the future, and always will be".)

See also: Clinical and Actuarial Compared; Law; Recommendation Engines