Three-Toed Sloth

On Shoggothim

Attention conservation notice: Self-promotion of a pay-walled piece which combines a trendy topic with what even I admit is a long-held semi-crank notion.

Henry Farrell and I have an essay in The Economist, riffing off the meme that every large language model is really a shoggoth. Our point is that this is right, because an LLM is a way of taking the vast incohate chaos of written-human-language-as-recorded-on-the-Web and simplifying and abstracting it in potentially useful ways. They are, as Alison Gopnik says, cultural technologies, more analogous to library catalogs than to individual minds. This makes LLMs recent and still-minor members of a larger and older family of monsters which similarly simplify, abstract, and repurpose human minds: the market system, the corporation, the state, even the democratic state. Those are distributed information-processing systems which don't just ingest the products of human intelligence, but actually run on human beings --- a theme I have been sounding for while now.

The piece is paywalled, but Henry has a Twitter thread that provides a good summary, and Brad DeLong has excerpts, along with thoughtful commentary. (I agree with Henry's response to said comments.)

Some things we didn't include:

• Thanks to the editorial staff at The Economist, both for the opportunity and for their very professional work.
• Thanks to Ted Chiang (!) for helpful comments on a draft.
• Any discussion of LLMs as artifacts, in the sense of Herbert Simon's Sciences of the Artificial. (I for one learned this way of thinking of markets and hierarchies as information-processing systems from Simon...)
• Any discussion of Dan Sperber's account of culture as "the precipitate of cognition and communication in a human population", the role in that process of chains of alternately private-mental and public-physical representations, and LLMs as public-representation-producing artifacts
• Any discussion of Arthur Stinchcombe's work on the positive role of abstraction and formalities in institutions
• "More is different": These things emerge from the massed results of human social interaction and individual intelligence, and therefore are very different from human minds. In particular, they tend to have their own intrinsic dynamics, which are usually not things anyone intends, and often things no-one wants. (Someday I will write that essay about blackouts and alienation.) That doesn't mean they can't be controlled; it means control is hard, and usually itself impersonal.
• An adequate discussion of monster-taming and its limits, which would necessarily include extended praise of social democracy (though see DeLong's post)
• Any mention of the the primal scene of AI.
• Henry's reflections on modern neo-Lovecraftian fiction, which I hope he will publish elsewhere.

Update, 23 June: Small wording tweaks and additions. More important: insightful and generous commentary from Daniel "\( D^2 \)" Davies. (It's virtually a blogosphere reunion.) Incorporated (sorry) by reference: Beniger, The Control Revolution; Yates, Control through Communication; Ashby, "Design for an Intelligence Amplifier".

(I know I learned that the correct plural of "shoggoth" is "shoggothim" from reading Ruthanna Emrys, but I cannot now locate the passage --- it may just be in her Lovecraft Reread series with Anne Pillsworth.)

Self-Centered; The Collective Use and Evolution of Concepts; Cthulhiana; The Great Transformation