Three-Toed Sloth

The Books I Am Not Going to Write

Attention conservation notice: Middle-aged dad contemplating "aut liberi, aut libri" on April 1st.

Re-Design for a Brain

W. Ross Ashby's Design for a Brain: The Origins of Adaptive Behavior is a deservedly-classic and influential book. It also contains a lot of sloppy mathematics, in some cases in important places. (For instance, there are several crucial points where he implicitly assumes that deterministic dynamical systems cannot be reversible or volume-preserving.) This project would simply be re-writing the book so as to give correct proofs, with assumptions clearly spelled out, and seeing how strong those assumptions need to be, and so how much more limited the final conclusions end up being.

Why I am not going to write it: It would be of interest to about five other people.

The Genealogy of Complexity

Why I am not going to write it: It no longer seems as important to me as it did in 2003.

The Formation of the Statistical Machine Learning Paradigm, 1985--2000

Why I am not going to write it: It'd involve a lot of work I don't know how to do --- content analysis of CS conference proceedings and interviews with the crucial figures while they're still around. I feel like I could fake my way through get up to speed on content analysis, but oral history?!?

Almost None of the Theory of Stochastic Processes

Why I am not going to write it: I haven't taught the class since 2007.

Statistical Analysis of Complex Systems

Why I am not going to write it: I haven't taught the class since 2008.

A Child's Garden of Statistical Learning Theory

Why I am not going to write it: Reading Ben Recht has made me doubt whether the stuff I understand and teach is actually worth anything at all.

The Statistics of Inequality and Discrimination

I'll just quote the course description:

Many social questions about inequality, injustice and unfairness are, in part, questions about evidence, data, and statistics. This class lays out the statistical methods which let us answer questions like "Does this employer discriminate against members of that group?", "Is this standardized test biased against that group?", "Is this decision-making algorithm biased, and what does that even mean?" and "Did this policy which was supposed to reduce this inequality actually help?" We will also look at inequality within groups, and at different ideas about how to explain inequalities between and within groups.

The idea is to write a book which could be used for a course on inequality, especially in the American context where we're obsessed by between-group inequalities, for quantitatively-oriented students and teachers, without either pandering, or pretending that being STEM-os lets us clear everything up easily. (I have heard too many engineers and computer scientists badly re-inventing basic sociology and economics in this context...)

Why I am not going to write it: Nobody wants to hear that these are real social issues; that understanding these issues requires numeracy and not just moralizing; that social scientists have painfully acquired important knowledge about these issues (though not enough); that social phenomena are emergent so they do not just reflect the motives of the people involved (in particular: bad things happen just because the people you already loathe are so evil; bad things don't stop happening just because nobody wants them); or that no amount of knowledge about how society is and could be will tell us how it should be. So writing the book I want will basically get me grief from every direction, if anyone pays any attention at all.

Huns and Bolsheviks

To quote an old notebook: "the Leninists were like the Chinggisids and the Timurids, and similar Eurasian powers: explosive rise to dominance over a wide area of conquest, remarkable horrors, widespread emulation of them abroad, elaborate patronage of sciences and arts, profound cultural transformations and importations, collapse and fragmentation leaving many successor states struggling to sustain the same style. But Stalin wasn't Timur; he was worse. (Likewise, Gorbachev was better than Ulugh Beg.)"

Why I am not going to write it: To do it even half-right, relying entirely on secondary sources, I'd have to learn at least four languages. Done well or ill, I'd worry about someone taking it seriously.

The Heuristic Essentials of Asymptotic Statistics

What my students get sick of hearing me refer to as "the usual asymptotics". A first-and-last course in statistical theory, for people who need some understanding of it, but are not going to pursue it professionally, done with the same level of mathematical rigor (or, rather, floppiness) as a good physics textbook. --- Ideally of course it would also be useful for those who are going to pursue theoretical statistics professionally, perhaps through a set of appendices, or after-notes to each chapter, highlighting the lies-told-to-children in the main text. (How to give those parts the acronym "HFN", I don't know.)

Why I am not going to write it: We don't teach a course like that, and it'd need to be tried out on real students.

Actually, "Dr. Internet" Is the Name of the Monsters' Creator

Why I am not going to write it: Henry will finally have had enough of my nonsense as a supposed collaborator and write it on his own.

Logic Is a Pretty Flower That Smells Bad

Seven-ish pairs of chapters. The first chapter in each pair highlights a compelling idea that is supported by a logically sound deduction from plausible-sounding premises. The second half of the pair then lays out the empirical evidence that the logic doesn't describe the actual world at all. Thus the book would pair Malthus on population with the demographic revolution and Boserup, Hardin's tragedy of the commons with Ostrom, the Schelling model with the facts of American segregation, etc. (That last is somewhat unfair to Schelling, who clearly said his model wasn't an explanation of how we got into this mess, but not at all unfair to many subsequent economists. Also, I think it'd be an important part of the exercise that at least one of the "logics" be one I find compelling.) A final chapter would reflect on the role of good arguments in keeping bad ideas alive, the importance of scope conditions, Boudon's "hyperbolic" account of ideology, etc.

Why I am not going to write it: I probably should write it.

Beyond the Orbit of Saturn

Historical cosmic horror mind candy: in 1018, Sultan Mahmud of Ghazni receives reports that the wall which (as the Sultan understands things) Alexander built high in the Hindu Kush to contain Gog and Magog is decaying. Naturally, he summons his patronized and captive scholars to figure out what to do about this. Naturally, the rivalry between al-Biruni and ibn Sina flares up. But there is something up there, trying to get out, something not even the best human minds of the age can really comprehend...

Why I am not going to write it: I am very shameless about writing badly, but I find my attempts at fiction more painful than embarrassing.

Self-Centered; Modest Proposals

Books to Read While the Algae Grow in Your Fur, March 2025

Attention conservation notice: I have no taste, and no qualification to opine on pure mathematics, sociology, or adaptations of Old English epic poetry. Also, most of my reading this month was done at odd hours and/or while chasing after a toddler, so I'm less reliable and more cranky than usual.

Philippe Flajolet and Robert Sedgewick, Analytic Combinatorics [doi:10.1017/CBO9780511801655]

I should begin by admitting that I have never learned much combinatorics, and never really liked what I did learn. To say my knowledge topped out at Stirling's approximation to $n!$ is only a mild exaggeration. Nonetheless, after reading this book, I think I begin to get it. I'll risk making a fool of myself by explaining.

We start with some class $\mathcal{A}$ of discrete, combinatorial objects, like a type of tree or graph obeying some constraints and rules of construction. There's a notion of "size" for these objects (say, the number of nodes in the graph, or the number of leaves in the tree). We are interested in counting the number of objects of size $n$. This gives us a sequence $A_0, A_1, A_2, \ldots A_n, \ldots$.

Now whenever we have a sequence of numbers $A_n$, we can encode it in a "generating function" \[ A(z) = \sum_{n=0}^{\infty}{A_n z^n} \] and recover the sequence by taking derivatives at the origin: \[ A_n = \frac{1}{n!} \left. \frac{d^n A}{dz^n} \right|_{z=0} \] (I'll claim the non-mathematician's privilege of not worrying about whether the series converges, the derivatives exist, etc.) When I first encountered this idea as a student, it seemed rather pointless to me --- we define the generating function in terms of the sequence, so why do we need to differentiate the GF to get the sequence?!? The trick, of course, is to find indirect ways of getting the generating function.

Part A of the book is about what the authors call the "symbolic method" for building up the generating functions of combinatorial classes, by expressing them in terms of certain basic operations on simpler classes. The core operations, for structures with unlabeled parts, are disjoint union, Cartesian product, taking sequences, taking cycles, taking multisets, and taking power sets. Each of these corresponds to a definite transformation of the generating function: if $\mathcal{A}$s are ordered pairs of $\mathcal{B}$s and $\mathcal{C}$s (so the operation is Cartesian product), then $A(z) = B(z) C(z)$, while if $\mathcal{A}$s are sequences of $\mathcal{B}$s, then $A(z) = 1/(1-B(z))$, etc. (Chapter I.) For structures with labeled parts, slightly different, but parallel, rules apply. (Chapter II.) These rules can be related very elegantly to constructions with finite automata and regular languages, and to context-free languages. If one is interested not just in the number of objects of some size $n$, but the number of size $n$ with some other "parameter" taking a fixed value (e.g., the number of graphs on $n$ nodes with $k$ nodes of degree 1), multivariate generating functions allow us to count them, too (Chapter III). (Letting $k$ vary for fixed $n$ of course gives a probability distribution.) When the parameters of a complex combinatorial object are "inherited" from the parameters of the simpler objects out of which it is built, the rules for transforming generating functions also apply.

In favorable cases, we get nice expressions (e.g., ratios of polynomials) for generating functions. In less favorable cases, we might end up with functions which are only implicitly determined, say as the solution to some equation. Either way, if we now want to decode the generating function $A(z)$ to get actual numbers $A_1, A_2, \ldots A_n, \ldots$, we have to somehow extract the coefficients of the power series. This is the subject of Part B, and where the "analytical" part of the title comes in. We turn to considering the function $A(z)$ as a function on the complex plane. Specifically, it's a function which is analytic in some part of the plane, with a limited number of singularities. Those singularities turn out to be crucial: "the location of a function's singularities dictates the exponential growth of its coefficients; the nature of a functions singularities determine the subexponential factor" (p. 227, omitting symbols). Accordingly, part II is a crash course in complex analysis for combinatorists, the upshot of which is to relate the coefficients in power series to certain integrals around the origin. One can then begin to approximate those integrals, especially for large $n$. Chapter V carries this out for rational and "meromorphic" functions, ch. VI for some less well-behaved ones, with applications in ch. VII. Chapter VIII covers a somewhat different way of approximating the relevant integrals, namely the saddle-point method, a.k.a. Laplace approximation applied to contour integrals in the complex plane.

Part C, consisting of Chapter IX, goes back to multivariate generating functions. I said that counting the number of objects with size $n$ and parameter $k$ gives us, at each fixed $n$, a probability distribution over $k$. This chapter considers the convergence of these probability distributions as $n \rightarrow \infty$, perhaps after suitable massaging / normalization. (It accordingly includes a crash course in convergence-in-distribution for combinatorists.) A key technique here is to write the multivariate generating function as a small perturbation of a univariate generating function, so that the asymptotics from Part B apply.

There are about 100 pages of appendices, to fill gaps in the reader's mathematical background. As is usual with such things, it helps to have at least forgotten the material.

This is obviously only for mathematically mature readers. I have spent a year making my way through it, as time allowed, with pencil and paper at hand. But I found it worthwhile, even enjoyable, to carve out that time. §

(The thing which led me to this, initially, was trying to come up with an answer to "what on Earth is the cumulative generating function doing?" If we're dealing with labeled structures, then the appropriate generating function is what the authors call the "exponential generating function", $A(z) = \sum_{n=0}^{\infty}{A_n z^n / n!}$. If $A$'s are built as sets of $B$'s, then $A(z) = \exp(B(z))$. Turned around, then, $B(z) = \log{A(z)}$ when $A$'s are composed as sets of $B$'s. So if the moments of a random variable could be treated as counting objects of a certain size, so $A_n = \mathbb{E}\left[ X^n \right]$ is somehow the number of objects of size $n$, and we can interpret these objects as sets, the cumulant generating function would be counting the number of set-constituents of various sizes. I do not regard this as a very satisfying answer, so I am going to have to learn even more math.)

Arthur L. Stinchcombe, Information and Organizations [Open access]

A series of essays on organizations --- mostly for-profit corporations, but also universities --- as information-processing systems. The main thesis is that organizations "[grow] toward sources of news, news about the uncertainties that most affect their outcomes" (pp. 5--6), and then react to that news on an appropriate (generally, quick) time-scale. This is a functionalist idea, but Stinchcombe is careful to try to make it work, making arguments about how an organization's need to perform these functions comes to be felt by actual people in the organization, people who are in positions to do something about it. (Usually, his arguments on this score are persuasive.) This is by far the best thing I've seen in sociology about social structures as information-processing systems; I'm a bit disappointed in myself that I didn't read it a long time ago. §

Zach Weinersmith and Boulet, Bea Wolf

The first part of Beowulf, through the defeat of Grendel, adapted into a comic-book about joyously ill-behaved kids in an American suburb. Rather incredibly, it works.

(Thanks to Jan Johnson for the book.)

Books to Read While the Algae Grow in Your Fur; Mathematics; Automata and Calculating Machines; Enigmas of Chance; Commit a Social Science; The Dismal Science; The Collective Use and Evolution of Concepts; The Commonwealth of Letters

The Distortion Is Inherent in the Signal

Attention conservation notice: An overly-long blog comment, at the unhappy intersection of political theory and hand-wavy social network theory.

Henry Farrell has a recent post on how "We're getting the social media crisis wrong". I think it's pretty much on target --- it'd be surprising if I didn't! --- so I want to encourage my readers to become its readers. (Assuming I still have any readers.) But I also want to improve on it. What follows could have just been a comment on Henry's post, but I'll post it here because I feel like pretending it's 2010.

Let me begin by massively compressing Henry's argument. (Again, you should read him, he's clear and persuasive, but just in case...) The real bad thing about actually-existing social media is not that it circulates falsehoods and lies. Rather it's that it "creates publics with malformed collective understandings". Public opinion doesn't just float around like a glowing cloud (ALL HAIL) rising nimbus-like from the populace. Rather, "we rely on a variety of representative technologies to make the public visible, in more or less imperfect ways". Those technologies shape public opinion. One way in particular they can shape public opinion is by creating and/or maintaining "reflective beliefs", lying somewhere on the spectrum between cant/shibboleths and things-you're-sure-someone-understands-even-if-you-don't. (As an heir of the French Enlightenment, many of Dan Sperber's original examples of such "reflective beliefs" concerned Catholic dogmas like trans-substantiation; I will more neutrally say that I have a reflective belief that botanists can distinguish between alders and poplars, but don't ask me which tree is which.) Now, at this point, Henry references a 2019 article in Logic magazine rejoicing in the title "My Stepdad's Huge Data Set", and specifically the way it distinguishes between those who merely consume Internet porn, and the customers who actually fork over money, who "convert". To quote the article: "Porn companies, when trying to figure out what people want, focus on the customers who convert. It's their tastes that set the tone for professionally produced content and the industry as a whole." To quote Henry: "The result is that particular taboos ... feature heavily in the presentation of Internet porn, not because they are the most popular among consumers, but because they are more likely to convert into paying customers. This, in turn, gives porn consumers, including teenagers, a highly distorted understanding of what other people want and expect from sex, that some of them then act on...."

To continue quoting Henry:

Something like this explains the main consequences of social media for politics. The collective perspectives that emerge from social media --- our understanding of what the public is and wants --- are similarly shaped by algorithms that select on some aspects of the public, while sidelining others. And we tend to orient ourselves towards that understanding, through a mixture of reflective beliefs, conformity with shibboleths, and revised understandings of coalitional politics.

At this point, Henry goes on to contemplate some recent grotesqueries from Elon Musk and Mark Zuckerberg. Stipulating that those are, indeed, grotesque, I do not think they get at the essence of the problem Henry's identified, which I think is rather more structural than a couple of mentally-imploding plutocrats. Let me try to lay this out sequentially.

1. The distribution of output (number of posts) etc. over users is strongly right-skewed. Even if everyone's content is equally engaging, and equally likely to be encountered, this will lead to a small minority having a really disproportionate impact on what people perceive in their feeds.
2. Connectivity is also strongly right-skewed. This is somewhat endogenous to algorithmic choices on the part of social-media system operators, but not entirely.
   (One algorithmic choice is to make "follows" an asymmetric relationship. [Of course, the fact that the "pays attention to" relationship is asymmetric has been a source of jokes and drama since time out of mind, so maybe that's natural.] Another is to make acquiring followers cheap, or even free. If people had to type out the username of everyone they wanted to see a post, every time they posted, very few of us would maintain even a hundred followers, if that.)
3. Volume of output, and connectivity, are at the very least not negatively associated. (I'd be astonished if they're not positively associated but I can't immediately lay hands on relevant figures.) *
4. People who write a lot are weird. As a sub-population, we are, let us say, enriched for those who are obsessed with niche interests. (I very much include myself in this category.) This of course continues Henry's analogy to porn; "Proof is left as an exercise for the reader's killfile", as we used to say on Usenet. **
5. Consequence: even if the owners of the systems didn't put their thumbs on the scales, what people see in their feeds would tend to reflect the pre-occupations of a comparatively small number of weirdos. Henry's points about distorted collective understandings follow.

Conclusion: Social media is a machine for "creat[ing] publics with malformed collective understandings".

The only way I can see to avoid reaching this end-point is if what we prolific weirdos write about tends to be a matter of deep indifference to almost everyone else. I'd contend that in a world of hate-following, outrage-bait and lolcows, that's not very plausible. I have not done justice to Henry's discussion of the coalitional aspects of all this, but suffice it to say that reflective beliefs are often reactive, we're-not-like-them beliefs, and that people are very sensitive to cues as to which socio-political coalition's output they are seeing. (They may not always be accurate in those inferences, but they definitely draw them ***.) Hence I do not think much of this escape route.

--- I have sometimes fantasized about a world where social media are banned, but people are allowed to e-mail snapshots and short letters to their family and friends. (The world would, un-ironically, be better off if more people were showing off pictures of their lunch, as opposed to meme-ing each other into contagious hysterias.) Since, however, the technology of the mailing list with automated sign-on dates back to the 1980s, and the argument above says that it alone would be enough to create distorted publics, I fear this is another case where Actually, "Dr. Internet" Is the Name of the Monsters' Creator.

(Beyond all this, we know that the people who use social media are not representative of the population-at-large. [ObCitationOfKithAndKin: Malik, Bias and Beyond in Digital Trace Data.] For that matter, at least in the early stages of their spread, online social networks spread through pre-existing social communities, inducing further distortions. [ObCitationOfNeglectedOughtToBeClassicPaper: Schoenebeck, "Potential Networks, Contagious Communities, and Understanding Social Network Structure", arxiv:1304.1845.] As I write, you can see this happening with BlueSky. But I think the argument above would apply even if we signed up everyone to one social media site.)

*: Define "impressions" as the product of "number of posts per unit time" and "number of followers". If those both have power-law tails, with exponents \( \alpha \) and \( \beta \) respectively, and are independent, then impressions will have a power-law tail with exponent \( \alpha \wedge \beta \), i.e., slowest decay rate wins. )To see this, set \( Z = XY \) so \( \log{Z} = \log{X} + \log{Y} \), and the pdf of \( \log{Z} \) is, by independence, the convolution of the pdfs of \( \log{X} \) and \( \log{Y} \). But those both have exponential tails, and the slower-decaying exponential gives the tail decay rate for the convolution.) The argument is very similar if both are log-normal, etc., etc. --- This does not account for amplification by repetition, algorithmic recommendations, etc. ^

**: Someone sufficiently flame-proof could make a genuinely valuable study of this point by scraping the public various fora for written erotica and doing automated content analysis. I'd bet good money that the right tail of prolificness is dominated by authors with very niche interests. [Or, at least, interests which were niche at the time they started writing.] But I could not, in good conscience, advise anyone reliant on grants to actually do this study, since it'd be too cancellable from too many directions at once. ^

***: As a small example I recently overheard in a grocery store, "her hair didn't used to be such a Republican blonde" is a perfectly comprehensible statement. ^

Actually, "Dr. Internet" Is the Name of the Monsters' Creator; Kith and Kin

Tenure-Track Opening in Computational Social Science at CMU (a.k.a. Call to Pittsburgh, 2024 edition)

Attention conservation notice: Advertising an academic position in fields you don't work in, in a place you don't want to live, paying much less than the required skills can get from private industry.

We have a tenure-track opening at the intersection of statistics and complex social systems, a.k.a. computational social science:

The Department of Statistics and Data Science at Carnegie Mellon University invites applications for a tenure track position in Computational Social Science at the rank of Assistant Professor starting in Fall 2025. This position will be affiliated with the Institute for Complex Social Dynamics.

The Department seeks candidates in the areas of social science statistics and data science, as well as related interdisciplinary fields. Potential areas of interest include network science, social simulation, data science for social good, simulation-based inference, cultural evolution, using large text and image corpora as data, and data privacy. Candidates with other research interests related to the work of both the Department and the Institute are also highly encouraged to apply.

The Institute for Complex Social Dynamics brings together scholars at Carnegie Mellon University who develop and apply mathematical and computational models to study large-scale complex social phenomena. The core members of the Institute are based in the Departments of Statistics and Data Science, Social and Decision Sciences, and Philosophy. Interests of the Institute include studies of the emergence of social behavior, the spread of misinformation, social inequality, and societal resilience.

As tenure-track faculty, the successful candidate will be expected to develop an independent research agenda, leading to publications in leading journals in both statistics and in suitable social-scientific venues; to teach courses in the department at both the undergraduate and graduate level; to supervise Ph.D. dissertations; to obtain grants; and in general to build a national reputation for their scholarship. The candidate will join the ICSD as a Core Member, and help shape the future of the Institute.

CMU's statistics department is unusually welcoming to those without traditional disciplinary backgrounds in statistics (after all, I'm here!), and that goes double for this position. If this sounds interesting, then apply by December 15th. (I'm late in posting this.) If this sounds like it would be interesting to your doctoral students / post-docs / other proteges, then encourage them to apply.

(If you'd like to join the statistics department, but are not interested in complex social dynamics what's wrong with you? we have another tenure track opening, where I'm not on the hiring committee.)

Kith and Kin; Enigmas of Chance

Come Post-Doc with Me!

Attention conservation notice: Soliciting applications for a limited-time research job in an arcane field you neither understand nor care about, which will at once require specialized skills and pay much less than those skills command in industry.

I am, for the first time, hiring a post-doc:

The Department of Statistics and Data Science at Carnegie Mellon University invites applicants for a two-year post-doctoral fellowship in simulation-based inference. The fellow will work with Prof. Cosma Shalizi of the department on developing theory, algorithms and applications of random feature methods in simulation-based inference, with a particular emphasis on social-scientific problems connected to the work of CMU's Institute for Complex Social Dynamics. Apart from by the supervisor, the fellow will also be mentored by other faculty in the department and the ICSD, depending on their interests and secondary projects, and will get individualized training in both technical and non-technical professional skills.

Successful applicants will have completed a Ph.D. in Statistics, or a related quantitative discipline, by September 2025, and ideally have a strong background in non-convex and stochastic optimization and/or Monte Carlo methods, and good programming and communication skills. Prior familiarity with simulation-based inference, social network models and agent-based modeling will be helpful, but not necessary.

Basically, I need someone who is much better than I am at stochastic optimization to help out with the matching-random-features idea. But I hope my post-doc will come up with other things to do, unrelated to their ostensible project (God knows I did), and I promise not to put my name on anything unless I actually contribute. If you don't have a conventional background in statistics, well, I'm open to that, for obvious reasons.

Beyond that, the stats. department is a genuinely great and supportive place to work, I hope for fabulous things from ICSD, and CMU has a whole has both a remarkable number of people doing interesting work and remarkably low barriers between departments; Pittsburgh is a nice and still-affordable place to live. Apply, by 15 December!

--- If I have sold you on being a post-doc here, but not on my project or on me, may I interest you in working on social networks dynamics with my esteemed colleague Nynke Niezink?

(The post-doc ad is official, but this blog post is just me, etc., etc.)

Self-Centered

30 Years of Notebooks

Attention conservation notice: Middle-aged dad has doubts about how he's spent his time.

In September 1994, I wanted to write a program which would filter the Usenet newsgroups I followed for the posts of most interest to me, which led me to writing out keywords describing what I was interested in. I don't remember why I started to elaborate the keywords into little essays and reading lists (perhaps self-clarification?), but I did, and then, because I'd just learned HTML and was playing around with hypertext, I put the document online. (My records say this was 3 October 1994, though that may have been fixing on a plausible date retroactively.) I've been updating those notebooks ever since, recording things-to-read as they crossed my path, recording my reading, and some thoughts. The biggest change in organization came pretty early: the few people who read it all urged me to split it from one giant file into many topical files, so I did, on 13 March 1995, ordered by last update, a format I've stuck to ever since (*).

This was not, of course, what I was supposed to be doing as a twenty-year-old physics graduate student. (Most of the notebook entries weren't even about physics.) Unlike a lot of ideas I had at that age, though, I stuck with it --- have stuck with it. Over the last thirty years, I've spent a substantial chunk of my waking hours recording references, consolidating what I understand by trying to explain it, and working out what I think by seeing what I write, by using Emacs to edit a directory of very basic HTML files. (I learned Emacs Lisp to write functions to do things like add links to arxiv.)

Was any of this a good use of my time? I couldn't begin to say. Long, long ago it became clear to me that I was never going to read more than a small fraction of the items I recorded as "To read:". I sometimes tell myself that it's a way of satiating my hoarding tendencies without actually filling my house with junk, but of course it's possible it's just feeding those tendencies. I do use the notebooks, though honestly the have-read portions are the most useful ones to me. Some of the notebooks have grown into papers, though many more which were intended to be seeds of papers have never sprouted. I know that some other people, from time to time, say they find them useful, which is nice. (Though I presume most people's reactions would range from bafflement to "wow, pretentious much?") Whether this justifies all those hours not writing papers / finishing any of my projected books / gardening / hanging out with friends / being with my family / playing with my cat (RIP) / drinking beer / riding my bike / writing letters / writing al-Biruni fanfic / actually reading, well...

The core of the matter, I suspect, is that if anyone does anything for a decade or three consistently, it becomes a very hard habit to break. By this point, the notebooks are so integrated into the way I work that it would take lots of my time and will-power to stop updating them, as long as I keep anything like my current job. So I will keep at it, and hope that it is, at worst, a cheap and harmless vice.

I never did write that Usenet filter.

*: A decade later, I started using blosxom, rather than completely hand-written HTML, and Danny Yee wrote me a cascading style sheet. I also was happy to use first HTMX, and then MathJax, to render math, rather than trying to put equations into HTML. ^.

The Professoriate Considered as a Super-Critical Branching Process

Attention conservation notice: Academic navel-gazing, in the form of basic arithmetic with unpleasant consequences that I leave partially implicit.

A professor at a top-tier research university who graduates only six doctoral students over a thirty year career is likely regarded by their colleagues as a bit of a slacker when it comes to advising work; it's easy to produce many more new Ph.D.s. (Here is a more representative case of some personal relevance.) That slacking professor has nonetheless reproduced their own doctorate six-fold, which works out to $\frac{\log{6}}{30} \approx$ 6% per year growth in the number of Ph.D. holders. Put this as a lower bound --- a very cautious lower bound --- on how quickly the number of doctorates could grow, if all those doctorate-holders became professors themselves. Unless faculty jobs also grow at 6% per year, which ultimately means student enrollment growing at 6% per year, something has to give. Student enrollment does not grow at 6% per year indefinitely (and it cannot, even if you think everyone should go to college); something gives. What gives is that most Ph.D.s will not be employed in the kind of faculty position where they train doctoral students. The jobs they find might be good, and even make essential use of skills which we only know how to transmit through that kind of acculturation and apprenticeship, but they simply cannot be jobs whose holders spawn more Ph.D.s.

The professoriate is a super-critical branching process, and we know how those end. (I am a neutron that didn't get absorbed by a moderator; that makes me luckier than those that did get absorbed, not better.) In the sustainable steady state, the average professor at a Ph.D.-granting institution should expect to have one student who also goes on to be such a professor in their entire career.

Anyone who takes this as a defense of under-funding public universities, of adjunctification, or even of our society having more non-academic use for quantitative skills than for humanistic learning, has trouble with reading comprehension. Also, of course this is Malthusian reasoning; what made Malthus wrong was not anticipating that what he called "vice" could become universal the demographic transition. Let the reader understand.

Learned Folly

Books to Read While the Algae Grow in Your Fur, September 2024

Attention conservation notice: I have no taste, and no qualifications to opine on world history, or even on random matrix theory. Also, most of my reading this month was done at odd hours and/or while chasing after a toddler, so I'm less reliable and more cranky than usual.

Marc Potters and Jean-Philippe Bouchaud, A First Course in Random Matrix Theory: for Physicists, Engineers and Data Scientists, doi:10.1017/9781108768900

I learned of random matrix theory in graduate school; because of my weird path, it was from May's Stability and Complexity in Model Ecosystems, which I read in 1995--1996. (I never studied nuclear physics and so didn't encounter Wigner's ideas about random Hamiltonians.) In the ensuing nearly-thirty-years, I've been more or less aware that it exists as a subject, providing opaquely-named results about the distributions of eigenvectors of matrices randomly sampled from various distributions. It has, however, become clear to me that it's relevant to multiple projects I want to pursue, and since I don't have one student working on all of them, I decided to buckle down and learn some math. Fortunately, nowadays this means downloading a pile of textbooks; this is the first of my pile which I've finished.

The thing I feel most confident in saying about the book, given my confessed newbie-ness, is that Potters and Bouchaud are not kidding about their subtitle. This is very, very much physicists' math, which is to say the kind of thing mathematicians call "heuristic" when they're feeling magnanimous *. I am still OK with this, despite years of using and teaching probability theory at a rather different level of rigor/finickiness, but I can imagine heads exploding if those with the wrong background tried to learn from this book. (To be clear, I think more larval statisticians should learn to do physicists' math, because it is really good heuristically.)

To say just a little about the content, the main tool in here is the "Stieljtes transform", which for an $N\times N$ matrix $\mathbf{A}$ with eigenvalues $\lambda_1, \ldots \lambda_N$ is a complex-valued function of a complex argument $z$, \[ g^{\mathbf{A}}_N(z) = \frac{1}{N}\sum_{i=1}^{N}{\frac{1}{z-\lambda_i}} \] This can actually be seen as a moment-generating function, where the $k^{\mathrm{th}}$ "moment" is the normalized trace of $\mathbf{A^k}$, i.e., $N^{-1} \mathrm{tr}{\mathbf{A}^k}$. (Somewhat unusually for a moment generating function, the dummy variable is $1/z$, not $z$, and one takes the limit of $|z| \rightarrow \infty$ instead of $\rightarrow 0$.)

The hopes are that (i) $g_N$ will converge to a limiting function as $N\rightarrow\infty$, \[ g(z) = \int{\frac{\rho(d\lambda)}{z-\lambda}} \] and (ii) the limiting distribution $\rho$ of eigenvalues can be extracted from $g(z)$. The second hope is actually less problematic mathematically **. Hope (i), the existence of a limiting function, is just assumed here. At a very high level, Potters and Bouchaud's mode of approach is to derive an expression for $g_N(z)$ in terms of $g_{N-1}(z)$, and then invoke the assumption (i), to get a single self-consistent equation for the limiting $g(z)$. There are typically multiple solutions to these equations, but also usually only one that makes sense, so the others are ignored ***.

At this very high level, Potters and Bouchaud derive limiting distributions of eigenvalues, and in some cases eigenvectors, for a lot of distributions of matrices with random entries: symmetric matrices with IID Gaussian entries, Hermitian matrices with complex Gaussian entries, sample covariance matrices, etc. They also develop results for deterministic matrices perturbed by random noise, and a whole alternate set of derivations based on the replica trick from spin glass theory, which I do not feel up to explaining. These are then carefully applied to topics in estimating sample covariance matrices, especially in the high-dimensional limit where the number of variables grows with the number of observations. This in turn feeds in to a final chapter on designing optimal portfolios when covariances have to be estimated by mortals, rather than being revealed by the Oracle.

My main dis-satisfaction with the book is that I left it without any real feeling for why the eigenvalue density of symmetric Gaussian matrices with standard deviation $\sigma$ approaches $\rho(x) = \frac{\sqrt{4\sigma^2 - x^2}}{2\pi \sigma^2}$, but other ensembles have different limiting distributions. (E.g., why is the limiting distribution only supported on $[-2\sigma, 2\sigma]$, rather than having, say, unbounded support with sub-exponential tails?) That is, for all the physicists' tricks used to get solution, I feel a certain lack of "physical insight" into the forms of the solutions. Whether any further study will make me happier on this score, I couldn't say. In the meanwhile, I'm glad I read this, and I feel more prepared to tackle the more mathematically rigorous books in my stack, and even to make some headway on my projects. §

*: As an early example, a key step in deriving a key result (pp. 21--23) is to get the asymptotic expected value of such-and-such a random variable. Using a clever trick for computing the elements of an inverse matrix in terms of sub-matrices, they get a formula for the expected value of the reciprocal of that variable. They then say (eq. 2.33 on p. 22) that this is clearly the reciprocal of the desired limiting expected value, because after all fluctuations must be vanishing. ^

**: We consider $z$ approaching the real axis from below, say $z=x-i\eta$ for small $\eta$. Some algebraic manipulation then makes the imaginary part of $g(x-i\eta)$ look like the convolution of the eigenvalue density $\rho$ with a Cauchy kernel of bandwidth $\eta$. A deconvolution argument then gives $\lim_{\eta \downarrow 0}{\mathrm{Im}(gx-i\eta)} = \pi \rho(x)$. This can be approximated with a finite value of $N$ and $\eta$ (p. 26 discusses the numerical error). ^

***: There is an interesting question about physicists' math here, actually. Sometimes we pick and choose among options that, as sheer mathematics, seem equally good, we "discard unphysical solutions". But sometimes we insist that counter-intuitive or even bizarre possibilities which are licensed by the math have to be taken seriously, physically (not quite "shut up and calculate" in its original intention, but close). I suspect that knowing when to do one rather than the other is part of the art of being a good theoretical physicist... ^

Fernand Braudel, The Perspective of the World, volume 3 of Civilization and Capitalism, 15th--18th Century

This is the concluding volume of Braudel's trilogy, where he tries (as the English title indicates) to give a picture of how the world-as-a-whole worked during this period. It's definitely the volume I find least satisfying. Braudel organizes everything around a notion of "world economies" borrowed from Immanuel Wallerstein (an unfortunate choice of guide), postulating that these are always centered on a single dominating city, and spends a lot of his time tracking the shifts of what he says is the dominating city of the European world economy. But by his own definition of world economy, I don't see how there was more than one during his period, because all his other "world economies (East Asia, India, sub-Saharan Africa, the Americas, etc., etc.) were all tied in to the same economic system as western-and-central Europe. In fact, Braudel goes on at great length about these ties! (At most, Australia and Oceania might have been outside the world economy during this period.) This is also the volume where the, let us say, eccentricity of Braudel's economic thought began to press on me *. It was his discussions of cycles, "the conjuncture" and time-series decomposition which however truly irritated me. Or, rather, it made me want to sit him down and give him a lecture on the Yule-Slutsky effect, because I am quite certain he was smart enough to grasp it **. --- All these remarks are, of course, the height of presumption on my part. §

Previously.

*: After quoting a detailed passage from Ricardo about how both Portugal and England are better off if the former grows wine and the latter grows wheat and they exchange, Braudel spends many pages going over how Portuguese wine-growers came to rely on credit from English merchants. Stipulating that this is all true, and even stipulating that in some sense those English merchants dominated the Portuguese vintners, it does not refute Ricardo! (The cooperative socialist commonwealth will care very much about comparative advantage.) Or, again, Braudel repeatedly talks about how certain cities or countries were dis-advantaged by their high wages, without ever considering that some employers there must have felt those wages were worth paying. Indeed many employers there must have, or those would not have been the prevailing wages. --- In general, I sympathize with wanting to rescue older perspectives, here those of the mercantilists, from the condescension of posterity, but I think Braudel takes that too far, to the detriment of his understanding of his material. ^

**: To be fair, there are some hints in those passages that Braudel might have been happy to accept Slutsky's perspective on the effect. Namely: the appearance of low-frequency cycles is the natural consequence of high-frequency noise (what Braudel would call "events") whose effects just take time to work their way through the economic system. (This reminds me that I need to actually read Barnett's biography of Slutsky one of these years.) ^

Tamim Ansary, The Invention of Yesterday: A 50,000-Year History of Human Culture, Conflict, and Connection

It's not quite true to say that this is an attempt to write Marshall Hodgson's never-completed world history as a volume of pop history. This is not quite true because it is also, and equally, inspired by McNeill and McNeill's The Human Web. The result is extremely engaging, and while I didn't particularly learn from it, I daresay most of the prospective audience will not, in fact, have read as widely in Ansary's sources as I happen to have done. §

Errata: When describing Mesopotamian civilizations, Ansary repeatedly refers to Sumerian as a Semitic language, which is wrong. This is not particularly consequential, and I didn't notice any other errors of fact.

Disclaimer: My grandfather and Ansary's father were friends, so he's a family connection.

Books to Read While the Algae Grow in Your Fur; Writing for Antiquity; The Great Transformation; Mathematics; Enigmas of Chance; The Dismal Science

Books to Read While the Algae Grow in Your Fur, August 2024

Attention conservation notice: I have no taste, and no qualifications to opine on world history. Also, most of my reading this month was done at odd hours and/or while chasing after a toddler, so I'm less reliable and more cranky than usual.

Fernand Braudel, The Wheels of Commerce, volume 2 of Civilization and Capitalism, 15th--18th Century

On re-reading, I am much more struck by Braudel's constant, but un-supported, assertions about who controlled various trades. "Power" and "control" are unavoidably causal notions, which means that at the very least they involve counterfactuals, the sort of thing B. would normally say he avoids. A bit more concretely: say Braudel is right that in the late 1500s and earlier 1600s, the European international currency market was funneled through a small number of mostly Genovese merchants/bankers operating at periodic fairs. (Subsequent scholarship seems to agree.) I'm sure those traders made money. In fact, I'm sure that they exploited network externalities to make supra-normal profits. But asserting that they controlled those currency markets implies that they could have imposed different outcomes --- on exchange rates? on discount rates? Braudel never bothers to say --- if they had wanted to, as opposed to having their actions more or less dictated by the real economic activity which generated bills of exchange, demand for currency exchange, etc. Braudel does not provide evidence for control, and it doesn't even arise as a problem within his horizon.

Wishing Braudel had collaborated with someone like (impossibly) Paul Krugman, or (more plausibly, but still impossibly) Charles Kindleberger, is both idle and impertinent, even philistine, but I can't help it.

Sequel.

Glen Cook, The Black Company, Shadows Linger, The White Rose (also an omnibus, Chronicles of the Black Company, not seen by me)

Mind candy military fantasy, in which Our Protagonists find themselves working as mercenaries for the Dark Lord's only-barely-less-evil ex-wife, on whom the narrator develops an unhealthy crush. (These are not spoilers.) There are a lot of these, and I might pick up more later, but this first trilogy comes to a satisfying ending point.

(I'd be very surprised if these weren't an influence on Graydon Saunders, though the prose style is rather different.)

Megan E. O'Keefe, The Blighted Stars

Mind candy: a very angsty romance wrapped in a shell of space opera. Both aspects of the story are left in media res, and I look forward to reading the sequels.

Phillip Kennedy Johnson et al., The Last God

Mind candy, comic book fantasy heavily influenced by D & D.

Wen Spencer, Tinker

Mind candy fantasy, in which Pittsburgh is transported to Elfland for most of each month. I read it for the local interest (admission to CMU plays a small part of the plot, and bad things happen in Turtle Creek [which I can only too easily believe]), and it was OK, but not good enough to make me pick up any of the many sequels.

Lilith Saintcrow, Moon's Knight

Mind candy portal fantasy, in which Our Protagonist's struggles to escape back to Mundania are rather complicated by her growing recognition that her life there sucked, actually, and maybe fighting strange beasts and stranger people isn't so bad in comparison...

Books to Read While the Algae Grow in Your Fur; Scientifiction and Fantastica; Writing for Antiquity; The Dismal Science; Heard About Pittsburgh PA

Books to Read While the Algae Grow in Your Fur, July 2024

Attention conservation notice: I have no taste. Also, most of my reading and viewing this month was done at odd hours and/or while chasing after a toddler, so I'm less reliable and more cranky than usual.

Elizabeth Bear, Ancestral Night

J. S. Dewes, The Last Watch and The Exiled Fleet

Space opera mind candy. The Bear is good, as usual, but not quite her best. Dewes is new to me, and I'd say not quite as good at either world-building or character development as Bear, but still enjoyable, and I'll get the third book in this trilogy when it comes out.

(Parenthetical with implicit spoilers for Ancestral Night: I can't decide if Bear's characters are merely fooling themselves when they assert that the Synarchy has progressed beyond money, or if Bear has not thought through what "providing more value than you use up" is going to entail, especially when one needs to balance, say, a recovered spacecraft hull against expended fuel. Since Bear's narrator is, demonstrably confused about a lot of important matters, I am inclined to think this is the character's mis-apprehension, perhaps encouraged by propaganda. [But then, I would.])

Prometheus

Spoilers for this movie from 2012 follow.

I enjoy a good re-telling of At the Mountains of Madness more than the next fan, and am pleased to learn that xenomorphs are, in fact, shoggothim. (That is: originally amorphous bits of protoplasm, built as weapons or tools, which learned to imitate their creators and then destroyed them.) But there's a huge part of the plot which makes no sense: if you thought humanity was the creation of beings who were merely an advanced alien species, merely engineers, why of why would you think they have any more of an answer to the riddles of existence than we do, or even a way of making human bodies last forever? The fact that the existentially-befuddled human beings in this scenario have created intelligent androids would seem to make this obvious. (In fact the android character basically says as much!)

Also: Am I right in thinking that this is the first time "being infected by the alien parasite is like being pregnant" has moved from subtext to text?

2001: A Space Odyssey

Re-watched for the nth time as a palette-cleanser after Prometheus. Icy perfection from start to finish. (The iciness is part of the perfection.) --- The influence of Stapledon's Last and First Men and Star-Maker on Clarke, and so on this movie, is, naturally, very patent to me on this re-watch.

The Hidden Fortress, Yojimbo, Sanjuro

And since I was staying up late watching beloved classics... I can't remember exactly how old I was when my father took my brother and me to see a revival of The Hidden Fortress at the old Biograph Theater in Georgetown, but we couldn't have been more than eleven, and we both imprinted. I am happy to say these movies, too, only improve with age and re-watching. §

Books to Read While Algae Grow in Your Fur; Scientifiction and Fantastica; Tales of Our Ancestors

The "Quality Control" Interview for Big Classes

Attention conservation notice: Advice on teaching, which I no longer follow myself.

I teach a lot of big classes --- the undergraduate advanced data analysis class passed 100 students many years ago, and this spring is over 230 --- which has some predictable consequences. I don't get to talk much to many of the students. They're mostly evaluated by how they do on weekly problem sets (a few of which, in some classes, I call "take-home exams"), and I don't even grade most of their homework, my teaching assistants do. While I try to craft problem sets which make sure the students practice the skills and material I want them to learn, and lead them to understand the ideas I want them to grasp, just looking at their scores doesn't give me a lot of information about how well the homework is actually working for those purposes. Even looking at a sample of what they turn in doesn't get me very far. If I talk to students, though, I can get a much better sense of what they do and do not understand fairly quickly. But there really isn't time to talk to 100 students, or 200.

About ten years ago, now, I decided to apply some of the tools of my discipline to get out of this dilemma, by means of random sampling. Every week, I would randomly select a fixed number of students for interviews. These interviews took no more than 30 minutes each, usually more like 20, and were one-on-one meetings, distinct from regular open office hours. They always opened by me asking them to explain what they did in such-and-such a problem on last week's homework, and went on from there, either through the problem set, or on to other topics as those suggested themselves.

In every class I did this in, it gave me a much better sense of what was working in the problems I was assigning and what wasn't, which topics were actually getting through to students and which were going over their heads, or where they learned to repeat examples mechanically without grasping the principle. There were some things which made the interviews themselves work better:

• Reading each students' homework, before the meeting. (Obvious in retrospect!)
• Handing the student a copy of what they turned in the week before. (Though, as the years went on, many brought their laptops and preferred to bring up their copy of the document there.)
• Putting a firm promise in the syllabus that nothing students said in the interview would hurt their grade. (Too many students were very nervous about it otherwise.)
• Putting an equally firm promise in the syllabus that not coming in to the interview, or blowing it off / being uncooperative, would get them a zero on that homework. (Obvious in retrospect.)
• Offering snacks at the beginning of the interview.

Setting aside a fixed block of time for these interviews didn't actually help me, because students' schedules are too all-over-the-place for that to be useful. (This may differ at other schools.)

Choosing the number of students each week to interview has an obvious trade-off of instructor time vs. information. I used to adjust it so that each student could expect to be picked once per semester, but I always did sampling-with-replacement. In a 15-week semester with 100 students, that comes out to about 3.5 hours of interviews every week, which, back then, I thought well worthwhile.

I gave this up during the pandemic, because trying to do a good interview like this over Zoom is beyond my abilities. I haven't resumed it since we went back to in-person teaching, because I don't have the flexibility in my schedule in any more to make it work. But I think my teaching is worse for not doing this.

Corrupting the Young

The Presentation Exchange for Workshops and Classes

Attention conservation notice: Advice for running an academic workshop, which I've only followed myself a few times.

Some years ago, Henry Farrell and I ran a series of workshops about cooperative problem-solving and collective cognition where we wanted to get people with very different disciplinary backgrounds --- political theorists, computer scientists, physicists, statisticians, cognitive psychologists --- talking to each other productively. We hit upon an idea which worked much better than we had any right to hope. (Whether it's ultimately due to him, or me, or to one of us tossing it out as obviously dumb and the other saying "Actually...", neither of us can now recall.) We've both used it separately a few times in other settings, also with good results. Since we both found ourselves explaining it recently, I thought I'd describe it in a brief note.

1. Every participant in the workshop writes a brief presentation, with enough lead time for the organizers to read them all.
   In the context of an inter-disciplinary workshop, what often works best is to describe an outstanding problem in the field.
2. The workshop organizers semi-randomly assign each participant's presentation to someone else, with enough lead time that the assignee can study the presentation.
   Again, in the interdisciplinary context, the organizers try to make sure that there's some hope of comprehension.
   (While I called this the "presentation exchange", it needn't be a strict swap, where A gets assignd B's presentation and vice versa.)
3. Everyone gives the presentation they were assigned, followed by their own comments on what they found interesting / cool / provocative and what they found incomprehensible. No one gives the presentation they wrote.
   In some contexts, I have found it helpful to institute the rule that the author don't get to speak until after the presentation is finished...

Doing this at the beginning of the workshop helps make sure that everyone has some comprehension of what everyone else is talking about, or at least that mis-apprehensions or failures to communicate are laid bare. It can help break up the inevitable disciplinary/personal cliques. It can, and has, spark actual collaborations across disciplines. And, finally, many people report that knowing their presentation is going to be given by someone else forces them to write with unusual clarity and awareness of their own expert blind-spots.

As I said, Henry and I hit on this for interdisciplinary workshops, but I've also used it for disciplinary workshops --- because every discipline is a fractal (or lattice) of sub-sub-...-sub-disciplinary specialization. I've also used it for student project classes, at both the undergrad and graduate level. That requires more hand-holding and/or pastoral care on the part of the teacher than a research workshop, and I've never tried to make it the way I start a class.

Learned Folly; The Collective Use and Evolution of Concepts; Corrupting the Young

Books to Read While the Algae Grow in Your Fur, July 2023

Attention conservation notice: I have no taste, and no qualifications to opine on the sociology and industrial organization of intellectuals, political philosophy, or American history. Also, most of my reading this month was done at odd hours and/or while bottle-feeding a baby, so I'm less reliable and more cranky than usual.

Allison Brennan, The Lost Girls, Make Them Pay, Breaking Point, Too Far Gone

Mind candy series mystery. As with many long-running series, the soap-operatic elements keep piling up, and I honestly enjoyed those less than seeing Lucy tackle the murder-or-kidnapping-of-the-week, but still fun. (Previously.) §

Daniel Drezner, The Ideas Industry: How Pessimists, Partisans, and Plutocrats are Transforming the Marketplace of Ideas

Popular social science. Drezner's main argument is as follows. He begins by distinguishing between "public intellectuals", who are critical and multi-sided, and "thought leaders", who have One Big Idea (if not One Weird Trick), which they push relentlessly. (I don't think the phrase "policy entrepreneur" appears in the book; the old-fashioned but apt term "projector" definitely doesn't.) Recent changes in the societies of the rich democracies have increased the sway of thought leaders, and reduced that of public intellectuals.

One of these is rising economic inequality ("plutocrats"): rich people are constitutionally more inclined to pay for advocacy, especially flattering or self-serving advocacy, than for critique. Here Drezner advances, without much fuss, some sensible-sounding notions about the relations between material interests and ideology. (I actually wish he'd elaborate a theory of ideology on this basis, but that would call for a different sort of book.)

A second change is the rise of partisanship. This makes it easier to ignore criticisms coming from the other side. (You will, after all, often be right in thinking that those criticisms are made ignorantly, in bad faith, or merely to posture before the critic's own side.) This is, of course, bad for reason and democracy.

The third change is the decline in trust in established institutions ("pessimism"). These have not been replaced by alternative gate-keeping institutions, but rather by more of a free-for-all scrum for attention. (Again: "Actually, 'Dr. Internet' is the name of the monsters' creator.") This exacerbates already-existing tendencies in intellectual life to highly-skewed, winner-take-a-hell-of-a-lot outcomes. His descriptions of the temptations to chase those rewards is vivid.

Drezner does little to address why plutocrats, partisans, and the plain people of the Internet should have such an appetite for intellectual fare. It's probably impossible for social animals of our sort to conduct our common lives without justifications and rationalizations (cf. Mercier and Sperber). That those rationales should be intellectual, that they should take the form of culturally-transmitted abstractions, general ideas, appeals to impersonal principle, appeals to evidence, attempts at logical argument, etc., is another matter and evidently far more contingent. Here I personally would gesture at the very high levels of education attained in all the countries Drezner is concerned with, and/or generations of the Flynn effect.

Drezner is careful to explain that the changes and prospects are not all grim. (There are real benefits to less gatekeeping, even for public intellectuals in Drezner's sense.) He's also careful to note that in many ways the social life of the mind has always been bad. (This is cold comfort, but at least avoids catastrophizing.) But he leaves me convinced that he's right about specific ways in which that social life has recently become dysfunctional than it was, with little prospect of improvement in the foreseeable future. §

Disclaimer: Drezner is a co-author of a co-author, and a fellow relic of the The Second Age of the Web early '00s blogging. But I don't think we've ever met, and I feel no obligations to read or to praise his books. (Especially not years after they come out...)

Cormac McCarthy, Blood Meridian, or, The Evening Redness in the West

Re-read in memoriam. This is a strange but effective fusion of truly ugly action and truly beautiful language. Revisiting after some decades, I can see how it's influenced a lot of other, later books I've read, some for the better, some very much not. (If it weren't for the dates, I'd think Stephen King's The Gunslinger was in the former set.)

Two thoughts: 1. As usual, it's a mistake to identify the opinions of characters --- even ones who are given a lot of room to opine --- with the opinions of the author. In particular, I see a lot of people quoting the judge's speeches as though they were Cormac's views, but the action of the novel makes it clear that the judge is a cunning, deceitful, possibly-inhuman villain! He is not to be trusted! (Reading is hard.) 2. Something about the narration's frequent recourse to the ancient, the primeval, to mysterious forces under the earth, etc., makes me wonder about what Cormac thought of Lovecraft. §

Disclaimer: I knew Cormac through SFI; not well, but well enough to call him Cormac.

Tommie Shelby, The Idea of Prison Abolition

This is a thorough and sympathetic, but ultimately very negative, investigation of case for abolition of prisons, from a view point that tries to meld analytical Marxism with what's come to be called the "black radical tradition" [1]. Much of the argument here proceeds by way of exposition and critique of the prison-abolitionist writings of Angela Davis [2].

Many self-proclaimed prison abolitionists seem to merely be expressing outrage at way we treat crime through hyperbole. But some of them mean it. (Some of them, I suspect, have been swayed by their own hyperbole.) In any event it's a morally serious issue, which deserves to be examined with some care, whatever one might think of some of its advocates. This Shelby does.

Shelby outright dismisses the idea that society might have a legitimate interest in meeting out retribution for crimes [3], but accepts interests in deterrence [4], in rehabilitation, and (I think) in incapacitation. He further explains that consequences for anti-social behavior will only deter if they are, in fact, unpleasant. This does not mean that those consequences need to be horrors, but unless people would rather not experience them, they simply will not work. Even if one wishes to emphasize gentler means that might better serve the aims of rehabilitation and (perhaps) incapacitation, those will need to be back-stopped by some kind of deterrence of those who are neither rehabilitated nor incapacitated.

Shelby tries his best to be fair to Davis's claims that the legitimate social functions of prisons can be better served without imprisonment, but ends up having to admit that there just isn't very much substance to those claims. I honestly doubted whether he was really being fair to Davis here, so at this point read her Are Prisons Obsolete?, and concluded that Shelby was being, in fact, far too generous.

To sum up, Shelby pretty convincingly demolishes the arguments for prison abolition, i.e., for thinking that prisons have no place in just societies. He is very careful to say that none of his arguments imply that current American prisons, or our criminal justice system more generally, are acceptable. §

Disclaimer: I met Shelby years ago at a workshop, where I was impressed by his presentation, and he was generous with his time in offering suggestions on work-then-in-progress. This contributed to my picking up his book.

[1]: Shelby elaborates on his conception of his own "Afro-Analytical Marxism" in this 2021 essay. Like most analytical Marxists, he seems more interested in fairly orthodox historical materialism and political economy --- the sort of topics someone shaped by the Second International, like Kautsky or Trotsky or Luxemburg, would've recognized --- than in the Frankfurt School. (Davis, of course, as Marcuse's student, owes more to Frankfurt.) Thus I think can continuing to view Joseph Heath as the world's leading, because only, rational-choice critical theorist. ^

[2]: Certain episodes in Davis's career go (tactfully?) unmentioned. ^

[3]: The dismissal is forthright, but perhaps a bit hasty. Those who are wronged by others, or their family and friends, will tend to seek retribution from those who have wronged them. In fact they will tend to seek disproportionate and intemperate retribution. Such excessive retribution is both unjust itself, and apt to set of a vicious cycle of feud and revenge. To prevent this, punishment of wrong-doers by the state must include, and be seen to include, reasonable and proportionate retribution. --- To be clear, I'm not saying this is unanswerable, just that I wish Shelby hadn't dismissed retribution so swiftly. ^

[4]: There is a disconcerting possibility about deterrence which Shelby doesn't discuss, but which his arguments do not, so far as I can see, foreclose. This is that punishing people for crimes they didn't commit would have much the same deterrent effect as punishing the guilty, so long as most people thought that they were guilty. Someone has to suffer in order to fulfill the legitimate public function of deterring wrong-doing [5], but it's trickier than I'd like to say why, ethically, it should be criminals who do the suffering. (Of course, the task becomes easier if one believes in retribution.) ^

[5]: Conversely, I could easily make a case for the authorities only convincingly pretending to punish anyone. But such a deception would be very fragile, with bad consequences when it unraveled; perhaps that's enough to rule it out. ^

Bryan Burrough, Days of Rage: America's Radical Underground, the FBI, and the Forgotten Age of Revolutionary Violence

A journalistic, but very thorough, history of violent left-wing radicals from the late 1960s through the early 1980s. (Right-wing violence during the same period is outside Burrough's scope, but it would make an interesting set of comparison cases.) Many of the figures he discusses --- including Davis! --- also show up in Shelby's book, albeit presented in rather different lights. §

Adolph Reed, Jr., The South: Jim Crow and Its Afterlives

If you like Reed's essays at nonsite.org (and I usually do), you will enjoy this, and if not, not. The marketing material from the publisher makes it seem vastly more ambitious than it really is, but Reed's introductory remarks make the scope clear. §

Simon Spurrier and Matías Bergara, Coda vols. 1, 2, 3

Comic book mind candy fantasy. Superficially, this is a cynical, post-apocalyptic subversion of the Matter of Middle Earth. In fact, the hard-bitten surface merely conceals a core which actually believes in epic fantasy, both in the content and in the classical form (a trilogy ending in a eucatastrophe). §

Books to Read While the Algae Grow in Your Fur; The Beloved Republic; The Progressive Forces; Philosophy; Commit a Social Science; The Collective Use and Evolution of Concepts; Pleasures of Detection, Portraits of Crime; Scientifiction and Fantastica; Tales of Our Ancestors; The Commonwealth of Letters

On Shoggothim

Attention conservation notice: Self-promotion of a pay-walled piece which combines a trendy topic with what even I admit is a long-held semi-crank notion.

Henry Farrell and I have an essay in The Economist, riffing off the meme that every large language model is really a shoggoth. Our point is that this is right, because an LLM is a way of taking the vast incohate chaos of written-human-language-as-recorded-on-the-Web and simplifying and abstracting it in potentially useful ways. They are, as Alison Gopnik says, cultural technologies, more analogous to library catalogs than to individual minds. This makes LLMs recent and still-minor members of a larger and older family of monsters which similarly simplify, abstract, and repurpose human minds: the market system, the corporation, the state, even the democratic state. Those are distributed information-processing systems which don't just ingest the products of human intelligence, but actually run on human beings --- a theme I have been sounding for while now.

The piece is paywalled, but Henry has a Twitter thread that provides a good summary, and Brad DeLong has excerpts, along with thoughtful commentary. (I agree with Henry's response to said comments.) Update, 7 July: Henry links to the longer, older version we cut down for The Economist.

Some things we didn't include:

• Thanks to the editorial staff at The Economist, both for the opportunity and for their very professional work.
• Thanks to Ted Chiang (!) for helpful comments on a draft.
• Any discussion of LLMs as artifacts, in the sense of Herbert Simon's Sciences of the Artificial. (I for one learned this way of thinking of markets and hierarchies as information-processing systems from Simon...) Update, 17 August: I endorse Maxim Raginsky's treatment of this topic.
• Any discussion of Dan Sperber's account of culture as "the precipitate of cognition and communication in a human population", the role in that process of chains of alternately private-mental and public-physical representations, and LLMs as public-representation-producing artifacts
• Any discussion of Arthur Stinchcombe's work on the positive role of abstraction and formalities in institutions
• "More is different": These things emerge from the massed results of human social interaction and individual intelligence, and therefore are very different from human minds. In particular, they tend to have their own intrinsic dynamics, which are usually not things anyone intends, and often things no-one wants. (Someday I will write that essay about blackouts and alienation.) That doesn't mean they can't be controlled; it means control is hard, and usually itself impersonal.
• An adequate discussion of monster-taming and its limits, which would necessarily include extended praise of social democracy (though see DeLong's post)
• Any mention of the the primal scene of AI.
• Henry's reflections on modern neo-Lovecraftian fiction, which I hope he will publish elsewhere. Update, 7 July: see.

Update, 23 June: Small wording tweaks and additions. More important: insightful and generous commentary from Daniel "\( D^2 \)" Davies. (It's virtually a blogosphere reunion.) Incorporated (sorry) by reference: Beniger, The Control Revolution; Yates, Control through Communication; Ashby, "Design for an Intelligence Amplifier".

(I know I learned that the correct plural of "shoggoth" is "shoggothim" from reading Ruthanna Emrys, but I cannot now locate the passage --- it may just be in her Lovecraft Reread series with Anne Pillsworth.) Update: and indeed it was (tracked down by Henry).

Self-Centered; The Collective Use and Evolution of Concepts; Cthulhiana; The Great Transformation

Books to Read While the Algae Grow in Your Fur, April 2023

Attention conservation notice: I have no taste, and no qualifications to opine on the biographies of 20th century tyrants, or the impact of the Internet on collective creativity. Also, most of my reading this month was done at odd hours and/or while bottle-feeding a baby, so I'm less reliable and more cranky than usual.

Wislawa Szymborska, Poems New and Collected, 1957--1997

Donald Hall, Selected Poems

I observe National Poetry Month by reading poets I really ought to have read already. (I'd seen Szymborska's "A Word on Statistics", of course, IIRC from Thomas Lumley.)

Leigh Bardugo, Hell Bent

Mind candy fantasy / campus novel, in which Yale is literally a gateway to Hell. It's a sequel to Ninth House, and it'll be much more enjoyable if you read that first, but there's enough cluing-in for the new reader that it's probably not necessary. Ends in media res. §

Andrea Fort et al., Songs for the Dead: Afterlife

Mind candy fantasy, comic book flavor. A satisfying conclusion to the story. §

Robert Louis Stevenson, The Strange Case of Dr. Jekyll and Mr. Hyde

Yes, I knew the story. No, I had never actually read it before. Yes, it's really good. §

Stephen Kotkin, Stalin, volumes I, Paradoxes of Power, 1878--1928 and II, Waiting for Hitler, 1929--1941

Writing an adequately-contextual biography of Stalin means, for Kotkin, pretty much writing a history of the world, as well as detailing the ups and downs of Ioseb Barionis Jughashvili. I think this is right, and am entranced at how well Kotkin tacks back and forth between different scales. One of the themes those constant changes of scale let Kotkin explore is the tension between large, structural forces or trends --- particularly the imperative pressure on any state that wanted to retain independence to industrialize (cf.) --- and fine-grained and contingent yet consequential facts of friendship and rivalry, of personality, even of sheer accident. (These are very non-Marxist books, which could only have been written by someone who had seriously wrestled with Marxist thought.) I very eagerly await the next volume (or volumes?). §

Clay Shirky, Cognitive Surplus: Creativity and Generosity in a Connected Age

Reading a 2010 book about the promise of the Internet for cooperation, especially for intellectual collaboration, in 2023 is, well, rather melancholy. Instead of carpooling, we have giant illegal taxi companies; instead of safe couch-surfing, we have giant illegal hotel chains; instead of sharing information about political violence, we have organizing political violence; and instead of sharing information about rare medical conditions, we have created multiple new forms of contagious hysteria.

One conclusion I draw from this is that Shirky was fundamentally right about how the Internet would unleash new forms of collective creativity, but far, far too optimistic about the value of that creativity. ("After all, to any rational mind, the greater part of the history of ideas is a history of freaks.")

The other conclusion --- one I've been tending to for a while --- is that as a teenager, I got caught up in a Utopian milieu, which somehow thought that integrating the Internet, and especially the Web, into civilized life would make things better. I spent my adult life in this environment, it was very good to me (and I daresay to Shirky). But, thirty years later... Well, I often find myself thinking on a passage from Fear and Loathing in Las Vegas, reflecting on another such hangover:

There was a fantastic universal sense that whatever we were doing was right, that we were winning...
And that, I think, was the handle --- that sense of inevitable victory over the forces of Old and Evil. Not in any mean or military sense; we didn't need that. Our energy would simply prevail. There was no point in fighting --- on our side or theirs. We had all the momentum; we were riding the crest of a high and beautiful wave...
So now, less than five years later, you can go up on a steep hill in Las Vegas and look West, and with the right kind of eyes you can almost see the high-water mark --- that place where the wave finally broke and rolled back.

Shirky was offering a view from the crest of the wave. This one didn't exactly break and roll back; it just left the same old rubbish as before in its wake, only sodden and salt-rimed. This is, perhaps, the best a utopia can hope to achieve. §

Disclaimer: I'd forgotten, until I was almost ready to post this, that back in the Second Age of the Web 2003--2004 Shirky and I were both parties to a discussion involving the exact shape of the degree distribution for weblogs. That dispute is irrelevant to the subject of this book, and has no bearing on my views of it. (For the record: he was wrong about the degree distribution.)

Books to Read While the Algae Grow in Your Fur; Writing for Antiquity; The Progressive Forces; Linkage; The Collective Use and Evolution of Concepts; Actually, "Dr. Internet" Is the Name of the Monsters' Creator; Scientifiction and Fantastica; The Commonwealth of Letters