May 06, 2020

Data Over Space and Time

Collecting posts related to this course (36-3467/36-667).

Posted at May 06, 2020 03:23 | permanent link

May 05, 2020

How Not to Fit a Trend

If one of The Kids in Data Over Space and Time turned in something like this, I'd fail them ask where I'd gone wrong patiently talk them through all the reasons why blind, idiot curve-fitting is, in fact, idiotic, especially for extrapolating into the future. If they told me "well, I fit a cubic polynomial to the log of the series", we would go over why that is, still, blind, idiot curve-fitting.

library("covid19.analytics")
temp <- covid19.data("ts-deaths-US")
cu_deaths <- colSums(temp[,-(1:4)])
deaths <- c(0,diff(cu_deaths))
covid <- data.frame(cu_deaths,
                    deaths,
                    date=as.Date(names(cu_deaths)))
rownames(deaths) <- c()
plot(deaths ~ date, data=covid, type="l",
     lty="solid",
     ylim=c(0,3500),
     xlim=c(min(covid$date),
            as.Date("2020-08-04")),
     lwd=3)
start.date <- "2020-03-01"
working.data <- covid[covid$date>start.date,]
cubic <- lm(log(deaths) ~ poly(date, 3),
            data=working.data)
lines(x=working.data$date,
      y=exp(fitted(cubic)),
      col="red", lty="dashed", lwd=3)
future.dates <- seq(from=max(working.data$date),
                    to=as.Date("2020-08-04"),
                    by=1)
lines(x=future.dates,
      y=exp(predict(cubic,
                    newdata=data.frame(date=future.dates))),
      lty="dotted", col="pink", lwd=3)

Of course, by reinforcing one of the most basic lessons about time series, I'd evidently be depriving them of the chance to join the Council of Economic Advisors to the President of the United States:

(Source)

I had always imagined that if we fell to pieces, it would be because we did something clever but deeply unwise. It is very depressing to realize that we may well end ourselves through sheer incompetence.

The Continuing Crises; Enigmas of Chance

Posted at May 05, 2020 18:26 | permanent link

April 30, 2020

Books to Read While the Algae Grow in Your Fur, April 2020

Attention conservation notice: I have no taste, and no qualifications to opine on American civil rights law, literary criticism, the psychology of reading, the literary ambitions of Karl Marx, Islamic history, or even, really, mathematical models of epidemic disease.

Lisa Sattenspiel with Alun Lloyd, The Geographic Spread of Infectious Diseases: Models and Applications [JSTOR]
This is an fine shorter book (~300 pp.) on mathematical descriptions and models of how contagious diseases spread over space and time. It alternates between chapters which lay out classes of models and mathematical tools, and more empirical chapters which cover specific diseases. The models start with the basic, a-spatial SIR model and its variants (ch. 2), then model expansions which run an SIR model at each spatial location and couple them (ch. 4), network models (ch. 6), and approaches from geographers, emphasizing map-making and regression-style modeling (ch. 8), which is where I learned the most. The introductory chapters do a good job of laying out the basic concepts and behavior of epidemic models, so in principle no previous background is required to read this, beyond upper-level-undergrad or beginning-grad mathematical competence. (There are no proofs, and only really elementary derivations, but the reader is expected to be familiar with differential equations, basic probability, and the idea of an eigenvalue of a matrix.)
The applications chapters cover, in order, influenza (especially seasonal influenza but also the 20th century pandemics), measles, foot-and-mouth disease, and SARS. A nice feature of the latter two chapters is a careful, somewhat skeptical look at how much policy-makers relied on mathematical models, and the extent to which such reliance helped.
This was on my stack to read as preparation for revising the "Data Over Space and Time" course, but prioritized by recent events; recommended.
Peter Diggle, Kung-Yee Liang and Scott L. Zeger, Analysis of Longitudinal Data
A standard text on longitudinal data analysis for lo these many years now, and deservedly so. The following notes are based on (finally) reading my copy of the first, 1994 edition all the way through; I have not had the chance to read the second edition (2002, paperback 2013).
What statisticians (especially biostatisticians) call "longitudinal" data is basically what econometricians call "panel" data: there are a bunch of people (or animals or factories, generically "units"), and we collect the same information for each of them, but we do it repeatedly, over time. (Different units may be measured at different times, and not every variable may be measured for every unit at every time.) We typically assume no interaction between the units. In symbols, then, for unit $i$ at time $t_{ij}$ we measure $Y_{ij}$ (which may be a vector, even a vector with some missing coordinates), there are covariates $X_{ij}$ (which may be constant over time within a unit or not), and we assume independence between $Y_{i}$ and $Y_{j}$.
From the point of view of someone trained to approach everything as a regression of independent variables on dependent variables, here a regression of $Y_{ij}$ on $X_{ij}$, longitudinal data is a tremendous headache, because all the observations within a unit are dependent, even when we condition on the independent variable. In the simple linear-model situation, we'd write $Y_{ij} = \beta_0 + \beta \cdot X_{ij} + \epsilon_{ij}$, but with the caveat that $\epsilon_{ij}$ and $\epsilon_{ik}$ are not independent, but have some covariance. (Generalized linear models are also treated extensively here, to handle binary and count data.) This means that the usual formulas for standard errors, confidence intervals, etc., are all wrong. This point of view, which the book calls that of "marginal modeling", is the implied starting point for the reader, and I think it's fair to say it gets most of the attention. The key is to come up with estimates of the covariance of the $\epsilon$s, perhaps starting from a very rough "working" covariance model, or even from looking at the residuals of a model which ignores covariance in the first place, and then using weighted least squares and robust standard errors to improve the inferences. There's a chapter on why ANOVA is not enough, which I suspect arose from students wanting to know why they couldn't just do ANOVA for everything.
One source of within-unit covariance is "random effects". The simplest form of this replaces the model I wrote above with $Y_{ij} = \beta_0 + Z_i + \beta \cdot X_{ij} + \epsilon_{ij}$, where now the $\epsilon_{ij}$ are uncorrelated over time (as in ordinary regression), and $Z_i$ is a unit-specific random variable. This implies that a unit which is above the regression line at one time will tend to also be above the regression line at another, creating a constant-over-time covariance structure. Equivalently, each unit has its own intercept, but those intercepts are clustered around some over-all $\beta_0$. This simple idea can be elaborated into letting the regression coefficients in $\beta$ also fluctuate from unit to unit. The book gives a pretty thorough treatment of estimation and inference for these random-effect models for longitudinal data, but doesn't look (much) at how to combine random effects with correlations in the $\epsilon$s, or at model-checking.
The book also considers a third point of view, that of "transition modeling", which is the one I find most congenial, as someone brought up in the dynamical systems / stochastic process tradition. This is to see longitudinal data as a pile of short, independent time series, so the natural thing to do is to pool the information to get a better estimate of the process, and/or to estimate the latent state of individual trajectories. That is, we try to learn $P(Y_{i(j+1)}|Y_{ij}, X_{ij})$ (or condition on more history, as needed), and/or $P(S_{ij}|Y_{ij}, X_{ij})$ for some latent state $S$ that's reflected in the $Y$s. I think this is much more straightforward and meaningful than ad hoc covariance functions for the noise, but I also realize that's my version of "So, why does <your field> need a whole journal, anyway?".
(For linear models with Gaussian noise, there are ways of inter-relating marginal models, random effects models and transition modeling, but these correspondences break down in more general situations, and the book is careful to explain why.)
The implied reader here has a pretty firm grounding in linear regression modeling, and general undergrad-level probability and statistical inference, but no previous experience with time series. There are lots of real-data examples, mostly bio-statistical, where the modeling is always done in a way which tries to actually illuminate the scientific (or medical or agricultural) problem at hand. (This extends to a whole chapter on missing data, and the particular ways it goes missing in longitudinal data collection.) Doing so helps the reader implicitly learn better statistical craft than just just treating the data as piles of numbers to be fed into S. As the fact that I wrote "S" and not "R" in the last sentence indicates, the computing is now a bit antiquated, but replicating the analyses in R will build character.
Sherri Cook Woosley, Walking Through Fire
Mind candy. This is a well-written the-magic-returns-and-our-world-changes contemporary fantasy, clearly setting up what should ordinarily be an enjoyable series, but of course the pandemic happened in the middle of my reading it. Suddenly, my reaction a vision of the post-apocalyptic mid-Atlantic states being balkanized by squabbling Mesopotamian gods is less "I'm amused by what she's done to that all-too-familiar stretch of I-70" and more "Not now, Tiamat, not now". (Picked up on Walter Jon Williams's recommendation.)
Richard Thompson Ford, Rights Gone Wrong: How Law Corrupts the Struggle for Equality
It is, of course, beyond my competence to comment on Ford's interpretation of the law. But he has a lot of persuasive-to-me things to say about how approaching social injustice as a matter of violating individual rights is not always wise or effective, particularly when what's at stake is, say, a complicated, multi-causal pattern of differences in outcomes rather than outright bigotry: "Civil rights are remarkably effective against overt prejudice perpetrated by identifiable bigots. But they have proven impotent against today’s most severe social injustices, which involve covert and repressed prejudice or the innocent perpetuation of past prejudice."
Though Ford doesn't use this parable, many of my readers will understand if I put things in the following way. Back when the ink was still wet on the Civil Rights acts, Thomas Schelling developed a deservedly-famous little model of how a slight preference for similar neighbors --- really, not wanting to be in too small a minority locally --- would lead an integrated residential pattern to unravel into segregated regions*. (IIRC, Ford mentions this model in other books, but not here.) Now suppose we prepare the Schelling model in a state of enforced segregation, and then remove the constraint. We'll find some desegregation, but the dynamics will lead to there still being a lot of segregation. Ford sees that Schelling-world would have a real social problem, but doesn't see who could be usefully sued, or protested, to make it better.
Naturally, Ford is a lot sketchier about what would be a better way to make progress towards equality.
Ford's page for the book links to several excerpts.
*: Schelling, not being an idiot or an apologist, did not think that this was the main explanation of US residential segregation circa 1970: "the subject of this paper might be put in third place" behind "organized action" on behalf of segregation, and economic inequality between the races [Schelling 1971, pp. 144--145]. (In this he was wiser than some later, lesser economists.) But he was right to point to it as an obstacle to integration which would be there if and when "organized action" and economic inequality were eliminated. ("Still, in a matter as important as racial segregation in the United States, even third place deserves attention.") ^
D. J. Daley and J. Gani, Epidemic Modelling: An Introduction
Briskly written and now-classic first textbook on epidemic models, especially compartment-type models. I think it would still make a good introduction, and I plan to borrow from it freely the next time I teach these models. (The measles data sets!) But there's a lot more emphasis on getting exact or nearly-exact solutions, by heroic exercises in manipulating probability generating functions, Laplace transforms, etc., than I think is really warranted, or at least useful for my sort of audience. (*) And of course, being from 1999, a lot of interesting stuff has been done since!
*: It's also a little bit curious, because by 1960, Bartlett was very much an advocate of the power of simulations in this field, and they're certainly very familiar with his work. But perhaps my own feeling that if we can't get clean theoretical results then we might as well just simulate for all the insight approximate Laplace transforms give us reflects generational turn-over, or even just a personal lack of imagination on my part.
William Clare Roberts, Marx's Inferno: The Political Theory of Capital [JSTOR]
I'm not sure what to make of this one.
Roberts argues that Marx modeled the literary structure of Capital on Dante's Inferno, with the progression through the different kinds of sins in the Inferno mirroring the stages of Marx's exposition of how capitalism works and why it's so awful, from incontinence (because everyone is at the mercy of market forces, no one can really show steadiness of will) to force and fraud to betrayal. This seems pretty weak and forced to me. The best that Roberts can point to, by way of specific textual evidence, is Marx's general enthusiasm for Dante in his letters, and a handful of classical allusions in Capital, so this mostly rests on how convincing you find Roberts's parallels between the order in which Marx anatomizes capitalism, and the order in which Dante see sins being punished. These aren't the worst literary parallels I've seen, but aren't compelling either. The other unusual aspect of Roberts's Marx* is that he's a rather unorthodox republican, with a strong belief in freedom as non-domination. I actually find this more persuasive than the Dante, but I also wonder how much "republicanism", in this sense, was an actual historical phenomenon, as opposed to a late 20th/early 21st century ideology that's been projected back on to the past... A third aspect of Roberts's exposition of Capital is his emphasis on how Marx was arguing with followers of other early socialists (especially Proudhon and Owen), which leads Roberts to want to trace parts of Capital to Marx counting coup on his opponents by turning their rhetoric or ideas against them. (Roberts also keeps sniping, from his footnotes, at Jon Elster and G. A. Cohen, but sniping footnotes are if anything a homage to Uncle Karl.)
I can't say I found this persuasive (though, really, I don't know enough to judge), but Roberts's Marx is certainly an interesting construction, and I did learn a lot about 19th century socialists other than Marx.
--- Reading this leads me to some doubtless-trite reflections on difficulties --- you might even say "contradictions" --- inherent in advancing a novel interpretation of a work which is already well-known and influential. Precisely because it's a novel interpretation, you have to be saying that everybody else has been mis-understanding the book all this time. This in turn implies that (A) the book as has been influential because of a mis-interpretation, and (B) that the author failed to communicate. Both of which could be true. But then it's really the mis-reading (or mis-readings) which have been influential, not the correctly-understood work. And even great authors are certainly capable of failing to get their point across**, just like other writers. But surely both (A) and (B) should be seen as implausible, precisely because the work in question is well-known and influential, and they should become more and more implausible over time. So one would need either extremely persuasive evidence for the new interpretation in its own right, or to make special cases for (A) and/or (B) themselves. I guess there would be an exception for a claim to be re-discovering a correct interpretation, which used to be common but has been lost, but then I'd want to see evidence that the book did, in fact, use to be interpreted that way, and ideally an explanation for how that knowledge came to be lost. (To be clear, Roberts is not saying that everyone used to realize Capital was modeled on the Inferno, etc.)
*: Roberts also offers a new-to-me twist on the sense in which Marx held a labor theory of value. This turns on the "socially necessary" part of the value of a commodity being the labor time socially necessary to produce it. For Marx, according to Roberts, there literally was no way of determining how much labor was socially necessary to produce a commodity, other than to see what value it could realize in exchange. Roberts's thought is that if (say) there is a glut of abaci, so that each abacus fetches fewer yards of linen, coats, bottles of whiskey, Bibles, etc., than before, that is what tells us that not all of the labor which went in to making abaci was in fact socially necessary. Whatever its merits as a theory of value in its own right, I have a hard time squaring this with what Marx says about how supply and demand make prices fluctuate around values, and indeed the general tenor of the texts. But I lack the time, and the will-power, to examine all the passages Roberts cites in support of this, compare them with all the others which might seem to point in other directions, and figure out what Marx had in mind. I.e., I am not a scholar of Marx, and didn't I warn you that you were probably wasting your time reading this? ^
**: For instance, since Marx's discussion of "the fetishism of commodities" in Capital has perplexed readers, and divided interpreters, essentially ever since it was published, I think we can agree that whatever he had in mind, he failed to get it across clearly. My hunch is that he hit upon a kind of murkiness here which is actually attractive, and keeps people coming back to this bit, feeling that there must be a way to make sense of it. In my more cynical moments, I wonder if Marx really had any clear idea in mind here. ^
David Koepp, Cold Storage
Mind candy: parasite porn thriller. Engaging, but there are bits where I found myself rooting for the parasite.
Anna Lee Huber, A Grave Matter and As Death Draws Near
Mind-candy historical mysteries. The fact that I tracked down volumes 3 and 5 of a series I've been reading out of order should tell you that I enjoy them... (Previously.)
John Billheimer, Dismal Mountain and Drybone Hollow
Mind-candy mysteries, in which a consulting transport planner returns to his home in rural West Virginia to deal with family and rural decay. (Previously.)
Barbara Paul, Your Eyelids Are Growing Heavy
Mind candy: Pittsburgh mystery from 1981, which begins with the matter-of-factly unflappable heroine waking up on a local golf course with no memory of how she got there, and builds to a complicated and fast-moving story. I admit I got an extra kick from the fact that it takes place in, largely, neighborhoods I know very well --- I walk over that golf course to work, have shopped at the fancy grocery store around the corner from the heroine's apartment, etc. --- but I think it'd be enjoyable even if you didn't care about Pittsburgh.
A. E. Stallings, Archaic Smile
An early (1999) collection by a favorite poet, saved up for an April. It turns out to be highly preoccupied with death and the underworld:
"Watching the Vulture at the Road Kill"

You know Death by his leisure --- take
The time we saw the vulure make
His slow, hot-air-balloon descent
To a possum smashed beside the pavement.
We stopped the car to watch. Too close.
He bounced his moon-walk bounce and rose
With a shrug up to the kudzu sleeve
Of a pine, to wait for us to leave.
What else can afford to linger?
The eagle has his trigger-finger,
Quails and doves their shell-shocked nerves---
There is no peace but scavengers.

Richard W. Bulliet, Conversion to Islam in the Medieval Period: An Essay in Quantitative History [ACLS Humanities E-Books] and Islam: The View from the Edge
The first of these books is an ingenious undertaking in using historical sources to shed light on something they weren't meant for. The second book takes the conclusions of the first as the starting point for an interesting revision of conventional views of Islamic history.
First book first. Biographical dictionaries were, interestingly enough, a widely-cultivated genre in many pre-modern Islamic countries. Each of these would give a large number of biographies of people (usually men) who were eminent or remarkable in some respect, such as religious scholars in a particular city or district. Part of those biographies would of course be the full name of the figure, which, in Arabic, included a chain of patronymics, of the form "A, the father of Z, the son of B, the son of C, ... the son of M". (Thus the pioneering social scientist generally known in English as "ibn Khaldun" was more properly 'Abd-ar-Rahmân Abû Zayd ibn Muhammad ibn Muhammad ibn Khaldûn, 'Abd-ar--Rahmân, the father of Zayd, the son of Muhammad, the son of Muhammad, the son of Khaldûn.) Bulliet's starting point is to suppose that the first appearance of an Arabic and especially of a Quranic name in this chain marks the family's conversion to Islam. (As a Persianist, he knows very well that later Muslims had little compunction about naming their children, say, Rustam or Tahmina.) Since the biographical dictionaries give dates, at least dates of death, with some reassonable-sounding guesses about the length of generations, typical age of conversion, and so on, this lets him work backwards to the approximate date of conversion of each patrilineage. (Since the same biographical dictionary will often contain multiple entries for relatives, one would want to (i) cross-check conversion dates, and (ii) only count each conversion once; I believe Bulliet does so but I can't find the passage.)
You can then chart the fraction of all known conversions which took place in a given time window. This is what Bulliet gets when he does this for "Iran", i.e., all Persian-speaking areas:

This looks even more impressive when plotted cumulatively:

(Click on either image to embiggen.)
As Bulliet says, this is a textbook-quality fit to a logistic curve, which is exactly what you'd expect from the simplest diffusion-of-innovations model. (In terms of epidemic models, an SI model.) I am legitimately impressed, and will certainly use this the text time I teach that subject. (Though I have also acquired enough of the statistician's carping, quibbling spirit that I'd want to see how much different choices about things like generation time would alter his results.)
Bulliet goes on to construct similar curves for other parts of the Muslim world --- Spain, Egypt, Syria, Iraq --- but admits that the data aren't as great, partly because of the quality of the sources, and partly because it's harder to use linguistic clues to identify conversions in those regions. (As he says, Spain is an exception here.) These are broadly similar in their shape, though timing is a bit different from one place to another.
The historical interpretation Bulliet builds on these findings is given briefly in Conversion, and considerably more elaborated in View from the Edge. This is that the Muslim community in fact grew fairly slowly at first, even when large territories had been incorporated into the caliphate. The new Muslims, he argues, tended to move to cities which were sites (initially) of Muslim garrisons (sometimes being founded as such), creating an unusually urban religious environment. Figuring out what it meant to be a Muslim, in this environment, was largely regulated by the hadith, the traditions concerning the Prophet and his companions, which became central to Islamic education and religion. View gives a very detailed account of how the hadith were transmitted and used, how both transmission and use changed over time, and how they came to be compiled in books rather than in oral tradition --- a change, he notes, that seems to have first taken root in Iran. Lots of the customs and institutions that became broadly characteristic of Muslim societies were, on Bulliet's account, products of "the edge", these frontiers of conversion, and especially Iran. These began to spread more broadly, he says, only after the process of conversion in Iran was largely complete, on the far side of the century or so when most of the conversions happened. Thus for instance the earliest madrassas founded outside Iran, for instance, were heavily loaded towards having Iranian professors, or non-Iranians who were themselves the immediate students of Iranian scholars.
Of all this, I will just say that it sounds very plausible, but it's far beyond my competence to evaluate.
Andrew Elfenbein, The Gist of Reading
An attempt to synthesize what psychologists know about how people read today, and especially what they remember of what they read, with literary-critical concerns about how they should read, and literary-historical interests in how people have read. As an outsider to all the disciplines involved, I found it fascinating, and a model of engaged, modest interdisciplinarity.

Books to Read While the Algae Grow in Your Fur; Pleasures of Detection, Portraits of Crime; Enigmas of Chance; Data Over Space and Time; Writing for Antiquity Islam; Biology; Scientifiction and Fantastica; The Commonwealth of Letters; The Progressive Forces; Tales of Our Ancestors; The Beloved Republic; Heard About Pittsburgh PA; Minds, Brains, and Neurons; Philosophy; Commit a Social Science

Posted at April 30, 2020 23:59 | permanent link

April 22, 2020

A Very Introductory Lecture on Epidemic Models

Attention conservation notice: Should you really be learning about epidemiology from someone who has to consciously inhibit an urge to turn everything into an Ising model?

I'm teaching data mining this semester, but The Kids were understandably interested in epidemic modeling, so I gave a lecture on it for the special-topics day in the syllabus. They liked the slides, and some people who saw them did to, so if you want a basic introduction the classic susceptible-infectious-removed model of epidemics, and how it plays with network structure, here you go.

Suggestions for improving them are welcome, since it seems very likely that I'll be teaching "data over space and time" again in the fall (assuming, inshallah, that there is a fall semester), and it'd be strange not to cover this topic.

Corrupting the Young; Enigmas of Chance; Networks; Biology

Posted at April 22, 2020 21:26 | permanent link

March 31, 2020

Books to Read While the Algae Grow in Your Fur, March 2020

Attention conservation notice: I have no taste, and no qualifications to opine about mathematical biology, the history of science, or comparative sociology and political science.

Jane Haddam, Fighting Chance
Mind candy mystery novel: the 29th, and sadly last, of her Gregor Demarkian series. This is the only one where the solution, while logical, "fair" and completely unexpected, did not feel right. (I refrain from saying any more to avoid spoilers.) It is also, as this profile notes, one where the background crime, outrageous though it sounds, was entirely real, and it's astonishing that it did not end in murder.
Nicolas Bacaër, A Short History of Mathematical Population Dynamics
This is very much a scientist's history of science, rather than a historian's. It consists of a series of very brief chapters, most of just a few pages, giving thumbnail biographies of historical figures, and explaining their contributions in modern terminology and notation. It begins with Fibonacci and his series, noting that this had absolutely no influence on any later work at using mathematics to understand population. The real development begins with stuff like life tables and actuarial calculations, Malthus and population growth, and so on. The re-appearance of the basic reproductive number in numerous contexts is a running, though not exactly high-lighted, theme. Developments in data collection (e.g., comprehensive censuses by states effective enough to actually count people, ascertain their ages, etc.) are mentioned only in passing, though without such data there would be nothing to model. The most fully developed historical study is actually one of the last, in the chapter on China's one child policy, in which a bunch of control engineers, cut off from the broader scientific community, manage to re-invent key ideas of demography, derive radical conclusions from their models, and get men in power to act on them.
I learned from this, I appreciate its perspective and its brevity, and I'd assign it to my students if they were curious, but I'd also like to see something more historically serious.
Meg Gardiner, UNSUB, Into the Black Nowhere, The Dark Corners of the Night
Mind-candy thrillers, psycho-killers-and-profilers flavored. Competitive with Shadow Unit as high-quality Criminal Minds pastisches.
Brian D. Ripley, Spatial Statistics
On the one hand, this is from 1981, so all the detailed computational advice is laughably obsolete. (At one point, Ripley discusses strategies for not having to keep all of a 128 kb image in main memory at once.) There has also been a lot of advances in some aspects of the theory, notably point processes. On the other hand, Ripley's basic advice --- visualize; do less testing for "randomness" and more model-building; simulate your models, visualize the simulations, and test modeling assumptions with simulations and visualizations; smooth, and remember that "kriging" is just the Wiener filter --- remains eminently sound.
--- I have been reading bits and pieces of this book, off and on, since around 2000, but I have a rule about not recommending something until I've finished it completely. Having finally now read it all, including the chapter on tomography (!), I can safely say: anyone seriously interested in spatial statistics probably ought to read this, but you can skip the tomography chapter as obsolete. I have to say that the idea of paying the list price for the paperback is outrageous, but lots of potential readers will have access to Wiley's online version, which is a perfectly decent scan of the printed book.
Victor LaValle, The Changeling
Magical-realist urban fantasy, about being a parent in contemporary New York. It's intense, but it's great because it honors the genre conventions of both literary fiction (the attention to character; language that tries to renew the perception of the ordinary world) and urban fantasy or horror (creepy, thrilling supernatural weirdness).
Göran Therborn, What Does the Ruling Class Do When It Rules?
A collection of long, tightly-linked essays from the 1970s, attempting to formulate a serious Marxist sociology of the state and of political authority. The starting point is his answer to the title question, which is, roughly, that the ruling class, through the state, acts to reproduce the social conditions for its own dominance. This basic thought is then elaborated with a wealth of historical and sociological examples, and a lot of intelligence given to ideas like different modes of state organization being different forms of technology, and different "formats of representation".
Therborn's over-all answer is a reasonable one, but not without its difficulties. There are two which especially struck me while reading, one of which he could have fixed fairly easily (intellectually if not personally), the other, I think, is more fundamental.
  1. Therborn quite properly includes the post-1917 post-1917 Communist states as examples of states with a definite class character, and analyzes how they acted to reinforce the dominance of the ruling class. These analyses are, however, completely undermined by his conviction that the ruling class in those states was the proletariat, which collectively appropriated the surplus for its own use. This was, of course, nonsense, and was easily observed to be nonsense at the time. Therborn even admits that management, Party officials, etc., are not exactly the proletariat and look an awful lot like a ruling class, but insists that they weren't, all appearances and his own criteria to the contrary notwithstanding. A more clear-eyed advocate of Therborn's own theoretical positions should, I think, have frankly admitted that the Party and the technocracy together constituted a new ruling class, engaged in collective expropriation of the surplus for its benefit and in ensuring the reproduction of its own conditions of domination. In other words, Therborn should have endorsed the thesis of Djilas's The New Class. Reading between the lines, I'm sure this would have been personally unacceptable to Therborn, at least at the time of writing, but it's where his logic should have taken him.
  2. The more serious problem, to my eyes, is the cognitive one. Therborn assumes that the ruling class knows how to reproduce its rule, but in reality it would need to figure out how to do this. Even if we suppose that the ruling class (and/or its state) wants to advance its interests, it's not obvious what concrete courses of action will do that, let alone do it well. Presumably it doesn't wait for Comrade Therborn to come along and tell it, and presumably it doesn't secretly employ historical materialism in its innermost councils, so what does it do? (Cf. Dewey.) How do these ideas about what the ruling class and its state should do become effective for real, concrete individuals? What mechanisms keep those individuals in line with class interest? What happens when the ruling class and/or the state are wrong about what will advance their interests? I am tempted, on Therborn's behalf, to advance a selectionist explanation: ruling classes don't have Marxometers pointing them in the right direction, so they try all kinds of things, but the ones which are wrong about how to secure the reproduction of their domination will cease to rule, so we'll only observe ones which were right. (Cf. Quine's "Creatures inveterately wrong in their inductions have a pathetic but praiseworthy tendency to die before reproducing their kind".) But changes in ruling classes are rare, so there'd be little evolutionary information in the signal of their failures (as in footnote 2 to my critique of Ober), so there would need to be some kind of selection at the level of practices or policies, and I don't see the feedback channel which would tell the state apparatus "stop doing that, it's not helping the dominant class reproduce itself". (There are plenty of other feedback channels.) In any case, such selectionism is entirely against the "feel" of Therborn's mode of argument.
To repeat, there's a lot of interesting historical and sociological material in here, arranged by an intelligent and honest partisan in support of some fixed convictions. I'd be very curious to see how Therborn's thoughts have moved on since 1978. (This edition is a 2008 reprint without any new material.)
Chelsea Cain et al., Man-Eaters (1, 2, 3)
Warren Ellis et al., The Wild Storm (1, 2, 3, 4)
Victor LaValle et al., Destroyer
Comic-book mind candy. Man-Eaters is a very broad satire about what would happen if the onset of menses caused teenage girls to turn into literal man-eating were-panthers. The Wild Storm is Ellis giving a familiar kaleidoscope (ancient aliens, secret space programs and hidden governments, personified electricity as the animating spirit of the 20th century) another shake. Destroyer melds some of the concerns about black parentage LaValle explores in The Changeling with a sequel to Frankenstein.
The Americans
A serial about a family of prosperous, hard-working immigrants raising kids in the DC suburbs in the 1980s, with complicated feelings about America and the Americanization of their children, might as well have been targeted at me.

Books to Read While the Algae Grow in Your Fur; Scientifiction and Fantastica; Pleasures of Detection, Portraits of Crime; Enigmas of Chance; Commit a Social Science; The Progressive Forces; Writing for Antiquity; Biology; Data over Space and Time

Posted at March 31, 2020 23:59 | permanent link

March 30, 2020

Of the Evaluation of Expertise ("I am not so good for that as an old roofer")

Attention conservation notice: A 2000-word reaction to conferences in 2011 where too many people wanted impossible things of social science and data mining, and too many people seemed eager to offer those impossible things. Too long by more than half, too pleased with itself by much more than half, it lacks constructive suggestions and even a proper ending. To the extent there's any value to these ideas, you'd be better off getting them from the source I am merely parroting. Left to gather dust in my drafts folder for years, posted now for lack of new content.

Q: You have an old house with a slate roof, right?

A: You know that perfectly well.

Q: Does the roof ever need work?

A: You just said it's old and slate, and I live in Appalachia. (In the Paris of Appalachia; but still, in Appalachia.) Of course it does.

Q: Do you do the work yourself?

A: I have no idea how; I hire a roofer.

Q: How do you know the roofer knows what he's doing?

A: I am not sure what you mean. He fixes my roof.

Q: Well, does he accurately estimate where leaks will occur over the next year?

A: No.

Q: Does he accurately estimate how much the roof is going to leak each year?

A: No.

Q: Does he accurately estimate how many slate tiles will crack and "need replaced" each winter?

A: No.

Q: OK, so he's not much into point forecasts, I can get behind that. Does he give you probability forecasts of any of these? If so, are they properly calibrated?

A: No. I don't see where this is going.

Q: Everyone agrees that the ability to predict is a fundamental sign of scientific and technological knowledge. It sounds like your roofer can't predict much of anything, so what can they know? You should really hire someone else, preferably someone well-calibrated. Does Angie's list provide roofers' Brier scores? If not, why not?

A: I don't believe they do, I can't see why they should, and I really can't see how knowing that help me pick a better roofer.

Q: It's your business if you want to be profligate, but wouldn't it help people who do not enjoy wasting money to know whether supposed experts actually deserve to be taken seriously?

A: Well, yes, but if you will only listen to roofers who are also soothsayers, I foresee an endless succession of buckets under leaking ceilings.

Q: So you maintain a competent (never mind expert) roofer needn't be able to predict what will happen to your roof, not even probabilistically?

A: I do so maintain it.

Q: And how do you defend such an obscurantist opinion? Do you suppose that a good roofer is one who enters into a sympathetic human understanding with the top of the house, and can convey the meaning of a slate or a gutter?

A: Humanist-baiting is cheap even for you. No, when it comes to roofs, I am all about explanation, and to hell with understanding. (People are different.) But an expert roofer no more needs to predict what happens to the roof than an expert engineer needs to predict how a machine they have designed and built will behave. Indeed, it would be a bizarre miracle if they could make such predictions.

Q: And why would a plain, straightforward prediction be such a wonder?

A: However expert the roofer is about the roof, or the engineer about the machine, what happens to them depends not just on the object itself, but the big, uncontrolled environment in which it's embedded. You will allow, I hope, that what happens to my roof depends on how much rain we get, how much snow, etc.?

Q: Not being a roofer, I don't really know, but that sounds reasonable.

A: Might it not also matter how many sunny days with freezing nights we have, turning snow on the roof to ice?

Q: Sure.

A: And so on, through contingencies I'm too impatient, and ignorant, to run through. But then, to predict damage to the roof, doesn't the roofer need to not only know what condition it started in, but also all the insults it will be subjected to?

Q: That does seem reasonable. (But aren't you asking a lot of questions for "A"?)

A: (Shut up, I explain.) So your soothsaying roofer must be a weather-prophet, as well as knowing about roofs. And the same with the engineer and their machine: they would need to foresee not just the environment in which it will be put, but also the demands which its users will place upon it. It sounds very strange to say that such prophetic capacities are a necessary part of expertise in roofing, or even engineering.

Q: It might sound strange; many true things sound strange; indeed, doesn't every science become more and more un-intuitive and strange-sounding as it progresses? It sounds strange to lay-people to say that the economy consists of one immortal, lazy, greedy, infinitely calculating person, who does all the work, owns all the assets, and consumes all that the goods, but just think of the triumphs of successful prediction which the macroeconomists have achieved on this basis! If we abandon the criterion of prediction just because it sounds strange, how shall we ever distinguish an expert roofer from a mere pundit of the slates?

A: Perhaps by seeing if they can do the things roofers are supposed to do.

Q: Such as?

A: Well, when something goes wrong with the roof, they should be able to diagnose what caused the problem, and in favorable cases prescribe a course of action which will fix it, and even carry out the operation.

Q: Do you only call in the roofer when something has gone wrong? Your house must be in a sad state if so.

A: No, a good roofer should also be able to diagnose situations which, while they cause no immediate problem, are apt to lead to problems later.

Q: There can be few of those if you should have a winter without rain, which, you must admit, is possible. Are you not just sneaking back in my probabilistic forecasts, which you poured such scorn on before, with your "sooth-sayers" and your "weather prophets"?

A: Not at all; it's enough to recognize conditions which would cause problems under a broad range of circumstances which there is some reason to fear. Apprehension about the effects of a meter of snow sitting atop the roofer for three months on end is not reasonable in Pittsburgh; expecting even ten days in the winter without some precipitation is folly. At the very most, I am calling for some ability to say, conditional on typical weather, what consequences should be expected; the problem of giving a distribution for the weather is no part of the roofer's expertise.

Q: And next I suppose that you will pretend that prescription doesn't rest on prediction?

A: It certainly requires knowledge of the form If we do X, then condition Y will result; but if we do X', then Y'. Such conditional knowledge, about roofs, is actually immensely easier for a roofer to acquire, than for them to learn the whole huge multidimensional distribution of all the environmental factors which could influence the state of a roof, and their dependencies over time. But it is the latter which you are presuming, with your Brier scores and your notion of what sort of prediction is an adequate sign of knowledge. And what the roofer cannot foretell about the roof, neither can the engineer about the machine, nor the doctor about the patient, nor the natural scientist about their object of study.

Q: You insist that scientists do not make predictions?

A: In your unconditional, categorical, absolute sense, certainly not. Scientists certainly possess all sorts of conditional knowledge, about what would happen, if certain conditions were to be imposed, certain manipulations were to be made. Unconditional predictions, even unconditional probabilistic predictions, are for the most part beyond them.

Q: So a chemist cannot predict the course of a chemical reaction? Don't bother dodging with contaminants, or mis-labeled reagents.

A: Me, quibble? No. But even with pure, known reagents in a sealed reaction vessel at STP --- well, you were a student at Berkeley, where the chemistry labs are a block or two downhill of the Hayward Fault. I don't think you could have said what would have happened if there'd been a tremor in the middle of one of your experiments.

Q: You can't think it's fair to ask a chemist to predict earthquakes, can you?

A: My point exactly. To put it formally, in terms of Pearl, you could know \( \Pr\left(Y\middle |\mathrm{do}(X=x)\right) \) exactly, for all \( x \), but not know \( \Pr\left(Y\middle|X=x\right) \), still less \( \Pr(Y) \). (The Old Masters of Pittsburgh would say: you can know the distribution of \( Y \) after "surgery" to remove incoming edges to \( X \), without knowing the un-manipulated distribution of \( Y \).) But it is the last which you are insisting on, with your calibrated forecasts and prediction as the sign of expertise.

Q: Well, astronomers make such predictions, don't they?

A: I defy you to find a single other science which can also do so. Even then, our astronomy's successes merely testify to our engineering's weaknesses. When our descendants (or the cockroaches'; no matter) become able to move around comets, or move planets, or build Dyson spheres and Shkadov thrusters, even the predictions of celestial mechanics will become contingent on the interventions of sentient (I will not say "human") beings.

Q: Even in those science-fictional scenarios, the choices of human (or post-human or non-human) beings would be functions of their microscopic molecular state, and so physically predictable, so shouldn't —

A: I indulged in science fiction as a rhetorical flourish, but now you are seriously arguing on the basis of technical impossibilities, sheer metaphysical conjecture and even mythology.

Q: Never mind then. Suppose, for the sake of argument, that I accept that someone could be a knowledgeable expert without making predictions. Surely you would agree, though, that someone who makes a lot of predictions which turn out to be wrong definitely doesn't know what they're doing?

A: I can think of at least one way in which that fails, which is that when they also have effective control.

Q: Better control means un-predictability? This I have to hear.

A: Suppose I want to maintain a constant temperature in my house. I look at the sky, guess the day's weather, and turn on the air conditioner or the furnace as needed. If I express myself by saying "It's miserably humid and the sun isn't so much shining as pounding, the house will be intolerably hot", you will point out, at the end of a day during which the air conditioner labored heroically and the electric bill mounted shockingly, that the house was in fact entirely comfortable, and say not only that I can't predict anything worthwhile, but that, empirically, you see no particular relationship between the weather outside the house and the temperature inside it.

Q: Aren't you creating an air of paradox, simply by being sloppy in expressing the prediction? It's really "The house will be intolerably hot today, unless I run the AC".

A: Granted, but we can only ever check one branch of the condition. And I can be completely accurate in predictions about what would happen, absent imposed controls, even if none of those things actually happen, because of my forecast-based control.

--- I never did figure out how to end this.

Of the Evaluation of Expertise ("I am not so good for that as an old roofer"); Commit a Social Science; Constant Conjunction Necessary Connexion

Posted at March 30, 2020 09:50 | permanent link

March 10, 2020

Ebola, and Mongol Modernity

Attention conservation notice: An old course slide deck, turned in to prose on the occasion of a vaguely-related news story from 2014. Not posted at the time because it felt over-dramatic. I have, of course, no authority to opine about either world history or epidemiology, and for that matter no formal training in networks.

Exhibit A

One of the books which most re-arranged my vision of the past was Janet Abu-Lughod's Before European Hegemony: The World System A.D. 1250--1350. It gave me the sense, as few other things have, of historical contingency, or more exactly of modernity as a belated phenomenon, and changed my teaching. She depicts an integrated (part-of-the-) world economy, an "archipelago of towns" linked by trade-routes stretching from Flanders to Hangzhou and centered in the Indian Ocean. This archipelago is where modernity should have begun. Beyond the market-oriented, urban-centered economy, China has the beginnings of an industrial revolution (a point explored by Mark Elvin in his Pattern of the Chinese Past, and his sources in Japanese scholars of Chinese economic history, and emphasized by William McNeill in his Pursuit of Power); the beginnings of a truly global perspective. All of this was politically supported by the unification of the most economically and technologically advanced regions (namely China and the Islamic world) under the Mongol Empire, admittedly at the cost of the occasional "shock and awe" campaign, destruction of Baghdad, etc.

Exhibit B

So what, according to Abu-Lughod, happened? What happened was Yersinia pestis, the bubonic plague, a bacterium transmitted by fleas that live on rodents. It long has been, and is, endemic to the rodents of Central Asia, such as the giant gerbil {\em Rhombomys opimus}, which seems to be perpetually perched at the edge of the epidemic threshold. The Mongol Empire didn't just unify the most advanced parts of Eurasiafrica; it brought them into intimate contact with Central Asia. And then, as usual, the plague followed the routes of trade and imperial travel:

It's impossible to know just how deadly it was, but estimates put it at around 25% of global population; up to 90% in some regions. It destroyed (according to Abu-Lughod) the world economy, that "archipelago of towns", leaving isolated and barely-functional fragments which could be dominated and re-purposed by western European pirates/traders poking into the post-pandemic landscape.

One aspect to this is how slowly, and how progressively, the plague spread. It took decades to spread from Central Asia to the peripheral region of western Europe, where it chewed steadily across the landscape:

As my old friend and sometime co-author Mark Newman and collaborators puts it, this is strong evidence that "the small-world effect is a modern phenomenon". The small-world effect, after all, is that the maximum distance between any two people in the social network of size $n$ grows like $O(\log{n})$. This implies that the number of people reachable in $d$ steps grows exponentially with $d$, which is hardly compatible with the steady geographic progress of the disease.

The argument here is so pretty that I can't resist sketching it. Suppose that every infected person passes it on to any one of their contacts with probability $t$, at least on average. We start the infection at a random person, say Irene, who selects a random one of their acquaintances, say Joey, for passing it on. The probability that Irene, or any other random person, has $k$ contacts is, by convention, $p(k)$. But Joey isn't a random person; Joey is someone reachable by following an edge in the social network. Joey's degree distribution is $\propto k p(k)$, since people with more contacts are more reachable. Specifically, Joey's degree distribution is $k p(k) / \langle K \rangle$, where $\langle K \rangle \equiv \sum_{k=0}^{\infty}{k p(k)}$, the average degree. If Joey gets infected, the number of additional infections he could create is up to $k-1$. So our initial random infection of Irene creates, on average, $t \sum_{k=1}^{\infty}{k(k-1)p(k)}/\langle K \rangle = t \langle K^2 - K\rangle / \langle K \rangle$ at one step away. At $d$ steps, we've reached $ \left(t \langle K^2 - K\rangle / \langle K \rangle \right)^d$ nodes. So long as $t > \langle K \rangle / \langle K^2 - K\rangle$, this is exponential growth:

If $t$ is above this critical level, the only way to avoid exponential growth is to have lots and lots of overlapping routes to the same nodes. [Some people might've had 1024 ancestors ten generations back, but most of us didn't.] Geographic clustering will do this, but the small-world effect makes it very hard to avoid. And the small-world effect is very much a part of modernity; whether the diameter of the social network is six or twelve or twenty is secondary to the fact that it's not a thousand.

Exhibit C

Zeynep Tufekci, "The Real Reason Everyone Should Panic: Our Global Institutions are Broken" (23 October 2014):
The conventional (smart) wisdom is that we should not panic about Ebola in the United States (or Europe). That is certainly true because, even with its huge warts, US and European health-care systems are well-equipped to handle the few cases of Ebola that might pop up.
However, we should panic. We should panic at the lack of care and concern we are showing about the epidemic where it is truly ravaging; we should panic at the lack of global foresight in not containing this epidemic, now, the only time it can be fully contained; and we should panic about what this reveals about how ineffective our global decision-making infrastructure has become. Containing Ebola is a no-brainer, and not that expensive. If we fail at this, when we know exactly what to do, how are we going to tackle the really complex problems we face?
Climate Change? Resource depletion? Other pandemics?
So, I have been panicking. ...
Globalization, in essence, means we really are one big family, in sickness and in health.
The more connected we are, the easier it is for a virus to spread wide and deep, before we get a chance to contain it.
And that is partially why Ebola is now ravaging through three countries in West Africa: it broke through in cities and large-enough settlements, and due to an accumulation of reasons, including recent civil wars, at a time when they were least equipped to handle it.
Containing an outbreak requires circumscribing the outbreak (isolating and treating the ill, tracing their contacts, isolating and treating them as well) so that it can no longer find new hosts, and healing those who are ill, or mourning those who die. Circumscribing an outbreak is easier when the cases are a few, or a few dozen, or a few hundred.
In fact, we know from previous Ebola outbreaks which parameter brings down the dreaded transmission rate: “the rapid institution of control measures.” It’s that simple.
After thousands of cases, this gets harder and harder.
After millions, it is practically impossible.

Of course, Ebola got under control (in 2014). It took far too much misery and fear and time, but it happened. But it left no sign that the powers that be had learned any lessons, about (to quote Tufekci again) either "basic math [or] basic humanity".

The Mongols, at least, had the excuse that they had no idea what they were doing. (It's not as though Nasir al-din al-Tusi had, between doing theology and pioneering Fourier analysis, worked out how network connectivity related to the likelihood of epidemic outbreaks.) Compared to them, our predecessors in globalization, we are as gods; we're just not very good at it.

Networks; Writing for Antiquity; The Continuing Crises; The Great Transformation; The Natural Science of the Human Species

Posted at March 10, 2020 23:46 | permanent link

February 29, 2020

Books to Read While the Algae Grow in Your Fur, February 2020

Attention conservation notice: I have no taste, and no qualifications to opine about epidemiology, sociology, or the history of ideas. Also, I'm writing book reviews during faculty meetings downtime.

István Z. Kiss, Joel C. Miller and Péter L. Simon, Mathematics of Epidemics on Networks: From Exact to Approximate Models
My attempt at a summary grew to a full-length review (which needs a better title).
Erik Olin Wright, How to Be an Anticapitalist in the Twenty-First Century
This is Wright's last book; it continues a long and (unironically) proud tradition of socialists re-writing the Manifesto: start by saying why capitalism sucks, while admitting some of its virtues; then explain how it can be bettered; identify the existing social force(s) which will replace it; and talk about how those forces will undertake the revolution. Wright says sound things about how capitalism molds people into selfish jerks (or crushes them), is undemocratic and (for most) unfree, and offends basic notions of fairness. (He sensibly refrains from asserting that capitalism either causes or requires racism, sexism, imperialism, etc., which is at the very least a highly debatable generalization, though of course racist and sexist capitalists can be expected to exploit workers in racist and sexist ways on top of everything else.) He then sketches what he'd like to see instead, which is a market economy with lots of public provision, some collective ownership, and a lot of worker and consumer cooperatives. Fantasies of central planning are, rightly, not part of his socialist vision. (He does not touch on the delicate problem of how to coordinate the democratic decisions of the members of a cooperative with the democratic decisions of the wider socialist commonwealth when the two disagree [e.g., about whether or not to shut down an oil company], or how to delineate the right scale for the cooperatives [is the oil company one cooperative, or does each rig, refinery and gas station become its own cooperative?].) He also delineates different ways of attacking or at least trying to replace capitalism, ranging from frontal assaults by violent revolution to separatist utopian communities to temporary carnivals of defiance to quietly trying to build alternative institutions that can grow to take over the larger capitalist ecosystem, a sort of vision of socialism as algal bloom. (That is not his image.) The end of the book looks at what would be required for a "collective actor" to try to effect such a transformation --- and there it ends, with a promise that he was just about to say how the trick would be turned.
We will never get to hear Wright's thoughts about how to solve that last riddle, because he died while the book was still incomplete, and what we have here was polished to publication, but not exactly completed, by friends and disciples. This is a fitting tribute to a scholar of real distinction, who made his reputation by combining sound* sociology with unorthodox, "analytical" Marxism that wasn't afraid to actually think, and who tried to remain connected to real-life struggles for a better world.
*: More waspishly, no worse, methodologically, than the rest of post-1960s American sociology.
Jane Langton, The Memorial Hall Murder, Natural Enemy, Good and Dead, Murder at the Gardner
Classic mysteries from the early 1980s, where the very particular settings, and often specific works of literature or art associated with them, are just as important as the murders. I read all these as a boy, but return to them now with delight, and perhaps more appreciation for their non-murder-mystery aspects. (For instance, Good and Dead is also a fine novel about the decline of "mainline" Protestantism, which rather passed me by as a teenager.)
Kwame Anthony Appiah, Lines of Descent: W. E. B. Du Bois and the Emergence of Identity
A fine study of what Du Bois took from German thinkers of the 18th and 19th centuries, especially from German Romanticism and from the "professorial socialism" of his teachers in Berlin. It'd probably help to have at least read The Souls of Black Folk (which Appiah suggests we could gloss as "the geist of the black volk"), but Appiah's exposition is so skillful that no deep knowledge of either Du Bois's life and work, or of German thought, is really needed.
--- There is a longer story here, which Appiah hints at, at how relevant early Romantic nationalism (like Herder) remains for contemporary ideas in fields like "ethnic studies", despite gestures towards acknowledging the internal heterogeneity and diversity within any such "community". (It's easy to see how a Herderian would be upset by "cultural appropriation", and hard to give a coherent account of the offense otherwise [cf. Richard Thompson Ford, Racial Culture: A Critique].) Whether this is historical continuity of tradition, and if so whether Du Bois was one of the channels of transmission, or whether on the contrary it is parallel re-evolution, would be a fascinating thing to know.
Allison Brennan, If I Should Die
Mind-candy thriller: in which outsiders try to start a legitimate business in a deeply criminal town in upstate New York, and plot ensues. A bunch of the soap-opera among the characters presupposing reading the earlier books in the series. (To be clear, I enjoy the soap opera.)
Tarquin Hall, The Case of the Reincarnated Client
Mind-candy mystery, the latest installment in Hall's "Vish Puri" series in contemporary Delhi. Delightful as always, and I think full enjoyable without the prior installments. (Read before sectarian riots in Delhi became topical again.)
Walter Jon Williams, Quillifer the Knight
Mind candy fantasy, but high quality mind candy. Quillifer (introduced in the book of that name) is an opportunistic social climber with a tragic backstory and a supernatural nemesis in a formerly-medieval fantasyland undergoing a renaissance. He's charming, deceptive, clever, realistically concerned with money and finance, and a convincing mixture of scoundrel and creature of his own conscience. I suspect that Williams owes some debt to such Renaissance-setting competence-porn as Hilary Mantel or Dorothy Dunnett, but as always he makes the material fully his own, so it's not just "Wolf Hall with magic".

Books to Read While the Algae Grow in Your Fur; Pleasures of Detection, Portraits of Crime; The Progressive Forces; Commit a Social Science; Writing for Antiquity; Scientifiction and Fantastica; Networks; Enigmas of Chance; Complexity

Posted at February 29, 2020 23:59 | permanent link

January 31, 2020

Books to Read While the Algae Grow in Your Fur, January 2020

Attention conservation notice: I have no taste, and no qualifications to say anything about Iranian history, political philosophy, sociology, or American literature and art.

Abbas Amanat, Iran: A Modern History
The title is, of course, a pun. On the one hand, it's a "modern" history because it's not an ancient or medieval one: it begins with the founding of the Safavid empire around 1500, and continues down to the now-abrogated nuclear deal with the US. (Bad decisions by mad and/or decadent rulers is one of his themes for a reason.) But it's also a modern history because Amanat is up-to-date on the scholarship about everything, and it's all here: high politics and its personalities and swings of fortune, village life and its rhythms and transformations; economic history across regions and centuries; the interplay between urbanism and nomadism, and the impact of motor cars on both; literature and philosophy, painting and movies, music and architecture; secular and religious law; the decline and growth of Iran's knowledge of the outside world, and the growth of orientalist scholarship on Iran and on Persianate culture more generally; Iranian imperialism (across multiple dynasties) and its rivals, Ottoman, Mughal, Uzbek, Portuguese, Dutch, French, British, Russian, Soviet and American; and of course religion. (Amanat is especially sensitive to the enduring role of messianic and millenarian movements in this history, going all the way back to the Safavids themselves.) It's a dazzling display of learning and sympathy, but also of skillful exposition. It's hard to imagine there's a better general history of Iran currently available. If you have time, read it before we start bombing.
W. G. Runciman, Great Books, Bad Arguments: Republic, Leviathan, and The Communist Manifesto
Runciman is a sociologist, distinguished for having worked out a genuinely evolutionary theory of social change (specifically, of the differential reproduction of social practices). This book is not about that. Rather, it's about an elderly, eminent sociologist, long interested in political philosophy, revisiting some of the classics of political theory and saying (more eloquently) "What were you guys smoking?" I am pleased to see that Runciman finds The Republic as baffling as I do, and his strictures against Hobbes, and against Marx and Engels, are for the most part sensible. His last chapter tries to rescue these books, on the grounds that they're not trying to be accurately descriptive nor ideally (idly?) normative, but "optative", hopeful. I am not sure even this can save Plato.
Linda Nagata, Silver
Far-future space opera. This is a direct sequel to Nagata's Edges, in which some of the characters from that novel find themselves on an artificial world which has been the site of a role-playing game going on for millennia. The world is now buggy, in plotful ways, and the megalomaniacal post-human entity who helped make it is about to return, much to the detriment of the really-very-nearly-human players, if not of everyone else in local space. Exploration, trickery, time dilation, improvisation, crop-dusting, self-pity, firmly-held misconceptions, journeys through the underworld and heart-break ensue.
As I've said before, Nagata is, to my taste, one of the best of her generation of hard-SF writers. Silver doesn't have the quite the same sense of cosmic strangeness and scale as Vast (which may be her masterpiece), but it's still really good, and shows (again) that she also has the talent to paint on a smaller scale.
(I feel compelled to note that while Nagata is a science-fiction author of a certain age, and this book unites the story-worlds of her "Nanotech Succession" series and Memory, she shows no other signs of brain-eater syndrome.)
Franco Moretti, Far Country: Scenes from American Culture
A version of what is (apparently) Moretti's long-run undergrad course on American literature and culture. Some of Moretti's characteristics are here --- he's really big on comparisons between multiple examples of genres, so there's a (brilliant) chapter contrasting Westerns with film noir, but it's definitely a contrast between the genres, rather than between any particular Western and any particular film noir. But I guess not even Moretti wanted to teach intro. lit. to freshmen through distant reading.
Paul Barnetson, Critical Path Planning: Present and Future Techniques (Princeton: Brandon/Systems Press, 1970, reprinting the London: Buttersworth, 1968 edition)
You may be amused, or horrified, to learn that when I was a small boy, my father the development economist used to tell me to always think about the "critical path" when planning my activities --- to figure out what part of the over-all project would take the longest to complete, and make sure everything else would fit inside that time. I don't remember how old I was when I first heard this, but it was definitely before he taught me linear programming, which was when I was 10 or 12. To this day, I feel vaguely guilty if I start preparing a tea-pot before setting water to boil.
When I ran across this little book (six years older than I am) at a sale, I couldn't resist seeing how close what I'd been taught was to the genuine article. If you ever need a short (102 page), clear explanation of the critical path method, aimed at reasonably smart but not very mathematical managers, complete with 1960s-vintage computer output, here you are. It's... not that different from what I learned as a boy, and now I don't know whether to be amused or horrified myself.
Tess Gerritsen
  1. The Apprentice
  2. The Sinner
  3. Body Double
  4. Vanish
  5. The Mephisto Club
  6. The Keepsake
  7. Ice Cold
  8. The Silent Girl
  9. Last to Die
  10. Die Again
  11. I Know a Secret
Mind-candy psycho-killer thriller novels. While not the apex of their genre, the fact that I read them all in a month shows they obviously do instill the "I need to find out what happens next" urge. (But they also instilled no desire to go back and read the first book in the series, or Gerritsen's other stuff.)

Books to Read While the Algae Grow in Your Fur; Pleasures of Detection, Portraits of Crime; Scientifiction and Fantastica; Philosophy; Commit a Social Science; Afghanistan and Central Asia; Islam and Islamic Civilization; Writing for Antiquity; The Beloved Republic; The Commonwealth of Letters; Mathematics

Posted at January 31, 2020 23:59 | permanent link

January 29, 2020

"Data science methods to reduce inequality and improve healthcare" (Also Next Week at the Statistics Seminar)

Attention conservation notice: Announcement of a highly technical talk, in a subject you're not interested in, at a university you're not near.

Emma Pierson, "Data science methods to reduce inequality and improve healthcare"
Abstract: I will describe how to use data science methods to understand and reduce inequality in two domains: criminal justice and healthcare. First, I will discuss how to use Bayesian modeling to detect racial discrimination in policing. Second, I will describe how to use machine learning to explain racial and socioeconomic inequality in pain.
Time and place: 4--5 pm on Wednesday, 5 February 2020, in Hamerschlag Hall B103

As always, talks are free and open to the public.

Enigmas of Chance; Commit a Social Science

Posted at January 29, 2020 13:45 | permanent link

January 28, 2020

"The Blessings of Multiple Causes" (Next Week at the Statistics Seminar)

Attention conservation notice: Announcement of a highly technical talk, in a subject you're not interested in, at a university you're not near.

I am very much looking forward to our first talk of the semester, which I am sure will evoke lively discussion:

Yixing Wang, "The Blessings of Multiple Causes"
Abstract: Causal inference from observational data is a vital problem, but it comes with strong assumptions. Most methods assume that we observe all confounders, variables that affect both the causal variables and the outcome variables. But whether we have observed all confounders is a famously untestable assumption. We describe the deconfounder, a way to do causal inference from observational data allowing for unobserved confounding.
How does the deconfounder work? The deconfounder is designed for problems of multiple causal inferences: scientific studies that involve many causes whose effects are simultaneously of interest. The deconfounder uses the correlation among causes as evidence for unobserved confounders, combining unsupervised machine learning and predictive model checking to perform causal inference. We study the theoretical requirements for the deconfounder to provide unbiased causal estimates, along with its limitations and tradeoffs. We demonstrate the deconfounder on real-world data and simulation studies.
Time and place: 4--5 pm on Monday, 3 February 2020, in Hamerschlag Hall B103

As always, talks are free and open to the public.

Enigmas of Chance; Constant Conjunction Necessary Connexion

Posted at January 28, 2020 12:06 | permanent link

Three-Toed Sloth