←August→
| Sun |
Mon |
Tue |
Wed |
Thu |
Fri |
Sat |
| |
|
|
|
|
|
1 |
| 2 |
3 |
4 |
5 |
6 |
7 |
8 |
| 9 |
10 |
11 |
12 |
13 |
14 |
15 |
| 16 |
17 |
18 |
19 |
20 |
21 |
22 |
| 23 |
24 |
25 |
26 |
27 |
28 |
29 |
| 30 |
31 |
|
|
|
|
|
Archives
Categories
Self-Centered
Books to Read While the Algae Grow in Your Fur
Books (etc.) I've read this month and
feel I can recommend (warning: I have no taste)
- Italo Calvino, If on a Winter's Night a Traveler
- Catherine Bracy, World Eaters: How Venture Capital is Cannibalizing the Economy
- Rachel Neumeier, Invictus: Captive
- Julia Spencer-Fleming, At Midnight Comes the Cry
- Dan Davies, The Unaccountability Machine
- Edmund Stump, Otherworld Antarctica: Ice, Rock, and Wind at the Polar Extreme
- Rachel Kushner, Creation Lake
- Michelle Sagara, Cast in Sorrow and Cast in Flame
- Stephen M. Camarata, Late-Talking Children
- Amarjit Budhiraja and Paul Dupuis, Analysis and Approximation of Rare Events: Representations and Weak Convergence Methods
- Liang Cai, Witchcraft and the Rise of the First Confucian Empire
- Elizabeth Hand, Hard Light
- Pierre-Simon Laplace, A Philosophical Essay on Probabilities
- Martha Wells, Queen Demon
- Carles Lalueza Fox, Inequality: A Genetic History
- Viktor Dotsenko, An Introduction to the Theory of Spin Glasses and Neural Networks
- Dirk Husmeier, Neural Networks for Conditional Probability Estimation: Forecasting Beyond Point Predictions
Upcoming Talks
|
July 31, 2026
Books to Read While the Algae Grow in Your Fur, July 2026
Attention
conservation notice: I have no taste, and only the memory of qualification to opine on thermodynamics. Also, most of my reading this month was done at odd hours and/or
while chasing after a pre-schooler, so I am probably less reliable and more
grumpy than usual.
- S. R. de Groot and P. Mazur, Non-Equilibrium Thermodynamics
- This is a very comprehensive treatise on the state of the subject as of the
1962. (As discussed below, that's fine). The theory considered is basically about small departures from
the equilibrium state. The intensive variables / thermodynamic forces
(temperature, pressure, chemical potential, electrical potential, etc.) are
near their equilibrium values, and the departures from equilibrium lead to
fluxes of the extensive variables (flows of heat, of volume displacement, of chemical reactions, of electrical current)
which are proportional in magnitude to the distance from equilibrium, the constants of proportionality being "phenomenological coefficients", in symbols (eq. IV.1)
\[
J_i = \sum_{k}{L_{ik} X_k}
\]
with the \( J \)'s being the fluxes (extensive variables per unit time per unit
volume); the \( X \)'s being the thermodynamic forces (intensive
variables minus their equilibrium values); and the \( L \)'s being the
phenomenological coefficients. The entropy production (change in entropy per unit time per unit volume) is bi-linear in the fluxes and forces (eq. III.21 and ch. IV.1):
\[
\sigma = \sum_{i}{J_i X_i} = \sum_{i,k}{L_{ik} X_i X_k}
\]
The second law of thermodynamics now amounts to \( \sigma \geq 0 \), forcing the matrix of phenomenological coefficients to be non-negative definite. Various symmetry arguments put further restrictions on the \( L \) matrix.
The authors then work out an extremely
comprehensive theory of the resulting dynamics (which can
be non-linear for a variety of reasons), including Gaussian stochastic
fluctuations, and apply it to a huge range of physical problems, of increasing
complexity. Comparisons to experimental data are mostly verbal and/or confined
to footnotes, but they are made.
- The book mostly aims, successfully, to stay at the level of pure
thermodynamics. It does, however, invoke statistical-mechanical
considerations, about an underlying microscopic structure to matter, in two
places. One, which is extremely important to the whole development, is the
"Onsager reciprocity relations"
(ch. IV.3 and VIII): if, say, a gradient of chemical potential induces a heat
flux (which it may), then a temperature gradient will also induce a
chemical-reaction flux, and the two phenomenological coefficients will be equal, \( L_{ik} =
L_{ki} \). (I'm glossing over the complications induced [you should pardon the
expression] by external magnetic fields, but the book handles all that in
detail.) The equality of the coefficients actually follows from the
time-reversibility of the microscopic physics, from which the thermodynamic
quantities emerge. The other important place where statistical-mechanical
considerations show up is in the chapter (IX) on the kinetic theory of gases,
showing how, at least sometimes, the phenomenological coefficients can actually
be theoretically predicted, and not just experimentally measured.
- There are some parts of the sciences where I get the impression that they
are just done, not because they are now boring or useless, but because
everything important has been worked out and received a settled expression.
Linear irreversible thermodynamics strikes me as almost this way: the
generation or two from Onsager
through Prigogine to
Mazur said almost everything worth saying, within their domain. The biggest
modification a really modern treatment of these phenomena would make would be
to add symmetry breaking and order parameters --- so "really modern" dates
to 1975. I realize that my
saying this makes it (forgive the expression) thermodynamically inevitable that
one reader will inform me of all the groundbreaking work done in linear
irreversible thermodynamics in recent years, while another reader explains how
the theory is entirely refuted. In the meanwhile, I very much enjoyed
revisiting thinking seriously about
physics. §
- (I browsed this in grad school, and sold my copy during one of my moves.
Years later, I got stuck on a project until I came up with a trick I was
briefly very proud of; then I realized that "my" trick was actually something
I'd read about here, the "progress variables" from chapter X on chemical
reactions. On the occasion of actually submitting the blessed paper, about
which I hope to say more soon, I read / re-read this whole book from scratch,
and I am glad I did, though I will confess I skipped the exercises. One thing
I had forgotten since the 1990s was that the authors use a system of notation
for matrices and tensors I have literally never seen anywhere else. --- I
bought my current copy in
2023 directly
from Dover, which at the time threw in an electronic copy for no extra
cost. [The e-book is an EPUB with decent but not perfect OCR and no
hyperlinks.] I can't tell from their website if they are still doing that.)
- Emily Croy Barker, The Thinking Woman's Guide to Real Magic
- Mind candy fantasy: in which an early-21st-century American young woman,
whose English-lit thesis and love life are going equally poorly, does a good
deed; gets rewarded with three wishes, without realizing it; and,
in consequence, wanders off, first into Faery and then into
a Fantasyland
which is convincingly poor, sexist and dirty, but also magical. Drama ensues. §
- Ilona Andrews, This Kingdom Will Not Kill Me
- Mind candy fantasy: in which an early-21st-century American young woman
wakes up in the world of her favorite fantasy series, which is not Game
of Thrones, but is also obviously inspired by Martin, and does her best
to thrive using her prodigious knowledge of the books. Inevitably, this is
something of a commentary on why anyone would want to read such books. §
Scientifiction and Fantastica;
Physics
Posted at July 31, 2026 23:59 | permanent link
July 07, 2026
Algal Programming Note
Attention conservation notice: Petty details of a desultory instance of a moribund medium.
Because there are half-a-dozen actual research projects I should be working
on, to say nothing of a grant report to write and fall classes to design, I
decided to procrastinate,
i.e., to go to my drafts folder and dust off some old pieces of book-chat:
While I was at it, I switched the buy-this-book links
from Powell's
to Bookshop.org. Powell's is still my
favorite (surviving) bookstore in the world, but it no longer has an affiliate program, and
a lot of the old links were just 404'd. Links for books with a 13-digit
ISBN should be OK now, but I am sure there are still some links which
don't resolve; since they didn't work before, either, you have nothing to
complain about.
Tomorrow I may feel up to rolling 1d6 to decide what to edit.
Update, 10 July: My urge to procrastinate was
strong.
Self-Centered;
Books to Read While the Algae Grow in Your Fur
Posted at July 07, 2026 16:15 | permanent link
June 30, 2026
Books to Read While the Algae Grow in Your Fur, June 2026
Attention conservation notice: I have no taste, and no qualifications to opine on socialist theory.
- "William B. Fuckley", Glasslands
- Mind candy: a thriller in the southwestern desert, and the Republic of
California, some decades after the American collapse. The author does a good
job of suggesting a plausible-sounding future, without tedious
exposition (which is too often what is meant by "world-building"). This is
clearly an amateur effort (there was a whole repeated paragraph), but as such
it's very impressive and I am happy to have paid money for it. §
- Kim Stanley Robinson, Forty Signs of Rain
- Mind candy, at once near-future science
fiction, scientist
fiction, and climate fiction. I loved this, but partly this was the
pleasant shock of recognition: much of the action revolves around the National
Science Foundation and, physically, its old headquarters in Arlington, which
Robinson captures extremely well (even the atmosphere of review panels). Two
of the main characters are a federal scientist and her policy-wonk husband
living in Bethesda, and having grown up there with such parents, he nails that
too. (The only other work of art which has given me such a feeling of finally
getting some cultural representation for my people
is The
Americans.) Of course, all of this has been deliberately wrecked by
barbarians, so these details now carry an air of melancholy they didn't have
when written. As for the idea of NSF scientists deliberately leading a charge
to do something useful about climate change, words fail
me. §
- This is the first of a trilogy; I will be devouring the rest.
- Moira J. Moore, Resenting the Hero
- Mind candy. Comparing this to The Fifth Season, it is striking how different two fantasy novels about worlds over-supplied with natural disasters, and the people with the psychic gifts to stop those disasters with their minds, can be in tone and execution. This is immensely fluffier, but it has no pretensions to be anything except candy, and succeeds as such. §
- (My off-line reading log says I read some later books in this series in 2010 while flying long-distance, but I retain no memory of that and [because] I didn't blog about them.)
- Mike Beggs, Ben Burgis, and Bhaskar Sunkara, The Blueprint: How Socialism Can Work in the Real World
- This is the best --- the most accessible and the most forceful --- case for
market socialism
since Alec Nove
in the 1980s. I will not rehearse my own arguments for
either socialism, or
for market socialism over
planning. (This book argues both points, in some detail. While not making
a big deal of it, they are quite clear that Marxist political economy is no
help at all at designing a functioning socialist economy.) Instead, I will
take it for granted both that some form of socialism is desirable, and that
market socialism is the only one which has any chance of working well.
- The main move here is to replace private ownership of the means of
production with representative democracy within each firm. People and firms
still buy and sell at whatever prices they can negotiate, firms can go
bankrupt, etc., but firms are owned by their workers, so workers get both a
fixed wage and a fluctuating profit share. (Though they do not emphasize it,
the authors are aware that this way
of
re-distributing
surplus value allocating profit means workers would have less
diversification in their income than they would under something
like Roemer's
stock-market socialism.) Some workers have more decision-making authority
within the firm; all workers get to vote on who those people are.
Employee-managed firms have, empirically and theoretically, a reluctance to
hire new workers, but the authors have thought carefully about how to
ameliorate this.
- There is a system of competing publicly-owned banks, instead of a stock
market and the rest of the current finance system, plus the central bank
handling monetary policy and the payments system. There are some suggestions
here about replacing simple loans from the public banks with more risk-sharing
arrangements. I don't think this amounts to taking an equity position
by another name, but I'd want someone better at finance than I to go over that.
In any event, some (much?) of the profits of the public banks get passed back
to the state budget. I think the authors would probably regard allocating part
of the income stream of the public banks to a citizen's dividend as a friendly
amendment (cf.).
- There are some places where I have my hesitations. Three of these have to
do with the boundaries of the firms, with the banks, and with working for civil society organizations.
- With workplace democracy, just which activities are carried on within a
firm, and which by market transactions between firms, may become much more
contentious than they are now. (I
have long had a
thing about this.) The authors are aware of this, but I think they might
be under-rating it. Let me make up an example: there will, presumably, be
chemical engineering design firms in the socialist commonwealth. (We may
presume the actual chemical plants will be heavily regulated for their
environmental impacts, if not mandatorily
practicing green chemistry, that
appropriate pollution charges
will be applied, etc.) Those firms will need janitorial, book-keeping, etc.,
services which are not themselves chemical engineering. It is not at all clear
whether the janitors, book-keepers, and the rest should be voting members of
the firm, or whether instead the engineering firm should hire the services of
janitorial, book-keeping, etc., firms, which will themselves be workers'
collectives. On the one hand, provided everyone has safe, dignified working
conditions, a democratic say in how their job is done, and enough pay to live a
decent life, there doesn't seem to be much of an ethical issue about
whether the janitors are part of the chemical firm or of their own firm. On
the other hand, even in a socialist commonwealth, there are likely to be
differences in profit rates between chemical engineering and janitorial
services! (The human and material costs to entering the janitorial-services
market are probably much lower than those of entering the chemical-engineering
market, hence
more competition and thinner margins in the former than the latter.) Of course
in our world such boundary questions are set by a mixture of convenience to
stock-holders, regulatory arbitrage, and transaction costs, which may be very
bad for many people, but the socialist commonwealth will need some way
to fix these boundaries, too.
- The public banks play a very big role in this scheme. They finance the
formation and expansion of worker-owned enterprises. In fact, they are suppose
to catalyze the formation of new enterprises. The authors realize all this
means that the work of the bankers will need to be rewarded, but they are
(understandably) not very specific about how, exactly, to do that. It's only
too easy to imagine the work being done poorly ("they pretend to pay us, and we
pretend to invest"), or the bankers in fact becoming a very rich and
influential class, or indeed both at once.
- Turning to civil society: it's an important part of the scheme that all
enterprises be worker-run collectives. (They allow for a small number of
freelancers/sole-proprietors.) But there is a bit of a tension here with
having a vibrant civil society. A bunch of citizens can get together as a
society for the prevention of cruelty to animals (or androids, same
difference), and volunteer their own time and resources: not a problem. But
when the SPCA gets big enough that its administrative needs strain
volunteerism, then what? It would seem that it can't just hire some clerical
help, because those would be employees and not worker-owners. The best I can
come up with is that the staff of the SPCA would have to form a collective firm
with one client, viz., the SPCA. (Of course that firm might grow and come to
serve multiple civil-society groups, which gets us back to some of the stuff about the boundaries of organizations.)
I want to make it clear that I do not regard any of these issues as fatal flaws.
- The authors are extremely experienced at writing for popular audiences, and
this shows (*). They have clearly read, and wrestled with, a lot of the
academic literature, but this is not at all an academic book, and is a lot
stronger for it.
(Roemer
or Stiglitz, let
alone Kornai,
this is not.) They do a good job of positively arguing for their
vision, and of anticipating and countering objections. The one set of
potential objections they are categorically silent on is "Well, how do we get
there?", i.e., how we might transition from actually-existing capitalism to
their form of market socialism. But I think it is enough that they have
described, thoughtfully and accessibly, an attractive future which we
might want to get
to. §
- Disclaimers: One of the authors sent me an advance copy; I was very impressed and ended
up writing a blurb. (We'll see if it gets used.)
- *: Burgis and Sunkara, in particular, are skilled
participants in the online attention economy, whose voluminous ephemeral
writings are marked by the usual adaptive vices of that world. All this is,
mercifully, not present is this book. ^
Books to Read While the Algae Grow in Your Fur;
Scientifiction and Fantastica;
The Progressive Forces;
The Dismal Science
Posted at June 30, 2026 23:59 | permanent link
June 18, 2026
Data Over Space and Time
Collecting posts related to this course (36-3467/36-667).
- Fall 2018:
- Books to Read While the Algae Grow in Your Fur, June 2018
- Course Announcement
- Lecture 1: Introduction to the Course
- Lectures 2 and 3: Smoothing, Trends, Detrending
- Lecture 4: Principal Components Analysis I
- Lecture 5: Principal Components Analysis II
- Lecture 6: Optimal Linear Prediction
- Lecture 7: Linear Prediction for Time Series
- Lecture 8: Linear Prediction for Spatial and Spatio-Temporal Random Fields
- Lectures 9--13: Filtering, Fourier Analysis, African Population and Slavery, Linear Generative Models
- Lectures 14 and 15: Inference for Dependent Data
- Lecture 17: Simulation
- Lectures 18 and 19: Simulation for Inference
- Lecture 20: Markov Chains
- Lectures 21--24: Compartment Models, Optimal Prediction, Inference for Markov Models, Markov Random Fields, Hidden Markov Models
- Self-Evaluation and Lessons Learned
- Books to Read While the Algae Grow in Your Fur, December 2018
- Books to Read While the Algae Grow in Your Fur, January 2019
- Books to Read While the Algae Grow in Your Fur, February 2019
- Fall 2020 (inshallah):
Posted at June 18, 2026 22:42 | permanent link
June 03, 2026
Family Amusements
I realize my book-blogging has, like all my blogging of any kind, fallen off
drastically. Fortunately, my younger brother
has decided to take up book-blogging,
so I will, in the meanwhile, refer you to the more suave and sociable Shalizi
brother. He is also,
as our
mother once put it to us, "a real scientist", i.e., an experimental
biologist like her, and not a mere theoretician. This shows,
to the
reader's great
advantage.
Kith and Kin
Posted at June 03, 2026 10:15 | permanent link
May 31, 2026
Books to Read While the Algae Grow in Your Fur, May 2026
Attention conservation notice: I have
no taste, and no qualifications to opine on history or poetry.
- Madeleine Robins, The Doxies Penalty
- Mind candy: continuing Robins's series of historical mysteries set in a very-slightly-alternate Regency London. (I re-read the last one, Sleeping Partner, before turning to this.) Robins continues to be great and I hope that many more of these will flow from her pen. (She has a fine eye for how to offer her protagonists poisoned gifts.) §
- Dan Abnett, Eisenhorn
- Mind candy, calling for some comment.
- I vaguely remember Warhammer 40,000 as a
role-playing game from around 1990, i.e., when I was in high school. (This would have been the first edition.) I can't remember if my group ever actually got around to playing it *. I am also
aware that it has become something of a nerd-cultural phenomenon, but have
genuinely not kept up with it. Now, I am the kind of nerd who has a certain disdain for novels written in licensed intellectual properties, but I readily acknowledge that it can, in fact, be great. (John M. Ford's Star Trek
books, for instance.)
- When this collection fell across my path cheap, I picked it up out of
curiosity. It's pretty decently adequate, as mind candy. There are places
where Abnett shows flashes of real talent. (To avoid spoilers, I'll just say:
The first alien planet; the opening scenes with the cranky biologist.) There
are other aspects of the novels which are pretty unsatisfying. (I don't feel
mean enough, today, to elaborate.) It'd be incomprehensible without some
background familiarity with the setting. But I don't see anything special. My
curiosity being satisfied, I am pretty unlikely to peruse any other fiction
from this brand, unless someone reliable points to a particular instance as
something extraordinary. (This is not an invitation to send me pointers.)
- --- The question arises of why this particular brand/game/setting
has become so popular; as this will let me take
some hobby-horses
out for a spin, I will pursue it. Why are there endless licensed books, to say
nothing of fan-produced material and casual allusions,
for Warhammer rather than any other role-playing game from around
1990? One possibility: if you are going to actually play a game, you
need to coordinate with other players on which game. (Also, more
notoriously, on a time and place...) This creates a role for focal points,
and focal
points are subject to increasing returns. Therefore, we should
expect some games to be vastly more popular than
others, even if all
games were equally good, spoke equally to our moment in history, etc. This
argument is somewhat undercut by the fact that some people are perverse enough
to buy and read rule-books for RPGs they don't actually play. (I am one of
them.) It also doesn't explain the tie-in material. But even when there are no
focal points, we know that cultural trends can be self-reinforcing
(Salganik, Dodds and
Watts
2006; Salganik and
Watts 2008). The important ingredient for such phenomena is that your odds
of encountering artifacts from the trend must go up, the more of your peers are
participating in the trend. Social media, and before
that search engines
and recommendation systems,
deliver this in spades, but even a best-seller list will do (that's essentially
what Salganik et al. used). I decline to seek an adaptive explanation
for Warhammer's success, or to interpret it as meaningful symptom
that Really Says Something About Our Society, unless someone shows that it's a
lot more successful, or differently successful, than we'd
expect from the nigh-inevitable tendency of cultural interaction to amplify,
and so lock in, random accidents. (In other words: there are very
high-amplitude branches of the wave-function where all the memes are
about Traveller, or Vampire the Masquerade.) This
concludes today's edition
of A Series of Footnotes
to Herbert
A. Simon. §
- *: We mostly did D&D and Call of Cthulhu, but I definitely remember Stormbringer, Beyond the Supernatural, Traveller and Gamma World. ^
- Marie Boas, The Scientific Renaissance, 1450--1630
- An old (1962) but still interesting history of the earlier part of the
scientific revolution. In every field, Boas traces a basically-humanist
impulse to discard medieval ideas and accomplishments, in favor of reviving
antiquity, the more antique the better, which only gradually turned towards
striving to go beyond Greco-Roman science to something actually new and
unprecedented. She offers sympathetic (but unvarnished) portraits of the
greats (Harvey, Gilbert, Vesalius, Tycho, Kepler, Galileo, Bacon, etc., etc.),
along with attentive readings of their major works, interspersed with
sketches of the broader environment and trends in learning and its
application. It's not as deep, conceptually, as something like Wootton, but it's readable and humane, and covers most of what's important. §
- Joseph Brodsky, Selected Poems, 1968--1996
- I am a Philistine, because I thought this was just OK. There were some
good poems, quite a few which I found boring, and absolutely nothing that has
stuck in my memory even a few weeks later. I realize that the contemporary
poets I enjoy have, in fact, a very different judgment of Brodsky, but I did
warn you that I have no taste. §
Books to Read While the Algae Grow in Your Fur;
Scientifiction and Fantastica;
The Commonwealth of Letters;
Writing for Antiquity;
The Great Transformation;
Tales of Our Ancestors;
Portraits of Detection, Pleasures of Crime
Posted at May 31, 2026 23:59 | permanent link
April 30, 2026
Books to Read While the Algae Grow in Your Fur, April 2026
Attention conservation notice: I have no taste, and no qualifications to opine on poetry or international political economy.
- Henry Farrell and Abraham Newman, Underground Empire: How America Weaponized the World Economy
- Regular readers (if I have any left) know that Henry is a long-time
co-author and, even more, a friend; it'd be absurd to pretend I could write an
objective review of one of his books.
- With that disclaimer out of the way, this is a great book, which
simultaneously illuminates some very, very important phenomena, and is
very well-written. The phenomena are all about the ways in
which globalization has turned out to create a series of "chokepoints",
central nodes in the various networks of finance and communication,
and to have placed those chokepoints under the control of the United
States, without anyone really intending or even expecting this.
Over-simplifying greatly: because the US was already the economic and
technological leader of the world (and had been
for some time),
it made sense --- it was cheaper or more efficient --- to route connections
through America, often through very specific parts of America. (Such as office
parks in suburban northern Virginia.) Having created these connections, and
the very material infrastructure which implements them, it then made more sense
for further connections to route through those American nodes, rather than
going through the expense of creating parallel, and perhaps little-used,
routes. (There is a reason I finally posted
about search and increasing
returns while finishing this book.) Equally accidentally, and largely as a
consequence of the War on Terror, the US government woke up to the fact that
it could control those chokepoints, i.e., deny adversaries access to
those networks; what was almost as good, it could very credibly threaten to do
so.
Hence "weaponized
interdependence". Having become conscious of these powers (not least
because of Henry and Newman's own article), the US government has proceeded to
use them with abandon, provoking nigh-inevitable responses from other countries
and other actors.
- This is both a sophisticated work of social science (I can see where
specific works on network theory or path dependence have shaped their thinking,
and I daresay there are non-complex-systems influences I miss), and one which
conveys a lot of detail about specific institutions and technologies. It is
nonetheless not ponderously learned or monographic, but rather
readable, even exciting, and something which can, and should, be read very
widely. (Henry has
a nice
blog post about they ways they deliberately learned from thrillers and
science fiction about how to narrate complicated systems.) It is, to repeat,
a great book, and I am proud to call one of its authors a friend.
- (Long ago, I made myself a promise that I would never recommend or review a
book unless I finished it cover-to-cover. So it is that I only this month got
to reading the last chapter, about how these powers could be restrained and/or
turned towards good ends, such as combating climate change. This is painful
reading in light of everything that has happened since the book came out in
2023, but there might still be some value to it, once the Republic emerges from
its Babylonian captivity.) §
- Jen Williams, The Bitter Twins
- Mind candy fantasy: sequel to The Ninth Rain (in my backlog; suffice to say: good), and continuing
the high standard of that book. I will say there was a bit in the middle where
I set it down for a couple months, before I made myself press through it, but
that was because I could tell that horrible things were about to happen to all
of the separated bands of heroes, and I cared about the characters too much.
(Horrible things did happen, but it was worth it.) I look forward to
reading the
sequel. §
- Ludovico Ariosto (trans. David R. Slavitt), Orlando Furioso: A New Verse Translation [doi:10.4159/9780674053519]
- At the border of "mind candy" and "classics of the Western canon" (hence the DOI). I shall
emphasize the former. This is a big fat (600+ pp.) epic fantasy which follows
an over-lapping set of characters, switching between viewpoints, as they go on
quests, meet in single combat and in vast battles, fall in and out of love,
perform daring rescues and even more daring feats of skull-duggery, listen to
and enact prophecies, retrieve magic artifacts and tame or fight mythic
beasts, go mad (hence the title), etc., etc. (It's even a sequel to a popular
series, but continued by another author.) It is, in short, very much
a recognizable genre work, both in content and form. It's also an
epic poem, composed and translated in verse, and very, very much a product of
a Renaissance world-view, projected back to the
time of Charlemagne.
- My grasp of Italian is that of a pre-kindergartener. (This despite the best
efforts of my mother, and of Signora Anna and the other teachers at Silver
Spring Bilingual Montessori.) I can have no opinion about whether the
translation is accurate, but it is pretty good English verse. §
Books to Read While the Algae Grow in Your Fur;
Scientifiction and Fantastica;
Commit a Social Science;
Networks;
The Continuing Crises;
The Dismal Science;
The Beloved Republic;
Kith and Kin;
The Commonwealth of Letters
Posted at April 30, 2026 23:59 | permanent link
March 31, 2026
Books to Read While the Algae Grow in Your Fur, March 2026
Attention conservation notice: I have no taste.
- Rachel Neumeier, This Hour, Our Vigil
- Mind candy fantasy; A sequel to the Death's Lady books (in my
backlog), where good things happen to our favorite characters, without any
special drama. If I say it reads a bit like fanfic for Neumeier's own work,
that sounds like I'm damning it, but I actually enjoyed it, because, I wanted
good things to happen for those characters, who'd had drama enough and to
spare. Utterly pointless if you have not read the earlier books. §
- Doris Piserchia, A Billion Days of Earth
- Mind candy science fiction, from 1976. This is an especially weird member
of the Dying Earth genre. It's a billion days (= a few million
years) in the future, and the descendants of human beings are known as "gods",
while evolved (possibly uplifted) rats think of themselves as Homo
sapiens. One of those wakes Something which I think is an old
psycho-bio-weapon that preys on unconscious desires and assimilates its victims
into further bits of its silvery, amorphous self. Apocalypse ensues, as viewed
by an unusually level-headed rat-man, among others. It's
not pleasant, but it is skillful, and strange, and makes me interested
in Piserchia's other books. §
- (I picked this up on 26 July 2001, in a bookstore in Flagstaff which no longer exists...)
Books to Read While the Algae Grow in Your Fur;
Scientifiction and Fantastica
Posted at March 31, 2026 23:59 | permanent link
March 20, 2026
"Aware of All Internet Traditions: Generative AI as Information Retrieval and Synthesis" (Verbatim Remarks at the Cultural AI Workshop)
Attention conservation notice: Deep Thoughts about Generative AI and intellectual tradition, from a statistician who has not made a contribution to either area of scholarship.
The talk I gave last week at
the NYU
Cultural AI workshop was an expansion of some remarks from
a blog
post a year ago, and some stuff
I said
in e-mail to Henry Farrell. (Many thanks
to Leif
Wetherby and Tyler Shoemaker for
the invitation and organization.) I was thinking of turning the talk
into a post of its own.
But Ben
Recht has just scooped me, in the most generous way possible. So, if
you're interested, read Ben,
and, for the real gluttons for punishment, my slides.
(I am meditating on Leif's suggestion that I am trying to revive
structuralism.)
Manual trackback (of a sort): Notes from a Small Press;
Brad DeLong
Self-Centered;
Minds, Brains and Neurons;
Automata and Calculating Machines;
Enigmas of Chance;
The Commonwealth of Letters;
The Collective Use and Evolution of Concepts
Posted at March 20, 2026 11:00 | permanent link
March 06, 2026
Search and Increasing Returns, or, No One Makes You Push to Github
Attention conservation notice: An economistic argument that computer networks are nigh-doomed to effective centralization, by someone who is neither an economist nor a computer scientist. Arcane, speculative, and not actionable even if correct. You would be better off spending your time reading a book.
Drafted in 2018, and deliberately not much updated. Posted now because I found myself re-using the joke of the subtitle in an e-mail.
Twitter is awful
for many reasons, but
not the least of them is the way it makes its users feel forced to keep using
it.
(One
such complaint among many.) Early visions of how it could contribute to a
beneficient or at least harmless ecosystem,
such Steven
Berlin Johnson's (*), presumed that it would be much less sticky, a
more old-fashioned website people could leave. Now, more recently Johnson has written a
spectacularly
wrong-headed piece hailing blockchain as a blow for re-decentralizing the
Web.
Of course, the primary working example of a block chain is Bitcoin,
which is heavily centralized in holdings, in
processing power, and in exchanges. And there are economies-of-scale
reasons to expect this would be true of anything using either proof of work
or proof of stake. But suppose the problem with Bitcoin is just that it's not
got a very compelling use:
imagine if keeping your car idling 24/7 produced solved Sudokus you could trade for heroin --- @Theophite, 16 August 2018
and that nobody has a use for blockchains, not really.
Git repositories, on the other hand, are the part of the blockchain idea that's actually good for something: a way of tracking changes, with many authors, where it is very, very hard to go back and alter history without being caught. And git repositories are things that are in-principle easy to copy and move around, and could be hosted on any machine running HTTP (or, heck, FTP). So why
does Github exist, let alone hold the position it does?
The answer, I think, comes in two parts. The first is that focal points
are subject to increasing returns via network effects. The second is that
search engines create and/or amplify focal points. Put these two together, and you get a very strong tendency for
any particular line of activity --- be it trading solved sudokus for heroin or
open-source software development --- to concentrate in just one, or at most
a few, online locations.
Incidentally: This doesn't necessarily lock in first-mover
advantage, because other forces can overcome this (revulsion, technical
superiority), but it does mean that it will be very hard to avoid
having at most a few dominant locations at any one time. It also means that
the transitions between dominant
locations will be brief.
If we really wanted to re-decentralize, we'd have to (1) get rid of
focal points, or (2) get rid of search engines as we've known them, or (3)
somehow make search conduct users to a distributed, decentralized focal "non-point" (focal
blob? focal rhizome?). I therefore strongly suspect we are not going
to re-decentralize. But then I didn't expect the Web would centralize as much
as it has, despite having literally learned my Brian Arthur and Paul David
at my father's knee, so what do I know?
*: To be clear, I have been a fan of Johnson's books since Interface Culture and Emergence. I am picking on two of his essays, but that's because I think he is too ready to find encouraging signs for a certain vision of how the Internet could transform the world
for the better. To be clear, I share the vision, but now hold out little
hope for it.
(Tweakage, 2 June 2026: Small changes in the last paragraph, avoid re-using the word "point" so much.)
The Dismal Science;
Networks;
Linkage
Posted at March 06, 2026 22:50 | permanent link
Statistical Complexity of Link Prediction
Attention conservation notice: Link to a two-page fragment of a mathematical paper that was abandoned over two decades ago.
I had not one, but two mentors in graduate school who, when approached with
an idea or a question, were apt to go to their filing cabinet and pull out
notes, from years or even decades back, which addressed that very issue, or one
very close to it, and which they had never gotten around to finishing. (G. and
C. had never met, and otherwise had little in common.) As a juvenile
scientist, I found this both humbling and intensely irritating: why didn't
they publish? As a middle-aged professor with a big directory of
partially-finished projects, I have more sympathy, but still want to do better
myself. I am therefore going to try to make a point of posting a lot of my
fragments, and just ask that if someone decides to build on one, they put me in
the acknowledgments.
In 2004, because I was thinking a lot
about the
statistical complexity of predicting stochastic processes, and learning
about networks
from Mark, Cris
and Aaron, I tried my hand at
defining statistical complexity for link prediction.
The resulting complexity measure seemed
straightforward-in-principle but too hard to calculate for anything very interesting
(except maybe exponential-family
random graphs, where it ends up being the entropy of increments to the
minimal sufficient statistics).
In the ensuing 22 years, I have done literally nothing with the idea, but
something reminded me of it the other day, so here's the two-page
fragment I abandoned in March 2004.
The one thing I would add to the fragment is to consider
a stochastic
block model, where each node $ i $ has a latent discrete random variable $
X_i $, IIDly across nodes; $ X_i $ says which "block" (or "community" or
"module", etc.) node $ i $ lives in. The probability of an edge between nodes
$ i $ and $ j $ is a function of $ X_i $ and $ X_j $ alone, independent of all
other dyads or anything else. The pair $ (X_i, X_j) $ is thus the "state"
which fixes the distribution of the dyad. Of course, as pair of latent
variables, this is not a statistic, a function of the observable graph
$ G $. But there are many circumstances where, as we see larger and larger
graphs, we can infer all the $ X_i $ from $ G $, with the probability of making
any errors tending to zero. (Ed McFowland and I tried to summarize those
conditions in our paper, because
we wanted to use them as tools for something else.) In these situations, the
sufficient statistic for predicting $ G_{ij} $ from the rest of the graph will
in fact tend towards $ (X_i, X_j) $ as $ n \rightarrow \infty $, and so the
limiting statistical forecasting complexity will be at most $ 2 H[X_i] $ (since
$ X_i $ and $ X_j $ are IID). I say "at most" because there could be
situations where distinct pairs of blocks have the same edge probability; if
that's ruled out, the asymptotic statistical complexity will indeed be twice
the entropy of the block variable for one node.
The same argument could extend to
any graphon, if
the node variables can be asymptotically recovered from the observed graph.
Enigmas of Chance;
Networks;
Complexity
Posted at March 06, 2026 21:09 | permanent link
February 27, 2026
How Statistics and Machine Learning Came To Have Two Different Kinds of Kernel Methods
Attention
conservation notice: Re-purposed teaching materials, about a confusing
point of terminology in two arcane disciplines. This is amateur history of
science, which will not help you learn or practice those disciplines, even if
you wanted to (which you don't). Also, it was written for an advanced
undergraduate class in one of those disciplines, and presumes some familiarity
with the jargon and concepts.
I have written several versions of this over the years, for various
classes; this one is from 18 February 2026, for a class which emphasized
(Nadaraya-Watson) kernel smoothing, splines, and kernel density estimation.
Posted now for lack of other material to refer to in the
future, rather than re-writing yet again.
TL;DR: In statistics and machine learning, "kernel methods" refer to two different families of methods, one based on convolution, the other on hiding a basis expansion in the guise of a sum over data points. These use two different (but overlapping) sets of kernels. In both cases, the name "kernel" comes from some problems involving integrals in mathematical physics. (TL;DR of the TL;DR: Blame the mathematicians.)
"Kernel" is one of those terms which is used in many distinct-but-related senses across different areas of mathematics (like "normal"). The common metaphor is "the seed (in some sense) from which some larger object or structure grows (in some sense)". In particular, in physics, there are a lot of problems which involve integral operators \( \mathcal{I} \) that map one function, say \( f \), to a new function \( \mathcal{I} f \), by the relationship
\[
(\mathcal{I}f)(x) = \int{f(z) K(x, z) dz}
\]
The inner function \( K(x,z) \) is called the kernel of the operator. (Or at least that's what it came to be called in English; a lot of this was first worked out in German, in the 1800s, and I don't know the original German technical term very early 1900s, and the German word was Kern. [See update below.])
A particularly important class of integral operators, both in physics and in a lot of other fields, take the form
\[
\int{f(z) G(x-z) dz}
\]
That is, the kernel isn't a two-argument function, of \( x \) and \( z \), but another one-argument function, only involving the difference between \( x \) and \( z \), say \( u = x-z \). This came to be called (in English) the convolution of the two functions \( f \) and \( G \). (The German original was Faltung; as late as 1933, an American mathematician writing about this uses that term because "there is no good English word" (*). I am told one would ordinarily translate this as something like "folding".) It was recognized a long time ago that if \( G(u) \) is a probability density function (pdf), then \( \int{f(z) G(x-z) dz} \) gives a weighted average of all the values of \( f \), with the weight given to \( f(z) \) depending on how close \( x \) is to \( z \). This implies that \( \overline{f}(x) = \int{f(z) G(x-z) dz} \) is a new function, related to \( f(x) \), but smoother, because \( \overline{f} \) is averaging out the oscillations and extremes of \( f \).
In the 1950s, the leading statisticians were, overwhelmingly, trained as
mathematicians, and a lot of
that mathematics was stuff which had grown out of mathematical physics, so
a great deal of this mathematics was very familiar to them. (They were
creating the graduate programs which would train statisticians as
statisticians.) In particular, one cluster of statisticians who were
doing (what we'd now call) signal processing were very interested in estimating
the power spectra of radio (and other) signals --- how much energy was
transmitted at each frequency. (This turns out to be very important
for prediction
and control.)
The raw power spectra were extremely noisy, and the statisticians realized that
by convolving those raw spectra with Gaussian (or other) kernels, they could
get something much smoother and more stable, in effect trading some bias for a
very large variance reduction. It was then
quickly
realized
that the same idea could be used to estimate probability densities, convolving the empirical distribution with smooth kernels. And a few years after that, in 1964, Nadaraya and Watson (independently) realized that you could use this to do regression.
So the "kernel" in "kernel density estimation" and "kernel smoothing" is kernel in the sense of function-you-convolve-the-data-with.
There is another set of methods, also called "kernel methods", which look rather different. Go back to how I defined integral operators:
\[
(\mathcal{I}f)(x) = \int{f(z) K(x,z) dz}
\]
This is a linear transformation on functions: \( \mathcal{I}(af + bg) = a\mathcal{I}f + b \mathcal{I}g \), for any scalars \( a, b \) and functions \( f, g \). Just as multiplying by a matrix is a linear transformation on vectors, integral operators are (one kind of) linear transformation of functions. Just as matrices have eigenvectors, integral operators have eigenfunctions, where
\[
\mathcal{I}\phi = \lambda \phi
\]
for some scalar \( \lambda \), the eigenvalue. (Eigen- is German again; roughly "self-" or "own-".) For some integral operators, i.e., for some kernels \( K(x,z) \), the eigenfunctions \( \phi_1, \phi_2, \ldots \) actually form a basis, meaning that, for any (well-behaved) function \( f \)
\[
f(x) = \sum_{i=1}^{\infty}{c_i \phi_i(x)}
\]
(Alternately, we can always define a space of functions as "everything we can get by taking linear combinations of these eigenfunctions".)
Writing arbitrary functions as weighted sums of basis functions is a very old trick in math. Doing regression in terms of some set of basis functions is almost as old as a trick in statistics. So the eigenfunctions of (nice) integral operators can be the basis functions we use for regression.
That doesn't single
them out from any other set of basis functions, but, again in the 1950s,
people realized that kernels which satisfy some properties (like symmetry and
non-negativity) can themselves be expressed in terms of the eigenfunctions and
eigenvalues:
\[
K(x,z) = \sum_{i=1}^{\infty}{\lambda_i \phi_i(x) \phi_i(z)}
\]
The usual (and correct!) explanation is to think of mapping \( x \) to the
vector of function values ("features") \( (\phi_1(x), \phi_2(x) , \ldots ) \);
doing the same thing to \( z \); and then taking the (weighted) inner product
between those vectors. As we say: "\( K(x,z) \) is an inner product in feature
space". Any linear method you can write in terms of inner products thus has a
"kernelized" equivalent. (For example.)
This also implies that a weighted sum of kernel functions is equivalent to a weighted sum of eigenfunctions:
\[
\sum_{i=1}^{n}{a_i K(x, x_i)} = \sum_{j=1}^{\infty}{b_j \phi_j(x)}
\]
(EXERCISE: Find an expression for \( b_j \).)
So doing a weighted sum of kernel functions (in this sense) is equivalent to
doing an infinite weighted sum of these eigenfunctions (which form a basis).
In fact, a lot of the time we can show that optimal function of the form \(
\sum_{j=1}^{\infty}{b_j \phi_j(x)} \) can in fact be written in the form \(
\sum_{i=1}^{n}{a_i K(x, x_i)} \), so that we really only have a
finite-dimensional optimization problem. (Such a result is called a
"representer theorem".) This was important when people, like
the statistician Grace
Wahba, worked out the mathematical details of smoothing splines. (One can
actually write out the kernel, in this sense, that's implicit in spline
smoothing, but I do not find it very illuminating.) (**)
What came to be called "kernel methods" or "kernel machines", in the 1990s,
were predictive models of the form \( \sum_{i=1}^{n}{a_i K(x, x_i)} \), where,
again, \( K \) is one of those two-argument kernels which lead to nice
eigenfunctions and eigenvalues when plugged into an integral operator. The
goal wasn't really smoothing (as it was with the convolution methods),
but to get the power of using a huge --- even an infinite! --- set of basis
functions, without having to explicitly estimate a huge set of coefficients, or
evaluate a huge set of functions when making predictions. People talked about "the kernel trick" as this way of using two-argument kernels to implicitly use vast function spaces, without explicitly calculating the functions.
There are some kernels, in this sense, which also work as kernels, in the smoothing sense (e.g., Gaussians). But the two sets of kernels are distinct, and the two sets of methods are really distinct. If you do kernel smoothing with a Gaussian kernel, there is just no way to write that as a sum of Gaussian kernels with fixed weights. In general, kernel smoothing regression give us predictions of the form
\[
S_G(x) = \sum_{i=1}^{n}{\frac{G(x-x_i)}{\sum_{j=1}^{n}{G(x-x_j)}} y_i}
\]
when we've seen data points \( (x_1, y_1), \ldots (x_n, y_n) \). In contrast, kernel regression in the implicit-function-expansion sense gives us predictions of the form
\[
R_K(x) = \sum_{i=1}^{n}{\alpha_i K(x, x_i)}
\]
It is an easy EXERCISE to show that if \( K(x, x_i) = G(x-x_i) \), and \( G(u) \) is a (non-degenerate) pdf, then there is no set of weights \( \alpha_1, \ldots \alpha_n \) which will make \( S_G(x) = R_G(x) \) for all \( x \). It is a harder EXERCISE to show that there is no combination of weights and two-argument kernel \( K \) which will make \( S_G(x) = R_K(x) \) for all \( x \).
If, back in the 1950s, the one line of work had talked about "convolutional smoothing" and the other "implicit basis function expansions" (or something like that), we would not have this confusion.
Update, 6 March 2026: On the other hand, at least the
convolutional smoothing people did not give their technique a misleadingly
psychological name...
Updates, 11 March 2025:
- Consulting the German original of Courant and Hilbert shows that the German term was Kern --- or at least, that's what was by 1924. (I should, of course, have thought to check this source before, but I didn't realize it was online!) If I can trust some German-English dictionaries, the primary meaning is something like "the edible part inside a hard-shelled nut or the skin of a fruit", which is very similar to the most basic meaning of the English "kernel". Turning to their citations, Courant and Hilbert refer to a fundamental 1903 publication on integral equations (and so on integral transforms) by Fredholm. This paper does not (so far as I can tell) ever introduce a name for what we'd call the kernel, it's «la fonction $ f(x,y) $» throughout. (Certainly «noyau», the modern French term, does not appear in Fredholm's paper.) Courant and Hilbert also cite Maxime Bôcher's 1909 Introduction to the Theory of Integral Equations (written in English), which flatly states (p. 13) "$ K $ is called the kernel of these equations", and adds, in a footnote, that the term was "first employed by Hilbert" in 1904 (in German). So "kernel" would seem to have been fixed as the English translation of the German Kern (in this context) some time between 1904 and 1909.
- The oldest appearance of "kernel" in the Annals of Mathematical Statistics is indeed Parzen (1961). By 1967, people proposing other modes of density estimation refer to the convolutional method as the "'kernel' method" or "'kernel' technique" (with the scare quotes), indicating that the term was in use but still novel.
*: Norbert Wiener, The Fourier Integral and Certain of Its Applications (Cambridge, England: Cambridge University Press, 1933), p. 45. ^
**: The oldest example I have run across of using kernel methods (in this sense) in statistics is Emanuel Parzen's work (1960/1963, 1961) on time series analysis, where it was motivated, in part, as a way around having to estimate power spectra; despite Parzen's eminence in statistics, this approach does not seem to have been much used, at least not then. (I'm sure it's no coincidence that Parzen was Wahba's doctoral adviser!) What makes this extra curious is that Parzen also wrote a 1962 paper where he (more-or-less) introduced convolutional kernel density estimation (in which he cites prior work on estimating power spectra), and that did take off. Someone with enough knowledge of mathematics and statistics, and access to his archives, could write a history-of-science paper on what led Parzen, in particular, to introduce both kinds of kernel methods into statistics. This would fascinate, oh, easily a dozen people other than myself. ^
Enigmas of Chance;
Mathematics
Posted at February 27, 2026 11:46 | permanent link
October 31, 2025
Books to Read While the Algae Grow in Your Fur, October 2025
Attention
conservation notice: I have no taste, and no qualifications to opine
on historical
genetics and
the transmission
of inequality. Also, most of my reading this month was done at odd hours and/or while chasing after a preschooler, so I'm less reliable and more cranky than usual.
(Left almost-finished in 2025, because I got interrupted, and posted in
2026, because I wanted
to procrastinate about
half-a-dozen research projects.)
- Martha Wells, Queen Demon
- Excellent fantasy mind-candy, continuing the story from Witch
King. As in that book from my back-log, the story is told both in the
the "past" time-line, in which Our Heroes rebel against Mysterious Evil
Overlords, and the "present" time-line, in which Our Much Aged Heroes Have to
Deal with Idiots Trying to Undo Everything They Built with Such Sacrifice in
Their Youth, and hints of returning Evil Overlords. (Analogy to the current
situation of the once-and-future free world is probably intended by the author,
and anyway irresistible). I do not think this book could really be enjoyed
without the previous book, but with it, it's lots of skillfully-written fun,
and leaves me eager for
more. §
- Carles Lalueza Fox, Inequality: A Genetic History [doi:10.7551/mitpress/14145.001.0001]
- There is a basic point here which is correct and important, and can be made
much more simply and starkly than Lalueza Fox puts it. Throughout most of the
life of our species, a large fraction children born died very young, far before
they had any chance to have children themselves. (In many times and
places, most children.) But this death was not entirely randomly
distributed: having richer and/or more powerful parents increased a child's
odds of surviving and reproducing, through means like "not starving to death",
"not being so malnourished as to be picked off by disease", and "not being
killed in a fight with the next valley over". (This is true even though what
passed for medicine was useless.) This implies that social inequality mattered
for whose genes got passed down. (It was not the only factor that
mattered, but it was one of them.) Social inequality thus left traces in our
gene pool. Getting information about past inequalities from the shape of the
present gene pool, and from ancient genomes we recover, means solving
a tricky
inverse problem, but that's something we're getting increasingly good at
doing.
- I think a great book could be written on all this. (Among other
things, it should be catnip to anyone who wants to call themselves
a historical materialist.)
This book, however, is merely OK. It combines accessible accounts of
scientific work, by Lalueza Fox and others, in historical genetics with
uninspired book reports about recent semi-popular social science on inequality,
in about a 2:1 mixture. There's nothing actively wrong with the
reportage, not that I could tell, but it's neither sure-footed nor
illuminating. (You'd think, to read this, that social science on inequality
began with Piketty.) I will probably
end up adding this to the reading list for
the statistics of inequality class, if I ever get
to teach it again, but with a bit of a sigh. Someone should write that great
book; in the meanwhile, this will have to
do. §
- (Over the last two hundred years or so, as societies have gone through the
singularity demographic transition, infant and childhood mortality has gone way down, and fertility has fallen dramatically. This all contributes to mute the impact of
social inequality on the gene pool. If 1/3 of all children born die before age 20, which children are in that third can carry a lot of information about social position; matters are very different if the mortality by age 20 is instead below 1%.)
Books to Read While the Algae Grow in Your Fur;
Scientifiction and Fantastica;
The Natural Science of the Human Species;
Writing for Antiquity;
Commit a Social Science;
Teaching: Statistics of Inequality and Discrimination
Posted at October 31, 2025 23:59 | permanent link
September 30, 2025
Books to Read While the Algae Grow in Your Fur, September 2025
Attention
conservation notice: I have no taste; and while I do have some
qualifications to opine on statistical mechanics and on machine learning, these
books are ones I began in 1999 (Dotsenko) and 2000 (Husmeier), so the works
themselves are unlikely to be of interest to you. Also, most of my reading this
month was done at odd hours and/or while chasing after a pre-schooler, so I am
probably less reliable and more grumpy than usual.
I will presume the reader understands the terms "spin glass" and "neural network".
(Left almost-finished in 2025, because I got interrupted, and posted in 2026, because I wanted to procrastinate about half-a-dozen research projects.)
- Viktor Dotsenko, An Introduction to the Theory of Spin Glasses and Neural Networks (1994; doi:10.1142/2460)
- This is a short (~150 pp), brisk first book on the topic for
physicists. Most of the book --- the first ten chapters --- are good
introduction to common models of spin glasses. On the mechanistic side, the
emphasis is on frustration, and how it leads to slow relaxation through
multiple local minima. On the theoretical side, the emphasis is overwhelmingly
on the replica trick and replica
symmetry breaking *. I appreciated the
detailed discussion of experimental results, but I would have appreciated still
more clarity about distinctions between quantities which are functionals of the
ensemble / probability distribution, and those which are functions of the
realized microscopic state **. Something which would have whizzed right past me
in graduate school is the role of distributions over matrices which are
symmetric under permutation of rows and columns, i.e.,
of exchangeable
arrays; now I want to read about connections between spin glasses
and graphons.
- The final chapters, about neural networks, especially
the Hopfield
model (which is perhaps back in
fashion). These are OK, but if you want to see the statistical mechanics
of neural networks (of that vintage), you're really better off with either "The
Statistical Mechanics of Learning a Rule"
(Watkin, Rau and Biehl, 1993),
or Engel and Van de Broeck
(2001).
- The first ten chapters are still a perfectly good introduction to spin
glasses and replica symmetry breaking for those with a command of statistical
mechanics at the level of a first graduate course (e.g., Landau and Lifshitz).
§
- *: Dotsenko has a whole later book about the replica method, but I haven't read it. ^
- **: In a lot of
statistical mechanics, we can be sloppy about this distinction, because considerations of ergodicity and concentration of measure imply there's not much difference (cf.). But spin glasses are different! ^
- ---Presumably (?) a different Viktor Dotsenko.
- Dirk Husmeier, Neural Networks for Conditional Probability Estimation: Forecasting Beyond Point Predictions (1999, based on a 1998 Ph.D. thesis; doi:10.1007/978-1-4471-0847-4
- The problem here is to make distributional forecasts of a (scalar) random variable \( Y \) given a (vector) random variable \( Z \), in the form of training a neural network to calculate the conditional cumulative distribution function \( \mathbb{P}\left( Y \leq y|Z = z \right) \), with \( y \) and \( z \) as inputs. The particular
version of this of most concern here is when we have a dynamical system or stochastic
process \( X_t \), and \( Z = (X_t, X_{t-1}, \ldots X_{t-k} \) and \( Y = X_{t+1} \), i.e., predictive distributions for time series, especially in a state-space reconstruction setting.
- Back in the 1990s, when we were all very impressed by the universal
approximation results for neural networks, lots of people did stuff with
three-layer neural networks (i.e., one hidden layer), and one actually had to
make a bit of argument if one wanted to do more than three layers. Husmeier uses
four layers, because he's worried about the following problem: Suppose
\( Y = f(Z) + \epsilon \), with independent noise \( \epsilon \), and we let
the variance of \( \epsilon \) shrink towards zero. The nonlinearities in the
neural network will let us represent a CDF, sure enough. but the nonlinearity
gets applied to a linear transformation of the input variables. In that
situation, it'd seem like the only way to approach a deterministic limit is if
the function \( f \) is linear in the input \( Z \). I am
not entirely sure this is right, but it's certainly true that the
extra layer Husmeier allowed himself made it a lot easier to get good results.
- Much of the book is about ringing changes on this basic arrangement,
emphasizing, as I said, the prediction of dynamical systems plus noise, with
good results. Several chapters (13--16)
concern ensemble
methods ("network committees"), with various quasi-Bayesian or
cross-validated schemes for weighting the members of the ensemble.
- Something which whizzed by me back in the day (if I read those parts at
all) is that, starting in chapter 7, Husmeier assigns random parameters to the
lowest layer(s) of the network and then leaves them fixed, only
optimizing the upper layers. This "random vector functional link net" approach
did not originate with him, but it does vastly speed up fitting,
without any real degradation in predictive performance compared to full
back-propagation (or other optimization). This is a precursor to, or even a
form of, what we now
call random feature
methods, and I am going to mine his references
(starting here) to
figure out where it came from and what happened to it.
- I should have paid a hell of a lot more attention to this book back when it came out. §
Books to Read While the Algae Grow in Your Fur;
Enigmas of Chance;
Physics
Posted at September 30, 2025 23:59 | permanent link
September 28, 2025
Unsolicited Opinions
Attention conservation notice:
Unsupported assertions, accumulated over many years in my drafts folder, last updated in early 2022, now more or less painful in retrospect.
In no particular order, except where they are. I will not elaborate or explain.
- "The Tyranny of Structurelessness" explains a hell of a lot about
modern life.
- If the problem is that powerful people are not accountable for their
actions, network forms of organization are the very opposite of solutions.
- Increasing returns $\Rightarrow$ monopolistic competition $\Rightarrow$
market failure explains a hell of a lot about modern life.
- Endogamy systematically confounds genetic and cultural inheritance. It
would be astonishing if personal-level variables which are entirely the product
of social structure, custom, tradition, and strategic action weren't
genetically predictable.
- During the 20th century, and in much of the world even today, genetic
variation in resistance to lead poisoning during brain development would,
psychometrically, look like a heritable general intelligence.
- Multiculturalists who expect different cultural groups to have different
values and standards of excellence should not expect those groups to be equally
represented in all occupations and professions, especially if people
are free to enter and leave different lines of work.
- There is a branch of the wave-function where Margaret Thatcher (trained
research chemist and honorary FRS) led a successful push by the international
community to address global warming through carbon pricing. (This probably
require postponing the collapse of the Soviet bloc by at least a few years, so
that her major
speech to the UN about the urgency of tackling global warming didn't happen
right before the Berlin Wall came down.) In that timeline, pollution charges
are regarded as one of the signature policies of neoliberalism.
- Modern civilization may be doomed because (i) we need to act
collectively and globally on climate change, (ii) the USA is the
indispensable nation at the center of the network, (iii) the Senate and
Electoral College give a veto over American policy to a combination of
plutocracy, reactionary folk Christianity and paranoia, and (iv) people like me
are unwilling to move to Wyoming, or even West Virginia, to counter this. (I
realize there is some tension between this and the previous statement.)
- A true Burkean conservative in America should work for the preservation
and development of the institutions bequeathed us by our wise and heroic
ancestors, serving us for half a century, a century, or more: the welfare
state; the regulatory agencies; international organizations and treaties; the
military-industrial complex; academia; affirmative action; Hollywood and the
rest of corporate popular culture.
- The quantitative social sciences would be in much better shape if the
first method everyone learned
was $k$-nearest-neighbors,
or maybe classification and regression trees, followed by
the bootstrap. Linear
models and $t$-tests should be, for social scientists, the
hyper-mathematical arcana at the back of the textbook which their methods class
skipped because there wasn't time.
- "Prophesying upon the eigenvectors" is more transparently about making stuff up than "interpreting the coefficients", and therefore preferable.
- There is something very odd about looking at the distribution of
node degrees within a network. It makes the network itself into an
odd hybrid of a single interdependent object (a relation) and a whole
population. Going "something something ergodicity mumble mumble sampling reveals a process grumble grumble shut up and calculate" makes this only a little less weird to me.
- A good philosophy of probability should explain why all applied Bayesian
statistics relies on Monte Carlo, the most frequentist method imaginable.
Ideally, it would also explain why Monte Carlo works with deterministic
pseudo-random number generators (so the algorithmic information content of the
pseudo-random numbers is extremely low).
- I have toyed for years with the phrases "the simulation interpretation
of probability" and "the birth of Trumpism out of the spirit of fandom", but I cannot work out what should stand behind them. (I hope they are unrelated.)
- Complexity science is a series of footnotes to Norbert Wiener and Herbert Simon.
- The mythopoetic impulse never goes away, but in populations which have
gone through
the Flynn/Gellner
transition to widespread clerk-ship, it expresses itself not as stories, but as
theories which aren't actually explanatory. (It gives us the Oedipus complex,
rather than Oedipus.) Every form of progress has its costs.
- Believing that there was a unique and irreversible transition from viewing
time as an eternal cycle to viewing time as unidrectional history is taking
sides.
- His ideas are flawed by a vicious circle; you are engaged in iterative approximation; I stand firm at a fixed point.
- Dentists are not, in fact, talking their book when they recommend
pre-emptive action for wisdom teeth.
- In a well-functioning social institution, incentives, information and
(internalized) norms ensure that it doesn't matter who fills various
roles in the institution, it behaves the same way regardless. Institutions are
always impersonal. It follows that institutions are information-hiding
abstractions, if we're thinking like programmers. But if we're thinking like
physicists: institutions are collective degrees of freedom.
- Blackouts are humanity confronting its own work, and the effects of its own collective actions, as a hostile and alien force.
- The parable of the Nazi bar is a folk version of the Schelling model.
- No one should be allowed to opine about artificial intelligence unless they've at least spent an hour or two with ELIZA and then stepped through the code.
- "Philip K. Dick and the Fake Humans" doesn't yet explain a hell of a lot about modern life, but it will.
Posted at September 28, 2025 12:32 | permanent link
|