November 13, 2018

Data over Space and Time, Lecture 20: Markov Chains


(.Rmd)

Corrupting the Young; Enigmas of Chance

Posted at November 13, 2018 16:50 | permanent link

November 12, 2018

Course Announcement: Advanced Data Analysis (36-402/36-608), Spring 2019

Attention conservation notice: Announcement of an advanced undergraduate course at a school you don't attend in a subject you don't care about.

I will be teaching 36-402/36-608, Advanced Data Analysis, in the spring.

This will be the seventh time I'll have taught it, since I took it over and re-vamped it in 2011. The biggest change from previous iterations will be in how I'll be handling class-room time, by introducing in-class small-group exercises. I've been doing this in this semester's class, and it seems to at least not be hurting their understanding, so we'll see how well it scales to a class with four or five times as many students.

(The other change is that by the time the class begins in January, the textbook will, inshallah, be in the hands of the publisher. I've finished adding everything I'm going to add, and now it's a matter of cutting stuff, and fixing mistakes.)

Advanced Data Analysis from an Elementary Point of View

Posted at November 12, 2018 14:51 | permanent link

Data over Space and Time, Lectures 18 and 19: Simulation for Inference


Lecture 18: The Bootstrap (.Rmd)

Lecture 19: The Method of Simulated Moments and Indirect Inference (.Rmd)

Corrupting the Young; Enigmas of Chance

Posted at November 12, 2018 13:55 | permanent link

November 03, 2018

In Memoriam Joyce Fienberg

I met Joyce through her late husband Stephen, my admired and much-missed colleague. I won't pretend that she was a close friend, but she was a friend, and you could hardly hope to meet a kinder or more decent person. A massacre by a deluded bigot would be awful enough even if his victims had been prickly and unpleasant individuals. But that he murdered someone like Joyce --- five blocks from where I live --- makes it especially hard to take. I am too sad to have anything constructive to say, and too angry at living in a running morbid joke to remember her the way she deserves.

Posted at November 03, 2018 14:25 | permanent link

November 01, 2018

Data over Space and Time, Lecture 17: Simulation

Lecture 16 was canceled.


(.Rmd)

Corrupting the Young; Enigmas of Chance

Posted at November 01, 2018 13:00 | permanent link

October 25, 2018

Data over Space and Time, Lectures 14 and 15: Inference for Dependent Data


Inference I: How Inference with Independent Data Really Works (.Rmd)

Inference II: Ergodic Theory and Inference under Dependence (.Rmd)

Corrupting the Young; Enigmas of Chance

Posted at October 25, 2018 19:40 | permanent link

October 18, 2018

Revised and Extended Remarks at "The Rise of Intelligent Economies and the Work of the IMF"

Attention conservation notice: 2700+ words elaborating a presentation from a non-technical conference about AI, where the conversation devolved to "blockchain" within an hour; includes unexplained econometric jargon. Life is short, and you should have more self-respect.

I got asked to be a panelist at a November 2017 symposium at the IMF on machine learning, AI and what they can do to/for the work of the Fund and its sister organizations, specifically the work of its economists. What follows is an amplification and rationalization of my actual remarks. It is also a reconstruction, since my notes were on an only-partially-backed-up laptop stolen in the next month. (Roman thieves are perhaps the most dedicated artisans in Italy, plying their trade with gusto on Christmas Eve.) Posted now because reasons.

On the one hand, I don't have any products to sell, or even much of a consulting business to promote, so I feel a little bit out of place. But against that, there aren't many other people who work on machine learning who read macro and development economics for fun, or have actually estimated a DSGE model from data, so I don't feel totally fradulent up here.

We've been asked to talk about AI and machine learning, and how they might impact the work of the Fund and related multi-lateral organizations. I've never worked for the Fund or the World Bank, but I do understand a bit about how you economists work, and it seems to me that there are three important points to make: a point about data, a point about models, and a point about intelligence. The first of these is mostly an opportunity, the second is an opportunity and a clarification, and the third is a clarification and a criticism --- so you can tell I'm an academic by taking the privilege of ending on a note of skepticism and critique, rather than being inspirational.

I said my first point is about data --- in fact, it's about what, a few turns of the hype cycle ago, we'd have called "big data". Economists at the Fund typically rely for data on the output of official statistical agencies from various countries. This is traditional, this sort of reliance on the part of economists actually pre-dates the Bretton Woods organizations, and there are good reasons for it. With a few notable exceptions, those official statistics are prepared very carefully, with a lot of effort going in to making them both precise and accurate, as well as comparable over time and, increasingly, across countries.

But even these official statistics have their issues, for the purposes of the Fund: they are slow, they are noisy, and they don't quite measure what you want them to.

The issue of speed is familiar: they come out annually, maybe quarterly or monthly. This rate is pretty deeply tied to the way the statistics are compiled, which in turn is tied to their accuracy --- at least for the foreseeable future. It would be nice to be faster.

The issue of noise is also very real. Back in 1950, the great economist Oskar Morgenstern, the one who developed game theory with John von Neumann, wrote a classic book called On the Accuracy of Economic Observations, where he found a lot of ingenious ways of checking the accuracy of official statistics, e.g., looking at how badly they violated accounting identities. To summarize very crudely, he concluded that lots of those statistics couldn't possibly be accurate to better than 10%, maybe 5% --- and this was for developed countries with experienced statistical agencies. I'm sure that things are better now --- I'm not aware of anyone exactly repeating his efforts, but it'd be a worthwhile exercise --- maybe the error is down to 1%, but that's still a lot, especially to base policy decisions on.

The issue of measurement is the subtlest one. I'm not just talking about measurement noise now. Instead, it's that the official statistics are often tracking variables which aren't quite what you want1. Your macroeconomic model might, for example, need to know about the quantity of labor available for a certain industry in a certain country. But the theory in that model defines "quantity of labor" in a very particular way. The official statistical agencies, on the other hand, will have their own measurements of "quantity of labor", and none of those need to have exactly the same definitions. So even if we could magically eliminate measurement errors, just plugging the official value for "labor" in to your model isn't right, that's just an approximate, correlated quantity.

So: official statistics, which is what you're used to using, are the highest-quality statistics, but they're also slow, noisy, and imperfectly aligned with your models. There hasn't been much to be done about that for most of the life of the Fund, though, because what was your alternative?

What "big data" can offer is the possibility of a huge number of noisy, imperfect measures. Computer engineers --- the people in hardware and systems and databases, not in machine learning or artificial intelligence --- have been making it very, very cheap and easy to record, store, search and summarize all the little discrete facts about our economic lives, to track individual transactions and aggregate them into new statistics. (Moving so much of our economic lives, along with all the rest of our lives, on to the Internet only makes it easier.) This could, potentially, give you a great many aggregate statistics which tell you, in a lot of detail and at high frequency, about consumption, investment, employment, interest rates, finance, and so on and so forth. There would be lots of noise, but having a great many noisy measurements could give you a lot more information. It's true that basically none of them would be well-aligned with the theoretical variables in macro models, but there are well-established statistical techniques for using lots of imperfect proxies to track a latent, theoretical variable, coming out of factor-analysis and state-space modeling. There have been some efforts already to incorporate multiple imperfect proxies into things like DSGE models.

I don't want get carried away here. The sort of ubiquitous recording I'm talking about is obviously more advanced in richer countries than in poorer ones --- it will work better in, say, South Korea, or even Indonesia, than in Afghanistan. It's also unevenly distributed within national economies. Getting hold of the data, even in summary forms, would require a lot of social engineering on the part of the Fund. The official statistics, slow and imperfect as they are, will always be more reliable and better aligned to your models. But, wearing my statistician hat, my advice to economists here is to get more information, and this is one of the biggest ways you can expand your information set.

The second point is about models --- it's a machine learning point. The dirty secret of the field, and of the current hype, is that 90% of machine learning is a rebranding of nonparametric regression. (I've got appointments in both ML and statistics so I can say these things without hurting my students.) I realize that there are reasons why the overwhelming majority of the time you work with linear regression, but those reasons aren't really about your best economic models and theories. Those reasons are about what has, in the past, been statistically and computationally feasible to estimate and work with. (So they're "economic" reasons in a sense, but about your own economies as researchers, not about economics-as-a-science.) The data will never completely speak for itself, you will always need to bring some assumptions to draw inferences. But it's now possible to make those assumptions vastly weaker, and to let the data say a lot more. Maybe everything will turn out to be nice and linear, but even if that's so, wouldn't it be nice to know that, rather than to just hope?

There is of course a limitation to using more flexible models, which impose fewer assumptions, which is that it makes it easier to "over-fit" the data, to create a really complicated model which basically memorizes every little accident and even error in what it was trained on. It may not, when you examine it, look like it's just memorizing, it may seem to give an "explanation" for every little wiggle. It will, in effect, say things like "oh, sure, normally the central bank raising interest rates would do X, but in this episode it was also liberalizing the capital account, so Y". But the way to guard against this, and to make sure your model, or the person selling you their model, isn't just BS-ing is to check that it can actually predict out-of-sample, on data it didn't get to see during fitting. This sort of cross-validation has become second nature for (honest and competent) machine learning practitioners.

This is also where lots of ML projects die. I think I can mention an effort at a Very Big Data Indeed Company to predict employee satisfaction and turn-over based on e-mail activity, which seemed to work great on the training data, but turned out to be totally useless on the next year's data, so its creators never deployed it. Cross-validation should become second nature for economists, and you should be very suspicious of anyone offering you models who can't tell you about their out-of-sample performance. (If a model can't even predict well under a constant policy, why on Earth would you trust it to predict responses to policy changes?)

Concretely, going forward, organizations like the Fund can begin to use much more flexible modeling forms, rather than just linear models. The technology to estimate them and predict from them quickly now exists. It's true that if you fit a linear regression and a non-parametric regression to the same data set, the linear regression will always have tighter confidence sets, but (as Jeffrey Racine says) that's rapid convergence to a systematically wrong answer. Expanding the range and volume of data used in your economic modeling, what I just called the "big data" point, will help deal with this, and there's a tremendous amount of on-going progress in quickly estimating flexible models on truly enormous data sets. You might need to hire some people with Ph.D.s in statistics or machine learning who also know some economics --- and by coincidence I just so happen to help train such people! --- but it's the right direction to go, to help your policy decisions be dictated by the data and by good economics, and not by what kinds of models were computationally feasible twenty or even sixty years ago.

The third point, the most purely cautionary one, is the artificial intelligence point. This is that almost everything people are calling "AI" these days is just machine learning, which is to say, nonparametric regression. Where we have seen breakthroughs is in the results of applying huge quantities of data to flexible models to do very particular tasks in very particular environments. The systems we get from this are really good at that, but really fragile, in ways that don't mesh well with our intuition about human beings or even other animals. One of the great illustrations of this are what are called "adversarial examples", where you can take an image that a state-of-the-art classifier thinks is, say, a dog, and by tweaking it in tiny ways which are imperceptible to humans, you can make the classifier convinced it's, say, a car. On the other hand, you can distort that picture of a dog into an image something unrecognizable by any person while the classifier is still sure it's a dog.

If we have to talk about our learning machines psychologically, try not to describe them as automating thought or (conscious) intelligence, but rather as automating unconscious perception or reflex action. What's now called "deep learning" used to be called "perceptrons", and it was very much about trying to do the same sort of thing that low-level perception in animals does, extracting features from the environment which work in that environment to make a behaviorally-relevant classification2 or prediction or immediate action. This is the sort of thing we're almost never conscious of in ourselves, but is in fact what a huge amount of our brains are doing. (We know this because we can study how it breaks down in cases of brain damage.) This work is basically inaccessible to consciousness --- though we can get hints of it from visual illusions, and from the occasions where it fails, like the shock of surprise you feel when you put your foot on a step that isn't there. This sort of perception is fast, automatic, and tuned to very, very particular features of the environment.

Our current systems are like this, but even more finely tuned to narrow goals and contexts. This is why they have such alien failure-modes, and why they really don't have the sort of flexibility we're used to from humans or other animals. They generalize to more data from their training environment, but not to new environments. If you take a person who's learned to play chess and give them a 9-by-9 board with an extra rook on each side, they'll struggle but they won't go back to square one; AlphaZero will need to relearn the game from scratch. Similarly for the video-game learners, and just about everything else you'll see written up in the news, or pointed out as a milestone in a conference like this. Rodney Brooks, one of the Revered Elders of artificial intelligence, puts it nicely recently, saying that the performances of these systems give us a very misleading idea of their competences3.

One reason these genuinely-impressive and often-useful performances don't indicate human competences is that these systems work in very alien ways. So far as we can tell4, there's little or nothing in them that corresponds to the kind of explicit, articulate understanding human intelligence achieves through language and conscious thought. There's even very little in them of the un-conscious, in-articulate but abstract, compositional, combinatorial understanding we (and other animals) show in manipulating our environment, in planning, in social interaction, and in the structure of language.

Now, there are traditions of AI research which do take inspiration from human (and animal) psychology (as opposed to a very old caricature of neurology), and try to actually model things like the structure of language, or planning, or having a body which can be moved in particular ways to interact with physical objects. And while these do make progress, it's a hell of a lot slower than the progress in systems which are just doing reflex action. That might change! There could be a great wave of incredible breakthroughs in AI (not ML) just around the corner, to the point where it will make sense to think about robots actually driving shipping trucks coast to coast, and so forth. Right now, not only is really autonomous AI beyond our grasp, we don't even have a good idea of what we're missing.

In the meanwhile, though, lots of people will sell their learning machines as though they were real AI, with human-style competences, and this will lead to a lot of mischief and (perhaps unintentional) fraud, as the machines get deployed in circumstances where their performance just won't be anything like what's intended. I half suspect that the biggest economic consequence of "AI" for the foreseeable future is that companies will be busy re-engineering human systems --- warehouses and factories, but also hospitals, schools and streets --- so to better accommodate their machines.

So, to sum up:

Manual trackback: New Savanna; Brad DeLong

The Dismal Science; Enigmas of Chance


  1. Had there been infinite time, I like to think I'd have remembered that Haavelmo saw this gap very clearly, back in the day. Fortunately, J. W. Mason has a great post on this.^

  2. The classic paper on this, by, inter alia, one of the inventors of neural networks, was called "What the frog's eye tells the frog's brain". This showed how, already in the retina, the frog's nervous system picked out small-dark-dots-moving-erratically. In the natural environment, these would usually be flies or other frog-edible insects.^

  3. Distinguishing between "competence" and "performance" in this way goes back, in cognitive science, at least to Noam Chomsky; I don't know whether Uncle Noam originated the distinction.^

  4. The fact that I need a caveat-phrase like this is an indication of just how little we understand why some of our systems work as well as they do, which in turn should be an indication that nobody has any business making predictions about how quickly they'll advance.^

Posted at October 18, 2018 23:30 | permanent link

Data over Space and Time, Lectures 9--13: Filtering, Fourier Analysis, African Population and Slavery, Linear Generative Models

I have fallen behind on posting announcements for the lectures, and I don't feel like writing five of these at once (*). So I'll just list them:

  1. Separating Signal and Noise with Linear Methods (a.k.a. the Wiener filter and seasonal adjustment; .Rmd)
  2. Fourier Methods I (a.k.a. a child's primer of spectral analysis; .Rmd)
  3. Midterm review
  4. Guest lecture by Prof. Patrick Manning: "African Population and Migration: Statistical Estimates, 1650--1900" [PDF handout]
  5. Linear Generative Models for Time Series (a.k.a. the eigendecomposition of the evolution operator is the source of all knowledge; .Rmd)
  6. Linear Generative Models for Spatial and Spatio-Temporal Data (a.k.a. conditional and simultaneous autoregressions; .Rmd)

*: Yes, this is a sign that I need to change my workflow. Several readers have recommended Blogdown, which looks good, but which I haven't had a chance to try out yet.

Corrupting the Young; Enigmas of Chance

Posted at October 18, 2018 22:49 | permanent link

September 25, 2018

Data over Space and Time, Lecture 8: Linear Prediction for Spatial and Spatio-Temporal Random Fields


(R Markdown source file)

Corrupting the Young; Enigmas of Chance

Posted at September 25, 2018 21:00 | permanent link

September 24, 2018

"Maximum Mean Discrepancy for Training Generative Adversarial Networks" (TODAY at the statistics seminar)

Attention conservation notice: Last-minute notice of a technical talk in a city you don't live in. Only of interest if you (1) care actor/critic or co-training methods for fitting generative models, and (2) have free time in Pittsburgh this afternoon.

I have been remiss in blogging the statistics department's seminars for the new academic year. So let me try to rectify that:

Arthur Gretton, "The Maximum Mean Discrepancy for Training Generative Adversarial Networks"
Abstract: Generative adversarial networks (GANs) use neural networks as generative models, creating realistic samples that mimic real-life reference samples (for instance, images of faces, bedrooms, and more). These networks require an adaptive critic function while training, to teach the networks how to move improve their samples to better match the reference data. I will describe a kernel divergence measure, the maximum mean discrepancy, which represents one such critic function. With gradient regularisation, the MMD is used to obtain current state-of-the art performance on challenging image generation tasks, including 160 × 160 CelebA and 64 × 64 ImageNet. In addition to adversarial network training, I'll discuss issues of gradient bias for GANs based on integral probability metrics, and mechanisms for benchmarking GAN performance.
Time and place: 4:00--5:00 pm on Monday, 24 September 2018, in the Mellon Auditorium (room A35), Posner Hall, Carnegie Mellon University

As always, talks are free and open to the public.

Enigmas of Chance

Posted at September 24, 2018 09:23 | permanent link

September 20, 2018

Data over Space and Time, Lecture 7: Linear Prediction for Time Series


(R Markdown source file)

Corrupting the Young; Enigmas of Chance

Posted at September 20, 2018 15:14 | permanent link

September 18, 2018

Practical Peer Review

Attention conservation notice: An exhortation to the young to demonstrate a literally-academic virtue which I myself find hard to muster.

Written a few years ago, and excavated from the drafts folder because I was preaching the same sermon in e-mail.

Having found myself having to repeat the same advice with more than usual frequency lately, I thought I would write it down. This is the importance of grasping, or really of making part of one's academic self, two truths about peer review.

  1. The quality of peer review is generally abysmal.
  2. Peer reviewers are better readers of your work than almost anyone else.

The first truth will speak to itself for any academic — or, if you're just starting out, trust me, it will soon. Drawing a veil over reports which mere products of nepotism and intrigue *, referee reports are often horrible. The referees completely fail to understand ideas we've adapted to the meanest understanding, they display astonishing gaps in their knowledge, and lots of them can't (as my mother puts it) think their way out of a wet paper bag. Even if you discard these as mere dregs, far too many of the rest seem to miss the point, even points which we've especially labored to sharpen. Really good, valuable referee reports exist, but they are vanishingly rare.

The second truth is perhaps even more depressing. Even making all allowances for this, your referees have (probably) read your manuscript with more attention, care, sympathy and general clue that most other readers will muster. In the first place: most papers which get published receive almost no attention post-publication; hardly anyone cites them because hardly anyone reads them. In the second place: if one of your papers somehow does become popular, it will begin to be cited for a crude general idea of what it is about, with little reference to what it actually says.

I hope readers will forgive me for illustrating that last notion with a personal reference. My two most popular papers, by far, are both largely negative. (I wish this were otherwise.) One of them might as well have been titled "So, you think you have a power law, do you? Well, isn't that special?", and the other "A social network is a machine for producing endogenous selection bias". Naturally, a huge fraction of their citations come from people using them as authorities to say, respectively, "Power laws, hell yeah!" and "I can just see peer effects". It's actually not uncommon for those papers to be cited as positively endorsing techniques they specifically show are unreliable-to-worthless. This has put me in the odd position, as an anonymous referee myself, of arguing with authors about what is in my own papers.

None of this should be surprising. One of my favorite books is one of the very few thoroughly empirical contributions to literary criticism, I. A. Richard's Practical Criticism. In an experiment lasting over several years in the 1920s, Richards took a few dozen poems, typed up in a uniform format and with identifying information removed, and presented them to literature students at Cambridge University, collecting their "protocols" of reaction to the poems. It is really striking just how bad the students were at receiving even the literal text of the poems, never mind providing any sort of sensible interpretation or reaction. And these were, specifically, students of literature at one of the premier institutions of higher learning in the world. As Richards said (p. 310), anyone who thinks their alma mater could do better is invited to try it **. Poems are not, of course, scientific papers, and I don't know of anyone who's done a translation of Richards's protocols to academic peer review. But I know of no reason to think highly-educated people are systematically much better at reading papers than poems.

The moral I would draw from this is not to seek a world without referees. It is this: whatever your referees find difficult, confusing or objectionable, no matter how wrong they might be on the merits, will give many of your other readers at least as much trouble. Since science is not about intellectual self-gratification but the advancement of public knowledge, this means that we have to take deep breaths, count backwards from twenty and/or swear, and patiently attend to whatever the referees complain about. If they say you're unclear, you were, by that very token, unclear. If they say you're wrong, you have to patiently, politely, figure out why they think that, and re-express yourself in a way which they will understand. Anger or sarcasm, however momentarily gratifying (and wow are they momentarily gratifying) will not actually change anyone's mind, and so they do not actually serve your long-term goal of persuading your readers of your conclusions.

"When the referees have a problem, there's a problem" is, quite literally, one of the most ego-destroying lessons of a life in science, but I am afraid it is a lesson, and the sooner it's absorbed the better.

*: Vanishingly rare, in my experience, but I am here to tell you that it does happen, and that posting an arxiv version with an inarguable date-stamp before you submit is always a good idea. ^

**: Admittedly, this was before access to higher education exploded after WWII, thereby driving up the average intellectual level of university students, but replications in the 1970s were not noticeably more encouraging. (I would be extremely interested in more recent replications.) ^

Learned Folly

Posted at September 18, 2018 23:25 | permanent link

Data over Space and Time, Lecture 6: Optimal Linear Prediction

In which we see how to use linear models without assuming that they are correct, or that anything at all is even remotely Gaussian.


(R Markdown source file)

Corrupting the Young; Enigmas of Chance

Posted at September 18, 2018 22:50 | permanent link

September 11, 2018

Data over Space and Time, Lecture 5: Principal Components Analysis II


(R Markdown source file)

Corrupting the Young; Enigmas of Chance

Posted at September 11, 2018 21:21 | permanent link

September 08, 2018

Reading _Capital_

Attention conservation notice: Over 14,000 words about an archaic monograph on political economy. Either you already care about it, in which case this adds nothing original, or you don't, in which it's vanishingly unlikely to spark an interest. Contains an extra 1920 words of quotations, a fair bit of linear algebra, and claims of being able to say what one of the great thinkers of western civilization meant better than he could. Finally, while I come from the sort of family where a great-uncle's middle name was, in fact, "Karl Marx", I am a mere squishy social democrat with liberal tendencies and a weakness for markets as tools for coordination and feedback, so I am at once too close for objectivity and too remote for involvement. Finally, I am, of course, neither an economist nor a historian of 19th century thought nor an expert on Marxism.

This was largely written over December 2015 and January 2016, and then left to the gnawing criticism of the mice. I post it now, because evidently it's time to clear out my drafts folder of ruminations on dead economists.

The other day, one of the occasional used-book dealers who comes by campus had, in addition to the usual collection of novels from the 1970s and Dover books on mathematics, a stout little ex-library hardback of Capital, volume I1. It was (perhaps appropriately) virtually free, and so I found myself moved to buy it, and then to re-read it, for the first time since I was a teenager. I will not pretend this is anything like a serious contribution to scholarship on Marx, something which already fills libraries.

Before plunging in, I should perhaps say that while I couldn't resist my title, what follows fortunately owes nothing, consciously, to Althusser.

Part I: Commodities and Money

Chapter 1: Commodities

Section 1: The Two Factors of a Commodity: Use Value and Value

Like everyone else, Marx recognizes that a commodity can have value or worth in two ways: it can be directly useful to human beings in some way, or it can be exchanged for something else. Marx wants to call the latter just "value"; this seems a bad idea, but I'll go along with it.

Marx does not explain why "these two commodities are equally valuable" must mean "these two commodities share some other common property", any more than "these two books contain the same number of words" does. Marx also does not explain why that common property must be "equal amounts of abstract, homogeneous labor are, at the present time, socially necessary to make them". The closest he comes is a footnote about that being the only logical way for neither party in an exchange to come out behind, in a way which suggests an opportunity-cost notion lying just out of reach.

Marx tries to clarify the "socially necessary" part of "socially-necessary labor time" by example of a new power loom reducing the time needed to make a given quantity of cloth, hence reducing the value of existing cloth. There is an interesting subtlety here, which is actually related to the problem that led Kantorovich to linear programming. If initially society has only one power loom, and more cloth is needed than that loom can supply, isn't the more laborious process also socially necessary? Would the value of cloth be set by some sort of average, or what? (I return to this problem below.)

The rest of the book does nothing to provide a better defense of the labor theory of value.

Section 2, The Twofold Character of the Labour embodied in Commodities

Section 3, The Form of Value, or Exchange Value

Marx's logical tools for reasoning about relations were totally inadequate. In this he was like absolutely everybody else until the very end of the 19th century, when those tools began to be developed by Peirce, Schroeder, etc. (In fairness, in sub-section B, "total or expanded form of value", Marx comes close to grasping what we'd now call an equivalence class. In further fairness, he doesn't actually grasp it.) To try to clear this up: "is just valuable as", or "is fairly exchangeable with", is a symmetric and transitive relation on bundles of commodities. If we extend it to also be reflexive, we obtain an equivalence relation, which, like any equivalence relation, partitions the space of bundles of commodities into equivalence classes. Any member of an equivalence class can of course be picked out and used as a representative of this class2. As a matter of fact, not logic, every one of those equivalence classes contains a representative which contains only a certain amount of a single commodity --- so many coats, or so many yards of linen, etc. (Strictly speaking, this pre-supposes that there is at least one commodity available in continuous amounts, unlike coats.) Moreover, these equivalence classes can be total-ordered: any two bundles of commodities can be compared in value, and if one set isn't worth more than the other, then the two sets belong to the same equivalence class. This "more valuable than" relation can be represented by the ordinary ordering on the numerical amount of any one pure commodity. Pretty much everything in sections 2, 3 and 4 that isn't wrong could be summed up in a page or two of math --- now; not in Marx's day.

Marx realizes that labor differs in regards to (for instance) skill, intensity, strength, etc., and that if he wants to have a labor theory of value, those all have to be equated somehow, reduced to some sort of common-denominator abstract, homogeneous labor. He recognizes, that is to say, that there is an issue here, but his solution is, in almost so many words, that the market takes care of that3.

Section 4: The Fetishism4 of Commodities

People act as though the (exchange) value of commodities was a relationship between commodities, when it's really a social relationship between their producers, or between their producers and the total labor-power of society. If production and distribution were organized differently, there would still be labor and labor time, but not commodities with exchange values proportional to socially-necessary labor time. As examples, he gives Robinson Crusoe, "patriarchal" (self-sufficient) families, payment of feudal dues in kind, and the future society of freely-associated producers, where everyone contributes labor and receives a sort of coupon proportional to their labor contribution, for a share of the total product not devoted to reproducing and expanding the productive apparatus.

The last bit is one of the handful of places in Capital where Marx says anything about what the post-capitalist economy is supposed to be like. (Admittedly, the point of the book is to explain the political economy of capitalism, not post-capitalism.) The difference between those coupons and money eludes me, unless the idea is that everyone gets an exactly pro-rata share of every good, in which case I predict tears, and the emergence of barter, or rather a de facto money, before bedtime. (My neighbors with the newborn needs more diapers than I do, they have no use for books on stochastics and political economy, neither they nor I can put away as much beer as Uncle Karl, etc.5) If, on the other hand, the coupons are basically money, then there need to be prices, denominated in coupons, on the goods. And so I fail to see how we won't still have commodity production and, to anticipate some later material, how there won't still be a gap between the length of the working day, i.e., value provided by a worker's labor power, and the value of the worker's coupon wages. The gap might be shrunk to the minimum necessary for the reproduction and expansion of society, without skimming off extra to support capitalists and their parasites, but Marx seems to be aiming at a bigger qualitative difference, and not getting there.

I see why this section on fetishism has been so influential, partly because Marx is getting at something important, and partly because the attempted de-mystification is itself pretty mysterious. A certain amount of mystery helps any ideology; it provides the moment of "click". Further commentary on this point is out-sourced to Ernest Gellner's "Notes Towards a Theory of Ideology", and to Raymond Boudon.

— If you want to play "people think \(X\) is a property of things, but really it's a social relationship in disguise" games, neo-classical economics has got at least as much to offer as Marx. Prices confront each individual price-taker as objective facts, but --- so goes the Walrasian myth --- they are in fact the outcome of a social process, reflecting at once the possibilities of production, the social facts of initial endowments, and the purely subjective wants of everyone. There isn't even, in the neo-classical view, something like "socially-necessary labor time" to ground prices in. Indeed, in neo-classical models, prices can be incredibly different, with exactly the same productive possibilities and desires, merely as a function of initial endowments with goods and money. One might say that Debreu's Theory of Value was the first mathematically rigorous contribution to theories of the social construction of reality.

Chapter 2: Exchange

People make things they don't want, in order to sell them for money, which they use to buy things they do want; since they start with a commodity, get money, and end with a commodity, Marx writes this C-M-C. He depicts this as a "metamorphosis" of the original commodity into a new commodity. This illuminates nothing, but serves the rhetorical function of writing (I paraphrase) "no one makes a profit on trading in a competitive market" in big block letters and underlining over and over. ("Profit" of course in terms of exchange value; Marx is quite clear that people only exchange at all because they prefer the use-value they get to the use-value they give up.)

Chapter 3: Money

Marx re-capitulates the usual origin myth of money, from barter to a commodity serving as numeraire to money. (The "double coincidence of wants" doesn't appear, but the idea that barter is very inconvenient certainly does.) It is important for Marx that money itself have value, and so be something which is itself the product of labor; metals, like gold and silver, are extra handy because they are durable, divisible, and (can be made) homogeneous, thereby representing abstract, homogeneous labor. Indeed, the value of the metal is, precisely, the quantity of socially-necessary labor required to produce it.

Marx states a basic form of the quantity theory of money: the amount of circulating money \(M\) in a country is determined by \[ M = \frac{\sum_{i}{p_i Y_i}}{v} \] where \(p_i\) is the money price of the ith commodity, \(Y_i\) is the amount of that commodity bought and sold per unit time, and \(v\) is the velocity of money, the number of transactions each coin goes through per unit time. Several paragraphs are given over to verbal descriptions of what happens when each of these terms moves up or down. (Here I do have to fault him, algebra was a thing in 1867, and there could have been few readers who found those paragraphs easier going than a few symbols.) He correctly notes that it's not necessary for all prices to go up in order for the quantity of circulating money to have to increase.

I deliberately wrote this equation with \(M\) by itself on one side, since Marx is quite insistent that changing the volume of circulating currency in the country cannot alter prices. (He explicitly takes issue with Hume on this point.) One might even go so far as to write it with an arrow for causal dependence, thus: \[ M \leftarrow \frac{\sum_{i}{p_i Y_i}}{v} \] I did not catch his argument for why this must be the case, especially since in a footnote he says that "bungling legislative interference" with \(M\) can induce stagnation of trade. I kind of get why he wants this to be true --- it goes along with the general labor-theory-of-value idea, if sheer quantity of money could drive prices that would be bad --- but I didn't see an actual argument. It's true that money could be hoarded rather than circulating, so, even with a metallic currency, \(M\) can't be identified with the sheer amount of gold in the country, but denying that changing \(M\) could change the \(p\)'s is just weird.

(Later on, Marx modifies this quantity theory of money to allow for credit and repayment of credit, but it doesn't alter the points above.)

— What Marx would have made of our current system, where our money has been completely cut free from any sort of commodity backing, and no more labor is required to make a \$ 100 bill than a \$ 1 bill (or to record "\$ 100" in an electronic account than "\$ 1"), is an interesting question. It is, indeed, something I am ashamed of not having taxed my Marxist friends with before. (Obviously having non-commodity money doesn't meant that the exchange ratios of commodities couldn't equal the ratio of their labor-contents, but it does create a problem when you want to talk about exchanging a commodity for money or money for commodities, the C-M and M-C parts of a "circuit".) The best answer my inner Marxist can come up with, absent any actual research, is this: a dollar bill, a credit card, etc., only function as money because a large set of inter-locking organizations and less formal institutions make them effective: it's not just a matter of printing the bill or the card, but the whole system of banking and finance which makes sure everyone recognizes them as means of payment, that transactions balance, that keeps track of who has which claims on money, that banks don't (mostly) just go belly up, etc., etc. The value of our current money (my inner Marxist continues) derives from the very considerable amount of labor required to keep these institutions operating and effective. (You could even use this to explain inflation, by arguing that the "productiveness" [see below] of the monetary system rises faster than the productiveness of the rest of the economy, hence money prices must rise.) That still doesn't answer why a \$ 100 bill represents a bigger share of the socially-necessary labor of the monetary system than does a \$ 1 bill, though.

Part II: The Transformation of Money into Capital

Chapter 4: The General Formula for Capital

Chapter 5: Contradictions in the General Formula of Capital

Chapter 6: The Buying and Selling of Labour-Power

Money becomes capital, rather than merely being hoarded, when it is used to make more money, i.e., when someone uses it to buy a commodity, and then sells that commodity again, which he writes M-C-M'. Marx follows Aristotle in regarding this as somewhat unnatural, and money-lending at interest as even more unnatural, as opposed to merely using money to lubricate exchange. (The Aristotlean inspiration is quite explicit.)

Problem: You can't make any profit in a fair exchange (that was the whole point of chapter 2), and you can't generally make profits by coming out ahead in unfair exchanges. (As we'd now say, that's at most zero-sum.) So where does "surplus value" come from?

Solution: what a laborer sells is their labor power, the use of their productive abilities, typically for a certain period of time; the capitalist can come out ahead if the value of labor power is less than value added by the labor. E.g., if the normal working day is 12 hours but the value of the labor power for that day is only 6 hours, the capitalist can get 6 hours of surplus value out of this.

The value of labor power is set, like that of any other commodity, by the quantity of average labor time socially necessary for its production, or rather reproduction, i.e., whatever's required to produce what workers typically require, given the climate and customs of their country, by way of food, clothing, fuel, housing, training, etc., to maintain themselves in work and to raise the next generation of workers.

Part III: The Production of Absolute Surplus-Value

Chapter 7: The Labour Process and the Process of producing Surplus-Value

Let's temporarily fix the length of the working day (or week or other period) at so many hours. The value of the labor power for that period is either greater than, equal to, or less than that number of hours. (Remember, value has units of labor time.) If the value of labor power is greater than the working period, everyone's in trouble: average workers don't produce enough to reproduce themselves during the period they work. If the length of the working day exactly equals the value of labor power, then the workers can, just, reproduce themselves. This means, however, that there is no point in hiring workers to make money. (Or so says Marx; but I wonder if it mightn't make sense to pay to hire the above-average workers.) Finally, if the length of the working day is longer than the value of a day's labor power, there is, potentially, a surplus. That difference in time is the surplus value produced by the worker. (Again, exchange value is measured in units of necessary labor time.)

The problem of equating different sorts of labor is raised again at the end of this chapter, and again Marx basically says that the market takes care of this. Indeed, the footnote pointing out that what counts as "skilled" and "unskilled" is often really just the distinction between what is and isn't in short supply, and hence able to command high wages, while it has merit, would seem to undermine the case for taking quantity-of-average-labor as fundamental. (I note in passing that, e.g. David Autor's recent paper, purportedly tracking how wages have changed for occupations with different levels of skill, just re-defines "skill" as the wage rates prevailing in 1979. This is done without any comment, or even the cross-check of seeing if the same results hold if a different year is used as the reference point.)

Chapter 8: Constant Capital and Variable Capital

It is actually more convenient for me to give the definitions of these terms below, under chapter 9.

Towards the end of this chapter: after a bad harvest, a given weight of cotton represents more labor than after a good one. How on Earth does this follow? If the harvest is bad because (e.g.) of drought or flood or disease, which affect only some regions but not others, the total harvest might be cut in two, but no more work expended on the surviving part than on the part that didn't make it. The best I can do is something like "well, we wouldn't even have that half of the harvest unless we had gone through all the work of the complete harvest", which might sometimes be true but is hardly a universal rule.

Chapter 9: The Rate of Surplus-Value

The absolute amount of surplus value which results from production is the difference \(s\) between the value of the output, and the value \(C\) of the inputs. The value of inputs consists of (1) "variable capital" \(v\), i.e., the value of the labor power hired, and (2) "constant capital" \(c\), so \(C=v+c\). Constant capital \(c\) in turn consists of (a) the value of the raw materials and supplies consumed, and (b) wear and tear, or amortization, on what we'd call capital goods (tools, machinery, buildings for carrying on work, fuel, etc.). Relative surplus value is defined as \(s/v\), the ratio of surplus value to variable capital alone. I note that these are flows rather than stocks --- the total value of the capital goods, and so (e.g.) set-up costs, does not figure into Marx's calculation (except perhaps through amortization).

Marx thinks it is very wrong to consider (say) the ratio \(s/C\) of surplus value to total capital charges. Only \(s/v\) is legitimate. I honestly can't fathom why, intellectually, this should be the case; for propaganda purposes however it seems pretty clear that he wants to say that if a coat has a value of \$ 20, of which \$ 10 is constant capital, \$ 5 is labor and \$ 5 is surplus value, the "rate of exploitation" is 100%, not 25% or even 33%. But why one should care about this "rate of exploitation", so defined, is again unclear. It's true that \(s/(s+v)\) gives (according to the labor theory of value) the fraction of the working day during which the laborer's efforts go purely to the benefit of the capitalist, and that, algebraically6, this can also be deduced from \(s/v\), and Marx makes much of this, but, again, why? For capitalists, labor power is just another input commodity, and they should care about the ratios of profits to total outlays. (Incidentally, it is striking in these theoretical chapters just how much Marx leans on constant returns to scale.) For laborers, surely the important variable is just \(v\) --- really it's wages, which is something else altogether (Part VI below).

Aside: Constant Capital and "Socially Necessary Labor Time"

There is an important, but largely un-heralded, interpretive point about the labor theory of value implicit in this chapter. This occurs at the very beginning, when it's asserted that the value of the goods produced is not \(s+v\), which is the quantity of labor used in this stage of production, but \(c+v+s\), which adds in the labor value of all of the material inputs, including the amortization and depreciation of capital goods7. This means that the phrase "social necessary labor time" has to be interpreted in a very special sense, as including all the labor time which went into making the input materials, some share of the labor time which went into making tools, machines and other means of production, and so on transitively for their production as well. The value of any commodity is thus the sum of a series which has, if not actually infinitely many terms, then at least has no real bound on their number.

Still, as I tell my students, one person's vicious circle is another's iterative approximation, and iterative approximations, when they work, converge on fixed points. The fixed point is actually something we can work out here, at least post-Leontieff8. Say that producing one unit of commodity \(i\) takes \(a_{ij}\) units of commodity \(j\) as inputs. (This includes capital goods, which get treated on the same basis as everything else.) Collect these into a matrix \(\mathbf{a}\), of dimension, say, \(k \times k\). Producing one unit of commodity \(i\) also directly requires \(v_i\) hours of abstract homogeneous labor, which we collect into the \(k \times 1\) matrix \(\mathbf{v}\). Self-consistency demands that the \(k \times 1\) matrix of socially-necessary labor times \(\mathbf{l}\) satisfy \[ \mathbf{l} = \mathbf{v} + \mathbf{a}\mathbf{l} \] or \[ \mathbf{l} = (\mathbf{I} - \mathbf{a})^{-1} \mathbf{v} \]

Expanding \((\mathbf{I}-\mathbf{a})^{-1}\) in a power series, \[ \mathbf{l} = \mathbf{v} + \mathbf{a}\mathbf{v} + \mathbf{a}^2\mathbf{v} + \ldots \] So the labor embodied in one unit of good \(i\) is, indeed, equal to the direct labor needed to make it, plus the labor directly needed to make the raw materials and means of production for good \(i\), plus the labor needed for their raw materials and means of production, etc. (If this doesn't exactly redeem my joke about iterative approximation, it at least makes it more literal.) We can thus extract the vector of constant capitals \(\mathbf{c}\) as \[ \mathbf{c} = \sum_{n=1}^{\infty}{\mathbf{a}^n \mathbf{v}} = \left( (\mathbf{I} - \mathbf{a})^{-1} - \mathbf{I}\right) \mathbf{v} \] We'll come back to this formula later. For now, I just want to high-light that all of the implausible assumptions (constant returns to scale, no unproduced means of production such as land, etc.) involved in this derivation are ones explicitly made by Marx. It is implied by his assumptions, and if you don't like it, you need to assume something else.

— I want to make it very clear that I claim no originality for this little bit of linear algebra; I won't try to trace the history, but it goes back at least to the 1960s. I can't remember if I learned it from my father, or just from his books.

Chapter 10: The Working Day

Being a worker in 19th century England was appalling. One of the reasons it was so appalling was the length of the working day, which employers did everything possible to extend. Every effort at limiting the length of the working day, or even to collect systematic information, was met with immense resistance, flouting of the laws, and proclamations of doom. This chapter lets Marx unleash his savage indignation, and equally savage sarcasm, on targets who fully deserved every particle of his venom.

Chapter 11: Rate and Mass of Surplus Value

Surplus value adds up across workers.

Part IV: Production of Relative Surplus-Value

This part has a very odd structure. The first chapter in it, 12, is a continuation of the stuff about surplus value from the previous part. The rest of it is a historical survey of the development of the forces of production, and with them the social relations of production, from the most basic cooperation with minimal tools to "modern industry", i.e., powered automatic machinery. The connection between the first chapter and the rest is frankly weak. Marx claims that he'll show all these developments were drive by to increase the fraction of the work day going to surplus value, which he never really delivers on.

Chapter 12: The Concept of Relative Surplus-Value

Recall that surplus value is the difference between the value produce by employing labour-power, and the value of the labour-power itself. Further recall that the value of labour-power is the quantity of labor, in the form of commodities, socially necessary for the worker to keep working and to bring up the next generation. If a capitalist wants to increase their surplus value, they can either lengthen the working day ("absolute surplus value"), or they can reduce the value of labor-power ("relative surplus value"). The latter requires increasing the productiveness of labor, so that less time is needed to produce the commodities which labor-power needs to reproduce itself.

Chapter 13: Cooperation

The "productiveness"9 of labor increases when workers cooperate, even if they don't specialize. Think of many people lifting a large weight together which none of them individually could shift, or being able to bring in a harvest before anything spoils, etc. Sometimes the gains from cooperation arise because the laborers can share equipment, even if it's just a work-place. Additional increases to productiveness can follow from even elementary division of labor (think of a bucket brigade for moving water from a well), as well as more refined ones.

Cooperation often needs people to organize it; when cooperation is based on shared equipment, someone might own it. These are the most basic ways in which the capitalist can gain an entry into the process of production, as the equipment owner and organizer of cooperation. Reading between the lines here a little, Marx doesn't (or at least shouldn't) see a problem with what economists would now call "factor payments to capital", i.e., putting some of the product aside to pay for wear and tear, or even for improvements. He shouldn't even, by his own rights, object to paying someone for doing the work of organization, since that would be part of the socially-necessary labor process. That capitalists, because they own the means of production, should get to claim the whole of the surplus product, that is the fundamental sticking point.

Chapter 14: Division of Labour and Manufacture

By "manufacture", Marx means the systems of refined division of manual labor within single trades --- the sort of thing famously celebrated in the opening of The Wealth of Nations. (Marx has some rather catty footnotes about how un-original Smith's passage was.) Something which might, before, have been done by a single artisan is instead broken up into a multitude of small steps. Each step gets its own crop of "detail-laborers", who do just that, often with specialized tools. This leads to savings of time (since the same laborer isn't switching tasks), and increase of skill, and hence increasing productiveness for the system as a whole. But, says Marx, it cripples the detail-laborers intellectual and often physically. (I wonder whether the detail laborers would have agreed about the former.)

Marx is at pains to draw out a couple of points here:

  • While division of labor is ancient, the manufacturing system had a definite historical period, which he dates "from the middle of the 16th century to the last third of the 18th century";
  • Manufacturing was not pursued because it increased productiveness or enhanced use values (as in the ancient arguments for the division of labor, going back to Plato if not before), but because it reduced costs and increased profits;
  • Being a detail laborer was no fun at all;
  • The manufacturing system had definite limits, precisely because it depended on manual labor.

Chapter 15: Machinery and Modern Industry

This is a really magnificent historical analysis of the effects of introducing machinery into production, especially when that machinery is driven by in-organic power sources. Marx begins with distinguishing a "machine" from a "tool" . He quotes some definitions which say the difference is that a "tool" is powered by a human body, a "machine" by something else. This he rejects as unsatisfying (fairly enough; it makes an ox-drawn plow a machine, but the same plow drawn by the farmer a tool). Rather, he says, what makes a machine is that the actual effective instruments are not wielded by a human being, but by the contraption. (He quotes Babbage favorably on this point.) Whether the ultimate motor is a human body or something else is beside the point. Once you've built the machine, substituting one form of power for another might be a tricky technical feat, but it is just a technical feat, and the sort of thing which technologists get very good at.

The tools wielded by machines generally begin with the detail-labor tools developed by the manufacturing system, rather than the less specialized ones of handicrafts. But they then adapt to the conditions of mechanical use, often but not always by multiplying. The actual laborer's role, since it is not to wield the instruments, is to supervise the machine, correct its faults, and bridge gaps between mechanical processes. The progress of what we would now call automation comes in here, moving more and more processes into the ambit of the machine, and reducing the number of mistakes which the human operative must correct.

Every form of mechanized industry demands a specialized division of labor, but it is a transient division, one which is going to constanty be changed by the progress of technology. Thus it demands an education which suits its recipients to turn their hands to many different kinds of jobs over the course of their life10. Cf. Gellner's Nations and Nationalism, and for that matter every education writer for at least a century.

Recouping the fixed costs of machinery, especially set-up costs, pushes capitalists towards continuous production --- hence an additional reason for lengthening the working day, for night work, etc.11 It also pushes towards the lower possible labor costs, so employing women and children, especially because muscle power is less important; and it pushes towards intensifying the working day if lengthening it isn't possible.

Marx realizes that it's possible to mechanize and industrialize agriculture and the countryside; he thinks that if this is done by capitalists, it will have bad long-term consequences for the soil. It's a little hard to for me to grasp exactly what he's getting at, but I think he's worried about disrupting chemical cycles, e.g. of carbon or nitrogen12. Why there couldn't be capitalist businesses collecting "consumed ... food and clothing" from towns and re-processing them into fertilizer, he doesn't say.

Part V: The Production of Absolute and Relative Surplus-Value

Chapter 16: Absolute and Relative Surplus Value

Chapter 17: Changes of Magnitude in the Price of Labour-Power and in Surplus-Value

I realize I'm sounding like a broken record, but really, would it have killed Marx to have used a little algebra, and spared us all this prose? Let \(t\) = length of the working day (units: hours), \(g\) = the value of goods required for reproduction of average labor (units: hours), \(\rho\) = productiveness of average labor at standard intensity of labor (unitless), with \(i =\) that intensity, typically 1 (also unitless). Then the value of labor power \(v = \frac{g}{\rho i}\), while \(s = t-v\) is the absolute surplus value, and \(s/v = \frac{t}{v} - 1\) is the relative surplus value (per Marx's definition). Thus \[ s = t - \frac{g}{\rho i} \] and \[ \frac{s}{v} = \frac{t}{g}\rho i - 1 \] All of the "laws" deduced here follow immediately.

(In passing: surely Marx is right that the intensity of labor matters, but how does this fit with his stated position of reducing everything to average labor? For that matter, how, non-tautologously, might we measure "intensity of labor" \(i\)? It would seem to be hopelessly confounded with productiveness \(\rho\).)

In these chapters, "rate of profit" is defined as the ratio of surplus value to working capital \(s/(c+v)\). (It's possible he also introduced the concept earlier and I missed it.) To motivate this, remember that the labor value of the product includes the labor value \(c\) of the raw materials and means of production used up in making it. So the total value of goods produced is \(s+c+v\), while the value of the inputs is \(c+v\), and so the profit is exactly the surplus value \(s\). Since the numerator is a flow (surplus \(s\)), it makes some amount of sense for the denominator, too, to be a flow (variable capital or wages \(v\) and constant capital or inputs and wear-and-tear \(c\)). It is thus not to be confused with our modern notion of "rate of return on investment", where the denominator is a stock of capital. Whether capitalists will care about rate of profit, in Marx's sense, or rate of return on investment, would seem to depend on how important start-up costs are.

Part VI: Wages

Since reproducing labor power requires goods (and services) which are themselves products of labor, labor power has a value. Since money requires labor to produce, it has a value. Therefore labor power has a price in money. This price is wages. Whether it's expressed as a price per unit time, or a price per unit output, is secondary; piece rates will get adjusted to reflect the productiveness of labor.

There are a lot of gyrations in these chapters to get around the fact that the value of labor power doesn't tell us much about real wages, "i.e., the means of subsistence placed at the disposal of the labourer".

Part VII: The Accumulation of Capital

Chapter 23: Simple Reproduction

Chapter 24: Conversion of Surplus-Value into Capital

Any society needs to reproduce its means of production if it's to go on. This means that some fraction of production must not be consumed in the usual way, but go towards at least replacing old means of production. (Though I do not think Marx says it in so many words, this would correspond to the "constant capital" portion of the total [working] capital.) In simple reproduction, society just replaces its means of production, without expanding them. This would leave the surplus value available for the consumption of the capitalist. Part of reproduction is providing the labor-force with the means of subsistence, which are of course things produced by the laborers; under capitalism, this takes the form of the variable capital or wages, but it is always, under any society, about workers collectively producing what they (and their families) need. This takes other forms under other modes of production, and Marx hints at the later discussion of "primitive accumulation", i.e., how capitalism could have gotten started from other modes of production/reproduction. (How social reproduction will work after capitalism is not discussed.)

Marx says that if the surplus value is \$ 100 a year on a capital of \$ 1000, after ten years the capital consists entirely of surplus value, even if the surplus value is entirely consumed each year. (I have changed his pounds to dollars to suit my keyboard.)

If a capital of \$ 1000 beget yearly a surplus-value of \$ 200, and if this surplus-value be consumed every year, it is clear that at the end of 5 years the surplus-value consumed will amount to \(5 \times \$ 200\) or the \$ 1,000 originally advanced. If only a part, say one half, were consumed, the same result would follow at the end of 10 years, since \(10 \times \$ 100= \$ 1,000\). General Rule: The value of the capital advanced divided by the surplus-value annually consumed, gives the number of years, or reproduction periods, at the expiration of which the capital originally advanced has been consumed by the capitalist and has disappeared.

I frankly do not follow his argument at all:

The capitalist thinks, that he is consuming the produce of the unpaid labour of others, i.e., the surplus-value, and is keeping intact his original capital; but what he thinks cannot alter facts. After the lapse of a certain number of years, the capital value he then possesses is equal to the sum total of the surplus-value appropriated by him during those years, and the total value he has consumed is equal to that of his original capital. It is true, he has in hand a capital whose amount has not changed, and of which a part, viz., the buildings, machinery, &c., were already there when the work of his business began. But what we have to do with here, is not the material elements, but the value, of that capital. When a person gets through all his property, by taking upon himself debts equal to the value of that property, it is clear that his property represents nothing but the sum total of his debts. And so it is with the capitalist; when he has consumed the equivalent of his original capital, the value of his present capital represents nothing but the total amount of the surplus-value appropriated by him without payment. Not a single atom of the value of his old capital continues to exist.

This seems, at best, like moralizing about what workers are owed, and not value accounting.

What makes rather more sense to me is the point that with compound interest, any initial capital that might have been accumulated by personal savings and effort is swiftly overwhelmed by capitalized surplus value.

Also more sensible are the two following points:

  1. Taken as a whole, the working class produces, through its labor, all its own means of subsistence. What this looks like, however, is that the laborers of any one capitalist exchange their wages for (for the most part) the goods sold by other capitalists.
  2. If the market works and laborers receive the value of their labor power, they are just able to reproduce themselves. To stay alive, they must therefore sell that labor power again.

The last point is worth emphasizing. Once someone becomes a laborer, for Marx, they have to pretty much stay one: they enter the process with nothing to sell except their labor power, and they get just the value which lets them keep on selling their labor power. They are trapped. There might, of course, be accidents which let some few individuals save enough to become capitalists, but they cannot be general.

Chapter 24 specifically focuses on the accumulation of capital, i.e., capitalists not consuming all of the surplus value, but converting (some of) it back into more capital used for production, both as constant capital (means of production) and variable (wages). Marx, quite correctly in my view, sees this as the fundamental thing which both distinguishes capital from earlier modes of production, and makes it (temporarily) progressive:

Except as personified capital, the capitalist has no historical value... And so far only is the necessity for his own transitory existence implied in the transitory necessity for the capitalist mode of production. But, so far as he is personified capital, it is not values in use and the enjoyment of them, but exchange-value and its augmentation, that spur him into action. Fanatically bent on making value expand itself, he ruthlessly forces the human race to produce for production's sake; he thus forces the development of the productive powers of society, and creates those material conditions, which alone can form the real basis of a higher form of society, a society in which the full and free development of every individual forms the ruling principle. Only as personified capital is the capitalist respectable. As such, he shares with the miser the passion for wealth as wealth. But that which in the miser is a mere idiosyncrasy, is, in the capitalist, the effect of the social mechanism, of which he is but one of the wheels. Moreover, the development of capitalist production makes it constantly necessary to keep increasing the amount of the capital laid out in a given industrial undertaking, and competition makes the immanent laws of capitalist production to be felt by each individual capitalist, as external coercive laws. It compels him to keep constantly extending his capital, in order to preserve it, but extend it he cannot, except by means of progressive accumulation.

And again, famously:

Accumulate, accumulate! That is Moses and the prophets!

This is followed by some rather acid remarks on the understandable tendency of individual capitalists to not want to subordinate everything to accumulation, but rather to enjoy themselves, and the idea that savings is "abstience" from consumption.

Going back just a little, however, I do not think Marx gives an adequate theoretical explanation for why accumulation is so vital to capitalism. Or, rather, he does have one, in two parts, but it fits awkwardly with the rest of his theory. Part 1 of the explanation is that

the development of capitalist production makes it constantly necessary to keep increasing the amount of capital laid out in a given industrial undertaking
and part 2 is that
competition makes the immanent laws of capitalist production to be felt by each individual capitalist, as external coercive laws

I think it's fairly easy to make sense of this, if one allows for substantial economies of scale. The argument would go like this:

  1. The larger the scale of production, the lower the cost at which it can be profitably sold.
  2. Firms which cannot match the lowest selling-price currently offered will tend to exit the market, either because they lose money, or lose market share, or both.
  3. Therefore, the surviving firms will have the same costs of production, which will all be the lowest currently feasible.
  4. A firm which re-invests some of its surplus can expand scale and lower its cost of production, so every other firm must match it or exit the market.

Of course, this presumes that the process of selection-by-competition invoked in (2) and (4) is quite fast, since otherwise it's an idle in-the-long-run-the-inefficient-are-dead observation. But leave aside Marx's reliance of the efficiency of competition. (In his own way, he really believed in capitalism.) This is yet another place where Marx makes sense, if we assume increasing returns to scale, but his value theory presumes constant returns to scale.

Marx may also have had in mind an argument about technical change:

  1. A change in technique which increases the productiveness of labor reduces the value of a commodity, because it reduces the labor time socially necessary to produce it. (Cf. the bit about the new power loom back in chapter 1.)
  2. Firms which can only produce a commodity by using more than the socially necessary amount of labor will exit the market.
  3. Generally, adopting the new, productiveness-enhancing technique will have costs, which will have to paid from accumulation.

However, I like this second line of interpretation less than the first, because I feel the balance of Marx's text is against thinking of the imperative of accumulation as something relying on technical change.

What I want to emphasize here is that, yet again, Marx has a pretty sound point, which fits very poorly with his theoretical apparatus, since the latter relies crucially on constant returns. In a constant-returns world, a capitalist who is content to just consume his surplus value every year suffers no disadvantage at all, vis-a-vis one who re-invests and accumulates. The latter may, ultimately, enjoy a larger stream of personal income, but that's neither here nor there. Under constant returns, a single large capital enjoys no advantages over a multitude of small capitals.

Chapter 25: The General Law of Capitalist Accumulation

The condition of laborers will just get worse and worse as capital accumulates. When capital accumulates very rapidly, wages might rise above the value of labor power, but if it does, capital accumulation will slow, and wages will come back down. As accumulation proceeds, Marx claims, the ratio of organic capital \(c\) to variable capital \(v\), \(c/v\) or the "organic composition of capital", will tend to grow. (I will avoid the phrase "organic composition of capital".)

I do not follow the argument for this proposition at all. The supposed cause is the "development of the productiveness of labor". By reducing the quantity of labor necessary for producing the goods with which labor-power is reproduced, it lowers the value of labor-power \(v\). But the value of constant capital is that of the socially-necessary labor time for replacing the goods making up constant capital, so increases in the productiveness of labor can also reduce \(c\). There is no argument given, that I can detect, that \(v\) should generally be reduced by more (in proportion) than \(c\) is reduced. The best I can do, attempting to make sense of Marx here, saves his conclusion only for a very special case. Since this is a bit involved, I will push this to an addendum at the end of this chapter, and just say, in conclusion, it doesn't work generally.

This idea, about the rising ratio of constant to variable capital, is linked in Marx's mind to the rising industrial reserve army. He has, I think, two valid points here:

  1. If the productiveness of a certain branch of industry rises, that will ceteris paribus reduce employment in it; those laborers are "made free".
  2. When production expands (at constant productiveness), the laborers have to come from somewhere. The first process throws people into the industrial reserve army, the second process draws them out. (Even if, like a modern economist, one thinks of the labor supply as being a curve, of so many hours offered at so many dollars per hour, there are still adding-up constraints.)

If we add that the productiveness of labor generally rises under modern industry, it still does not follow that the industrial reserve army must grow either absolutely or in proportion to the total population. Still less does it follow that the real income of the working class, measured in the quantity, range, and quality of commodities they can command for their wages, must fall. After all, increasing the productiveness of labor will reduce the exchange value (in labor units) of those commodities!

There are also hints, in this chapter, of an even-more-famous idea, the tendency of the rate of profit to fall, though IIRC that is only fully developed in Volume III. This is related to the point about the supposedly rising ratio \(c/v\). Recall that for Marx the rate of profit is \(s/(c+v)\). If \(v\) and \(s\) stay the same but \(c/v\) rises, then \(c\) must increase and the rate of profit will of course go down. Increasing the productiveness of labor in a given industry will tend to increase \(s\), so in order to make the rate of profit fall, it's especially important that the ratio \(c/v\) increase.

Unfortunately for Marx, as far back as 1961 Okishio Nobuo showed that a technological change which increases profits when introduced by one capitalist is necessarily one which increases the profit rate when generally adopted. (Okishio's paper doesn't seem to be online, but Sam Bowles has a lovely little paper, "prompted by" "his students' critical scepticism", re-proving the theorem in two pages.) This makes it very hard to see how "the general law of capitalist accumulation" could possibly lead to a falling rate of profit, unless capitalists are not generally driven by competition to increase their own profits.

Of course the profit rate could still tend to fall for other reasons, and whether it has fallen or not is a tricky empirical question. The claim, after all, is not about the volume of profits, nor about return on investment. The numerator isn't even, strictly speaking, about "profits" in the ordinary accounting sense, but about surplus value, which gets divided into ordinary profits and many other things (debt servicing, taxes, land rent, embezzlement, protection money to criminals, etc.). Moreover, the denominator is the flow of outlays for the production process (wages \(v\) plus raw materials plus amortization/depreciation of means of production \(c\)), not the stock of capital values (which is the denominator for return on investment). A very rough approximation, based on the usual system of national accounts, would be the share of "net operating surplus" in the national income. At least for the US, this actually did trend downwards from the beginning of the data in 1929 to about 1970, but has been, if anything, rising since about 1990 (source):

Enough, for now, about the rate of profit.

Finally, in this chapter Marx expresses well-deserved contempt towards some economists' views on wages and population, to the effect that rising wages leads to more baby laborers which pushes down wages. To paraphrase him a little, that process would take 15--18 years to take effect, while wage fluctuations are vastly faster.

Addendum: The effect of increasing the productiveness of labor on the ratio of constant to variable capital, a.k.a. the general tendency of the organic composition of capital to do whatever it feels like

Let me recall a result above, from the aside about embodied labor values. When \(\mathbf{v}\) is the input-output matrix of production, and \(\mathbf{v}\) is the vector of labor directly required to produce one unit of each output, the embodied labor values are \[ \mathbf{l} = (\mathbf{I} - \mathbf{a})^{-1} \mathbf{v} \] and the constant capital required per unit of each good is \[ \mathbf{c} = \left( (\mathbf{I} - \mathbf{a})^{-1} - \mathbf{a}\right) \mathbf{v} \equiv \mathbf{\alpha} \mathbf{v} \] introducing the abbreviation \(\mathbf{\alpha}\) for the matrix which sums up direct inputs, plus direct inputs' direct inputs, plus direct inputs' direct inputs' direct inputs, and so on ad infinitum. I need a name for the goods showing up in the \(i^{\mathrm{th}}\) row of \(\mathbf{\alpha}\), so I will call them the "ultimate inputs" used to produce \(i\), or just the "ultimate inputs" for short. It is these ultimate inputs which fix the ratio \(c_i/v_i\): \[ \frac{c_i}{v_i} = \frac{\sum_{j}{\alpha_{ij} v_j}}{v_i} = \alpha_{ii} + \sum_{j\neq i}{\alpha_{ij} \frac{v_j}{v_i}} \]

Now, a technical improvement which increases the productiveness of labor will have to reduce the elements of the vector \(\mathbf{v}\). Suppose for simplicity that it just reduces one element, say that \(v_i\) gets pushed down to \(v^{\prime}_i < v_i\). Further suppose, and this is crucial, that the technical change doesn't otherwise alter the input-output matrix \(\mathbf{a}\), and so \(\mathbf{\alpha}\). Call the new labor requirement \(v^{\prime}_i\). The old value of the ratio \(c_i/v_i\) was, remember, \[ \alpha_{ii} + \sum_{j\neq i}{\alpha_{ij} \frac{v_j}{v_i}} \] while the new one will be \[ \frac{c^{\prime}_i}{v^{\prime}_i} = \alpha_{ii} + \sum_{j\neq i}{\alpha_{ij} \frac{v_j}{v^{\prime}_i}} \] which is necessarily larger than before, since \(v^{\prime}_i < v_i\). So far so good for Marx.

But, and this is crucial, this last paragraph presumes a technical improvement which is only labor-saving, without using more of anything else. If labor becomes productive because it has more or better means at its disposal, those need to show up as larger elements in the input-output matrix \(\mathbf{a}\). So we're really facing a new matrix \(\mathbf{a}^{\prime}\) where \(a^{\prime}_{ij} > a_{ij}\), at least for some \(j\). (Of course some other entries could be smaller, e.g., now-obsolete tools might not be called for.) Thus we'd need to compare the old ratio \[ \alpha_{ii} + \sum_{j \neq i}{\alpha_{ij} \frac{v_j}{v_i}} \] to the new \[ \alpha^{\prime}_{ii} + \sum_{j \neq i}{\alpha^{\prime}_{ij} \frac{v_j}{v^{\prime}_i}} \]

The denominators of the ratios in the sums have shrunk, \(v^{\prime}_i < v_i\), but the new numerators \(\alpha^{\prime}_{ij} v_j\) could be larger or smaller than the old ones, \(\alpha_{ij} v_j\). The over-all effect will depend on whether the ultimate inputs used to produce \(i\) have shifted towards goods which require a lot of direct labor or goods which require little. There is certainly nothing in the logic which says whether the sum of the ratios must have increased.

Labor-saving technical change can thus cause the ratio \(c_i/v_i\) to fall, if it shifts the ultimate inputs towards ones which require little direct labor. Conversely, a labor-raising technical change could increase \(c_i/v_i\), if it shifts the ultimate input mix towards goods with high direct labor requirements, or even just raises \(\alpha_{ii}\) enough.

Finally, I want to consider one additional wrinkle, which is what happens to \(c_i/v_i\) when there is technical progress in another industry, say \(h\). In particular, let's go back to the case where it's only the direct labor requirements that get reduced, so that \(v^{\prime}_h < v_h\). This, as we say, increases the ratio \(c_h/v_h\), but what about industry \(i\)? We go from \[ \alpha_{ii} + \alpha_{ih}\frac{v_h}{v_i} + \sum_{j \neq i, h}{\alpha_{ij}\frac{v_j}{v_i}} \] to \[ \alpha_{ii} + \alpha_{ih}\frac{v^{\prime}_h}{v_i} + \sum_{j \neq i, h}{\alpha_{ij}\frac{v_j}{v_i}} \] which is evidently lower than before, unless \(\alpha_{ih} = 0\). So even a change which drives up \(c/v\) for one industry can lower this ratio for another, downstream industry which uses \(h\) as an ultimate input. If we care about the "organic composition of capital" for the entire economy, which of these two effects would win out would depend on the sizes of the industries affected. Why should raising effect on industry \(h\) should always overwhelm the lowering effect on all industries that ultimate use \(h\) as an input?

To sum up, Marx's idea, to the extent I can make sense of it, only works for the case of a pure improvement of technique, where less labor is needed to get the same output with exactly the same inputs. Even then it only applies to the industry directly affected, with a contrary effect on all downstream industries. Beyond this, the effect of increasing the productiveness of labor on the ratio of constant to variable capital would seem to depend strongly on the details of the technical change.

There could, of course, be some other consideration Marx could advance here, which would say that only certain patterns of technical change are favored under capitalism, and these would imply restrictions on \(\mathbf{\alpha}^{\prime}\) such that the ratio will rise. (Of course he couldn't have put it in just those terms.) But I don't see any such argument in this chapter, or elsewhere.

Part VIII: Primitive Accumulation

Chapter 26: The Secret of Primitive Accumulation

The official theory of "previous accumulation" through which capital arises is that the proto-capitalists are better at saving than other people are. Marx replies that the actual way primitive accumulation happened was simply cheating and theft. Many of Marx's earlier chapters were about how laborers will end up exploited even if the market proceeds exactly as it's supposed to; the following chapters are about how nothing happened the way it was supposed to.

Chapter 27: Expropriation of the Agricultural Population from the Land

Chapter 28: Bloody Legislation against the Expropriated, from the End of the 15th Century. Forcing down of Wages by Acts of Parliament

Chapter 29: Genesis of the Capitalist Farmer

Chapter 30: Reaction of the Agricultural Revolution on Industry. Creation of the Home-Market for Industrial Capital

Starting around 1500 in England, the process of primitive accumulation kicks off. This had two prongs: one was appropriating rights to the land, commons, etc.; the other was actively forcing peasants off the land13. Before, the peasants owned, or at least leased for generations, their means of production, especially the land. Afterwards, their only means of supporting themselves were wage labor, beggary or crime. (Many did not support themselves, and died.) A side-effect of driving families off the land is that they became much less self-sufficient, guaranteeing a market for the output of capitalist manufacturing, even if the latter were no more productive than the old subsistence handicrafts.

Marx takes it for granted that all these processes worked together to reinforce the growth of capitalism. I do not, however, see anything in the text which makes the growth of capitalism a final cause at which these aimed. I also don't see anything which rules that out. A lot of really bad, only-nominally-materialist social and economic theorizing has ever since been resolutely on Team Final Cause, and I wish he'd been clearer about what he meant.

The life of agricultural laborers in 19th century Great Britain and Ireland was nasty, brutish, and short, but over-crowded rather than solitary.

Marx mentions merchant capital in these chapters, and finance capital, but offers no clear account of how either works; there are hints, though just hints, that both are mere trickery and cheating. But transporting goods from one place to another is something that takes labor and creates use values, and so with the modern device of "indexing" goods by location, transport at least could be brought within Marx's framework of production. The value of, e.g., American cotton in Lancashire includes the labor necessary to transport it from New Orleans to Liverpool, plus the value embodied in wear and tear on the ship. Even storage could be put in this framework, indexing goods by time. So Marx should have thought of merchant capital as (at least in part) just a special case of capital employed in production14. Explaining finance capital the same way is more than I feel up to.

Chapter 31: Genesis of the Industrial Capitalist

Lots of rich British families got their money from things like the slave trade and the imperial exploitation of India. Some of them went on to become free-trade liberals.

Chapter 32: Historical Tendency of Capitalist Accumulation

This is the one hopeful chapter in the book. It is also the next to last one, and just three pages. It begins by re-capitulating that capitalist private property begins by expropriating individual private property: the laborers must lose their own means of production before they can become proper proletarians, and capitalists can employ them. But this was historically necessary, if we were ever to get beyond the "mediocrity" of petty production, to develop really powerful and advanced means of production. These are means which can only be used in common, can only be social rather than individual production, and it is one of the ironies of history that they develop under the form of the individual, private property of capitalists. As capitalist accumulation goes on, as technology advances, as the market expands, the working class will become more and more miserable (along with the industrial reserve army), and production will, in its capitalist form, become more and more centralized, while becoming, in reality, more and more socialized.

The upshot is famous, but enough of a rhetorical high point to deserve quotation:

As soon as this process of transformation has sufficiently decomposed the old society from top to bottom, as soon as the labourers are turned into proletarians, their means of labour into capital, as soon as the capitalist mode of production stands on its own feet, then the further socialisation of labour and further transformation of the land and other means of production into socially exploited and, therefore, common means of production, as well as the further expropriation of private proprietors, takes a new form. That which is now to be expropriated is no longer the labourer working for himself, but the capitalist exploiting many labourers. This expropriation is accomplished by the action of the immanent laws of capitalistic production itself, by the centralisation of capital. One capitalist always kills many. Hand in hand with this centralisation, or this expropriation of many capitalists by few, develops, on an ever-extending scale, the cooperative form of the labour process, the conscious technical application of science, the methodical cultivation of the soil, the transformation of the instruments of labour into instruments of labour only usable in common, the economising of all means of production by their use as means of production of combined, socialised labour, the entanglement of all peoples in the net of the world market, and with this, the international character of the capitalistic regime. Along with the constantly diminishing number of the magnates of capital, who usurp and monopolise all advantages of this process of transformation, grows the mass of misery, oppression, slavery, degradation, exploitation; but with this too grows the revolt of the working class, a class always increasing in numbers, and disciplined, united, organised by the very mechanism of the process of capitalist production itself. The monopoly of capital becomes a fetter upon the mode of production, which has sprung up and flourished along with, and under it. Centralisation of the means of production and socialisation of labour at last reach a point where they become incompatible with their capitalist integument. This integument is burst asunder. The knell of capitalist private property sounds. The expropriators are expropriated.

(A later, lesser paragraph re-assures us that overthrowing capitalism will be much less bloody and violent than the original primitive accumulation was.)

This is one of the great apocalyptic visions of modern times; indeed, over the last two centuries, it has rivaled in influence that of St. John the Divine. It is a real pity that it does not follow at all from the preceding analysis. There is nothing in that analysis which indicates that the capitalist mode of production cannot keep reproducing itself forever. Nor is there anything in that analysis which says that if capitalism does end, the replacement will be the socialization of the means of production in the hands of the organized working class.

Chapter 33: The Modern Theory of Colonisation

In one of the great anti-climaxes of modern intellectual history, Marx ends volume I with an extended polemic against a now-obscure political projector. The unfortunate E. G. Wakefield15 wrote a book which was (in Marx's account) all about how hard it was to get proper capitalism going in settler colonies like Australia, because there too many of the settlers owned their own means of production, rather than being property-less laborers, hence such acquisition of productive property ought to be discouraged by the state. The point Marx is trying to drive home is, of course, that capitalist private property has got nothing to do with holding on to the fruits of your own labor. Still, to go from the rhetorical heights of chapter 32 to an extended fisking of a fourth-rate policy entrepreneur (as we'd now call him) shows that Marx, whatever his other virtues as a writer, had little sense of drama.

Some Reflections

On matters of style

On "critique"

The subtitle of capital is A Critique of Political Economy. Evidently, by "critique" Marx didn't meant just subverting the concepts used by political economists; instead he tried to provide a better theory of political economy. I wish more of his successors shared this laudable ambition.

On Marx's modes of expression

I make no apology for having been very free in translating Marx into modern mathematical terms. If Marxism, or even Marxian economics, were a living tradition of scientific inquiry, rather than of scriptural exegesis, it would be continually refining the way it expressed its theory, in particular adopting new mathematical and logical tools, and even building new tools adapted to its own needs. The adherents of the tradition would simplify, formalize, generalize, abstract, apply, reformulate, articulate, refine, and replace. Capital would be as absurd a starting point for modern Marxian political economy as Principia Mathematica Philosophiæ Naturalis is for modern Newtonian mechanics and astronomy. Even if there were no important new discoveries (and who gets everything right the first time?), a progressive tradition would have access to all the ideas and tools of thought developed since Marx's time, and so should be able to improve on him.

It is no coincidence, comrades, that many of those who set on this road end up at not-very-Marxist destinations. (Sam Bowles and Herb Gintis, whose works have been very important for my own intellectual development, might serve as Exhibits A and A'.)

On the voices

Marx's voice --- his very characteristic voice, running here over a range from detached exposition through righteous indignation to acid sarcasm and lacerating contempt --- is the reader's constant companion through the whole book. He quotes, generously, from his predecessors in political economy (bourgeois though they be); he quotes, mockingly, from hapless ideological foils (sometimes other passages from those very same economists); he quotes, approvingly, from factory inspectors, public health workers, parliamentary reports; he quotes the ancients (often in the original Greek). What he hardly ever does is give voice to a laborer, and even when he does, I think it is always mediated through some middle-class social reformer or parliamentary record.

On the labor theory of value

This was not Marx's invention; he inherited it from classical political economy, it having been embraced whole-heartedly by Smith and Ricardo. (The latter was, in many ways, Marx's template.) It has remained a shibboleth of Marxism ever since. I can detect in this book nothing resembling a really solid argument on its behalf. The closest approaches are places where "quantity of socially necessary average homogenized labor" it is just begging to be replaced by "social opportunity cost". Among other advantages, this would let labor (or labor powers) be heterogeneous, like other factors of production.

As an explanation for prices, i.e., exchange values, the theory is a failure for reasons having to do entirely with its logical structure. I think this is important, but I don't know of any way of putting it non-mathematically. I will accordingly confine it to an excursus at the end of this section.

Much of what Marx has to say about technological change, the struggle over the length of the working day, etc., does not in fact depend on the labor theory of value, and can be explained in merely monetary or opportunity-cost terms. (I have tried to indicate this above.)

Marx thought his great advance over his predecessors was to distinguish between the value of labor power, and the value added by the exercise of that power. What remains of this? That there is a cost to creating and maintaining a person's capacity to do work, to producing or re-producing labor power, is obvious. That the opportunity cost of (re-)producing a day's labor power is not the same as the opportunity cost of withdrawing the use of that labor power for a day should also be obvious, at least upon reflection16. That the cost of producing labor power had better be less than the cost of going without it, or the whole society is in trouble, once again should be obvious. That, finally, this leaves a surplus, which has to be divided up somehow, is also true. None of this changes if instead of saying "labor power" we say "human capital"; knock yourself out that way if it makes you feel better.

But equally nothing has to change if in place of labor power / human capital we substitute any other factor of production which is in limited supply and has alternative uses. It could be linen or coats or arable land or potable water or microprocessors or 6mm ball bearings. Even the air, as we know only too well these days, has a limited capacity for dealing with the by-products of our productive processes and our consumption, which goes into the social opportunity cost. Indeed, the analytical Marxist John Roemer once proved that while capitalist profit is impossible without exploiting labor, it is equally impossible without exploiting every other commodity17.

It is still the case that the surplus which results from production must be allocated somehow, and why should it go to the capitalist, or even go through the capitalist? Production is positive sum; that always means there's a struggle over the surplus. This is fundamentally political, it's about the social relations that exist around production and not just technical questions about who does what.

Capitalism pre-supposes a market economy, and production for the market, "commodity production". But it has, classically, three more core features: (1) the means of production are privately owned; (2) workers who do not own the (other) means of production sell their labor power (rent out their human capital) for wages; (3) the owners of the means of production are residual claimants on the surplus product; (4) the owners of the means of production, or their agents, claim the right to direct the work process. (In the corporate form of capitalism, the means of production are owned by corporations, which are the residual claimants; shareholders do not actually own corporations.) All four are logically distinct, and I think could even be dissociated institutionally. There doesn't seem to be anything impossible about a world where the owners of means of production can hire laborers, but are taxed at 100% on any profits over the value of their time put into helping organize production. Capital goods might not fetch very high prices in such a world, and private investment might be very low, but those are different issues.

Excursus: embodied labor content versus exchange values

Recall back to where we worked out how much (abstract, homogeneous) labor is embodied in any given commodity. To make one unit of good \(i\) uses \(v_i\) hours of labor directly, and \(a_{ij}\) units of good \(i\). So the total labor embodied must satisfy \[ \mathbf{l} = \mathbf{v} + \mathbf{a}\mathbf{l} \] or \[ \mathbf{l} = (\mathbf{I} - \mathbf{a})^{-1} \mathbf{v} \] where \(\mathbf{a}\) is a \(k\times k\) matrix, \(\mathbf{v}\) is the vector (\(=k \times 1\) matrix) of direct labor values, and \(\mathbf{l}\) is the vector of total labor values, for all of the \(k\) different commodities in the economy.

For the labor theory of value / "law of value" to hold, we need the vector of exchange values \(\mathbf{p}\) to either equal \(\mathbf{l}\), or at least be proportional to it. But exchange values, too, need to obey an adding-up constraint. Each hour of labor earns a wage, but since hours of abstract homogeneous labor are our units for measuring value, we know that the wage is always \(=1\) numerically. The workers will exchange their wages for a bundle of goods they consume to reproduce their labor-power, say \(b_i\) units of good \(i\) per hour, or in vector form \(\mathbf{b}\). Thus producing one unit of good \(i\) will require \(a_{ij} + v_i b_j\) units of good \(j\), once wages are exchanged for commodities. Collect this into the matrix \(\mathbf{m} = \mathbf{a} + \mathbf{v}\mathbf{b}^T\). Assuming (as Marx emphatically does) a uniform rate of profit \(r\) across industries, can we find a self-consistent vector of exchange values \(\mathbf{p}\)? It would need to satisfy \[ \mathbf{p} = (1+r)(\mathbf{a}\mathbf{p} + \mathbf{v}) \] since, again, labor time is the unit for exchange value. But, precisely because the wage is numerically 1, we know that \(1 = \mathbf{b}^T\mathbf{p}\), so we can write \[ \mathbf{p} = (1+r)(\mathbf{a}\mathbf{p} + \mathbf{v}\mathbf{b}^T\mathbf{p}) \] or \[ \mathbf{p} = (1+r) (\mathbf{a} + \mathbf{v}\mathbf{b}^T)\mathbf{p} = (1+r) \mathbf{m}\mathbf{p} \] Thus the exchange values are the eigenvector of \(\mathbf{m}\) with eigenvalue \(\frac{1}{1+r}\). Since all the entries in \(\mathbf{m}\) are non-negative, the Perron-Frobenius theorem guarantees that it has a (dominant) positive eigenvalue, with all the entries in the corresponding eigenvector being non-negative. (That last is a good sanity-check for exchange values!)

We can say a little bit more if \(\mathbf{m}\) is irreducible, which is to say that tracing back the chain from any one good ultimately requires every other good (i.e., for every \(i, j\), \({\left(\mathbf{m}^{n}\right)}_{ij} > 0\) for some finite \(n\)). Then the dominant eigenvalue \( \frac{1}{1+r} \) is non-degenerate, with a unique, positive eigenvector \(\mathbf{p}\), and there are no other positive eigenvectors. That last uniqueness means that there isn't any other way of assigning exchange-values to goods which satisfies the adding-up constraints imposed by technology (including the reproduction of labor power).

But now we have trouble for the labor theory of value. Remember that the vector of labor contents is \[ (\mathbf{I} - \mathbf{a})^{-1} \mathbf{v} \] while we've just established that the unique vector of prices is the dominant eigenvector of \[ \mathbf{a} + \mathbf{v} \mathbf{b}^T \] These two vectors will not, of course, agree in general. For that to happen, we'd need the vector of labor contents to also be an eigenvector, so it would have to satisfy the equation \[ \left(\mathbf{a} + \mathbf{v} \mathbf{b}^T - \frac{1}{1+r} \mathbf{I}\right) (\mathbf{I} - \mathbf{a})^{-1} \mathbf{v} = 0 ~ (LTV) \]

Between \(\mathbf{a}\), \(\mathbf{v}\) and \(\mathbf{b}\), there are \(k^2+2k\) parameters here (\(r\) doesn't count, because it's fixed by the others). This last matrix equation, (LTV), is equivalent to a system of \(k\) equations those parameters must satisfy in order for the labor theory of value to hold (under all Marx's assumptions). There are more unknowns than equations, so one expects there will be some solutions. Naively, in fact, there'd be a \(k^2 + k\) dimensional space of solutions. (I say "naively", because because the matrix inversion, and the presence of the eigenvalue \(r\), make the equations non-linear in the parameters.) But it's certainly false that (LTV) will hold for all choices of parameters, nor will it hold for most, for any reasonable sense of "most". (Solutions are a measure-0 subset of parameter space, and, if we start from a solution, an arbitrarily small perturbation gives us a non-solution, while the reverse isn't true.) Generically, exchange values just can't be proportional to labor contents.

There are some potentially neat math-y questions here:

  1. Can one give a nice characterization of the part of parameter space where (LTV) holds? What is its dimension? (Someone may have done this.)
  2. Suppose instead of looking for exact solutions to (LTV), we looked for approximate ones, say ones where all exchange values were within \(\pm \epsilon\) of labor contents. Would that occupy much of the parameter space, for not-too-large \(\epsilon\)? If so, one might award the labor theory of value a sort of consolation prize.
  3. Suppose one started with an arbitrary vector of prices \(\mathbf{p}^0\). Exchange at those prices would lead to supra-normal profits in some industries, and losses in others. Under what assumptions about how people update prices will the price vector converge to the unique \(\mathbf{p}\)? This will require somehow modeling the out-of-equilibrium dynamics. My first thought is to just have every capitalist add a fixed mark-up rate to the cost of their inputs, but that will only get us to \(\mathbf{p}\) if the mark-up happens to be \(r\)... (Perhaps something where capitalists learn what mark-up to charge?) If it turned out that the only conditions under which \(\mathbf{p}\) was a stable, rapidly-achieved equilibrium also entailed (LTV), I would be astonished and impressed.

(It will not, I think, do any good to show that \(\mathbf{p}\) is an unstable equilibrium, or one which is reached only glacially slowly, because the whole point of the labor theory of value is that actual prices, which everyone admits fluctuate for all sorts of accidental and un-important reasons, are supposed to track labor contents. That means labor contents are equilibrium prices, and the only equilibrium prices are \(\mathbf{p}\).)

Even I, however, will admit that these are only diversions, not important either practically or even big-picture theoretically. The assumptions of classical political economy, which Marx shares, determine both labor contents and exchange values uniquely, and those are equal only by a great coincidence. If you want to hold on to the labor theory of value, I guess you still could, but you'll need to replace those assumptions with others, and show that the whole thing works. For my part, I don't see why anyone should bother.

— To amplify a point I made above, when solving for labor-contents, the fact that Marx didn't do this pretty simple math is not to his dis-credit. (After all, none of the other classical political economists, e.g., Ricardo, did either.) This problem, like the one about labor contents, is only simple if you have modern linear-algebraic notation (and the concepts that go with it), and Leontief's idea of an input-output matrix from the 1930s. This problem also needs the Perron-Frobenius theorem, from c. 1900. No blame attaches to Marx (let alone his predecessors) for not anticipating all this in the 1850s and 1860s. Later Marx-ists have less excuse.

"The" Cost of Production, or, Again with the Fixed Costs

It is central to Marx's value theory, and so to everything else here, that every commodity has a well-defined cost of production per unit, viz., the amount of (average, homogeneous) labor time socially necessary to replace that unit. There are two great problems here, about fixed costs and about multiple technologies. The first is an (unacknowledged) internal contradiction in Marx's economic thought; the second was simply a blind spot.

Whenever Marx tries to make his value theory rigorous, he presumes, more or less explicitly, that every unit of a commodity produced requires combining the same inputs in the same proportions, with labor time very much included as an input. With this assumption, there is indeed a well-defined value for the total amount of labor required to produce, or replace, a unit of a single commodity, as we've seen. (As we've also seen, this won't generally match exchange value, but that's another story.) But this assumption is empirically vulnerable in two ways.

The first vulnerability is that, even if we grant (as this assumes) that there is only one way of producing any given commodity at any one time, many technologies have fixed costs, say \(c_0\). If, for example, you're going to produce coats in a factory with powered machinery, you need to erect the factory, build the machines, arrange a power-supply, etc., before you can produce even one coat. The marginal cost of one extra coat may be very low --- so much labor \(v\), plus the labor embodied in the raw materials, the fuel, and the marginal wear and tear on the physical plant, totalling \(c_1\) --- and might even be constant as we increase the number of coats produced. (At some point we'd run in to physical limits of the plant and its workers, but let's ignore that for now.) But the total cost of producing \(x\) coats is going to be \(c_0 + x(v+c_1)\), meaning the average cost per coat will be \(v+c_1 + \frac{c_0}{x}\). So what is the cost of production, the quantity of socially-necessary labor time, for a coat? My inclination is that, because Marx is so concerned with the over-all process of production, he ought to go with the average cost, and that if someone could have expressed the point to him, he'd have said that a pro rata share of the fixed costs was included in the constant capital term \(c\). But, if so, his value theory would have to be totally re-done, in a way which somehow incorporates the scale of production. If, on the other hand, he goes with marginal cost, the "socially necessary" part of "socially necessary labor time" is being let go, because those fixed costs aren't just fripperies.

And, in fact, Marx was very aware of the importance of fixed costs, and how they lead to economies of scale. As I've indicated, he has very astute things to say about them, but generally when he's considering economic change rather than value theory, so they're important in the treatment of technological development in Chapter 15, and (I think) in the account of how competition among capitalists forces them to accumulate in Chapter 24. It's just that this recognition contradicts his value theory!

The other empirical vulnerability of Marx's assumptions about production is that he assumes there is one way of producing each commodity at any one time. If we allow that multiple techniques can be economically viable at the same time, then the cost of production of is ill-defined. At the very beginning of the book, Marx gives the example of the introduction of a new power loom thereby reducing the socially-necessary labor time embodied in existing cloth. But, as I said back then, suppose society (currently) has only one such loom, and wants (or needs) more cloth than it can supply. The old looms will, then, still have to be used. But this means that there is no one cost of production for cloth. Again, it was just such a situation, where the necessary output could not all be achieved using the most efficient techniques, that led Kantorovich to linear programming.

Now, Marx might say in response that the firm(s) using the new power loom will earn above-average profits, accumulate capital in the form of more of those looms, and eventually drive the old ones out of business. Even granting this, however, we would only have well-defined costs of production in some sort of long-run equilibrium in between episodes of technological change. The relevance of this to actually existing capitalism is, to say the least, unclear.

But what about power?

A logically distinct point that Marx runs together with extracting surplus value is that the capitalist, in hiring workers, gets to order them around --- that there is domination in the work-place, that bosses boss. This is emphatically true and not well-explained by either classical political economy or the Utopian-competition variety of neo-classical economics. After all, in most market transactions, the buyer doesn't care how the seller gets what they're selling, nor does the buyer claim the right to oversee the production of what's sold. There are resources within contemporary neo-classical economics for explaining the authority of bosses, and they suggest ways in which Marx was right about power, but not for the reasons he thought he was. This is consequential for where we go from here.

Was it worth re-reading?

For me, yes, though my "is this book worth reading?" threshold is notoriously low. I am still happy to acknowledge Marx as one of my intellectual ancestors. (Indeed, dear reader, he was probably one of yours, whether you are aware of it or not.) But in general, though, I'd say no. Even if you really want an understanding of how the economy works which doesn't presuppose the benevolence of capitalism, why go here? Too much of the book is bound up in its least defensible parts, and too much of it is a brilliant mind wrestling with basically mathematical problems without using enough math. (In some cases because the math didn't exist yet, but still.) This has become a book of great historical importance, but of merely historical importance.


  1. Specifically, the Modern Library reprint (n.d.) of the 1906 American edition prepared by Ernest Untermann. When I quote long passages, though, I have sometimes merely taken them from the online edition at marxists.org, which is evidently a slightly different translation.^

  2. Commentary on the role of this axiom of choice in a truly liberatory mathematics is referred to the appropriate literature.^

  3. "Skilled labour counts only as simple labour intensified, or rather, as multiplied simple labour, a given quantity of skilled labour being considered equal to a greater quantity of simple labour. A commodity may be the product of the most skilled labour, but its value, by equating it to the product of simple unskilled labour, represents a definite quantity of the latter labour alone. The different proportions in which different sorts of labour are reduced to unskilled labour as their standard, are established by a social process that goes on behind the backs of the producers, and, consequently, appears to be fixed by custom."^

  4. Marx means "fetishism" in the 19th century anthropological sense, not in that of a paraphilia, though of course generations of scholars since Freud have had fun with the obvious puns.^

  5. Cf. Russell's Proposed Roads to Freedom (ch. VIII, p. 129): "There would still have to be money, or something analogous to money, in any community such as we are imagining. The Anarchist plan of a free distribution of the total produce of work in equal shares does not get rid of the need for some standard of exchange value, since one man will choose to take his share in one form and another in another. When the day comes for distributing luxuries, old ladies will not want their quota of cigars, nor young men their just proportion of lap-dogs; this will make it necessary to know how many cigars are the equivalent of one lap-dog. Much the simplest way is to pay an income, as at present, and allow relative values to be adjusted according to demand. But if actual coin were paid, a man might hoard it and in time become a capitalist. To prevent this, it would be best to pay notes available only during a certain period, say one year from the date of issue. This would enable a man to save up for his annual holiday, but not to save indefinitely." --- I would add that the advantages of giving people money to spend as they wish are only enhanced if some young men would like lap-dogs and some old ladies would prefer cigars. But here I come to close to all-too-familiar ground.^

  6. If \(s/v = r\) is given (Marx doesn't give it a letter), then clearly \(s/(v+s) = 1/(1+v/s) = 1/(1+1/r)\) is fixed. But if we know \(s/(v+s) = R\), then \(r = 1/(1/R - 1)\).^

  7. This makes me even more unhappy about how Marx, in his chapters on value, assumes constant returns to scale and ignores the problem of fixed costs. To the extent there are costs in setting up a productive enterprise at all, regardless of how much is produced, e.g., erecting a factory building or installing an assembly line, they ought to show up in \(c\), since those acts of labor were socially necessary to production. But Marx turns this all into wear-and-tear, or amortization and depreciation, which is hardly adequate. (Never mind how we allocate the fixed cost for a factory that produces multiple commodities.) I am almost tempted, rather perversely I admit, to try to re-interpret everything as referring to marginal values.^

  8. What follows looks trivial, because, with modern mathematical and conceptual tools, i.e., linear-algebra notation and Leontief's idea of an input-output matrix, it is trivial. I don't know enough about the history of linear algebra to say whether the notation and mathematical concepts (or some equivalent precursors) were available in Marx's day, though I suspect they were (and, if so, would almost certainly have been expressed in German!). One can't, of course, blame Marx for not anticipate Leontief's work in the 1920s and 1930s. But an interesting question, which I am not at all competent to resolve, is whether the tables of the 18th century Physiocrats, with whom Marx was familiar, were close enough to Leontief's input-output matrices that he could have at least posed the question in its linear-algebraic form. Since I am writing a commentary on Capital and not writing alternate-history story where Marx spends decades in the British Museum compiling The Structure of the British Economy in 1849, followed by a further decade of inverting the I-O matrix by hand, I shall bring this aside to a close.^

  9. I am going to deliberately use Marx's word "productiveness", rather than the modern "productivity", because the latter refers to monetary output per unit of labor time, while Marx means the output of use-values, and I think that distinction is important here.^

  10. A paragraph is, I think, worth quoting in full (omitting Marx's footnotes). "Modern industry never looks upon and treats the existing form of a process as final. The technical basis of that industry is therefore revolutionary, while all earlier modes of production were essentially conservative. By means of machinery, chemical processes and other methods, it is continually causing changes not only in the technical basis of production, but also in the functions of the labourer, and in the social combinations of the labour-process. At the same time, it thereby also revolutionises the division of labour within the society, and incessantly launches masses of capital and of workpeople from one branch of production to another. But if modern industry, by its very nature, therefore necessitates variation of labour, fluency of function, universal mobility of the labourer, on the other hand, in its capitalistic form, it reproduces the old division of labour with its ossified particularisations. We have seen how this absolute contradiction between the technical necessities of modern industry, and the social character inherent in its capitalistic form, dispels all fixity and security in the situation of the labourer; how it constantly threatens, by taking away the instruments of labour, to snatch from his hands his means of subsistence, and, by suppressing his detail-function, to make him superfluous, we have seen, too, how this antagonism vents its rage in the creation of that monstrosity, an industrial reserve army, kept in misery in order to be always at the disposal of capital; in the incessant human sacrifices from among the working-class, in the most reckless squandering of labour-power and in the devastation caused by a social anarchy which turns every economic progress into a social calamity. This is the negative side. But if, on the one hand, variation of work at present imposes itself after the manner of an overpowering natural law, and with the blindly destructive action of a natural law that meets with resistance at all points, modern industry, on the other hand, through its catastrophes imposes the necessity of recognising, as a fundamental law of production, variation of work, consequently fitness of the labourer for varied work, consequently the greatest possible development of his varied aptitudes. It becomes a question of life and death for society to adapt the mode of production to the normal functioning of this law. Modern Industry, indeed, compels society, under penalty of death, to replace the detail-worker of to-day, grappled by life-long repetition of one and the same trivial operation, and thus reduced to the mere fragment of a man, by the fully developed individual, fit for a variety of labours, ready to face any change of production, and to whom the different social functions he performs, are but so many modes of giving free scope to his own natural and acquired powers."^

  11. Marx never, that I can tell, squares these (entirely correct) observations about fixed costs, which imply (initially) increasing returns to scale, with his reliance on constant returns to scale in his more theoretical chapters.^

  12. "In the sphere of agriculture, modern industry has a more revolutionary effect than elsewhere, for this reason, that it annihilates the peasant, that bulwark of the old society, and replaces him by the wage-labourer. Thus the desire for social changes, and the class antagonisms are brought to the same level in the country as in the towns. The irrational, old-fashioned methods of agriculture are replaced by scientific ones. Capitalist production completely tears asunder the old bond of union which held together agriculture and manufacture in their infancy. But at the same time it creates the material conditions for a higher synthesis in the future, viz., the union of agriculture and industry on the basis of the more perfected forms they have each acquired during their temporary separation. Capitalist production, by collecting the population in great centres, and causing an ever-increasing preponderance of town population, on the one hand concentrates the historical motive power of society; on the other hand, it disturbs the circulation of matter between man and the soil, i.e., prevents the return to the soil of its elements consumed by man in the form of food and clothing; it therefore violates the conditions necessary to lasting fertility of the soil. By this action it destroys at the same time the health of the town labourer and the intellectual life of the rural labourer. But while upsetting the naturally grown conditions for the maintenance of that circulation of matter, it imperiously calls for its restoration as a system, as a regulating law of social production, and under a form appropriate to the full development of the human race. In agriculture as in manufacture, the transformation of production under the sway of capital, means, at the same time, the martyrdom of the producer; the instrument of labour becomes the means of enslaving, exploiting, and impoverishing the labourer; the social combination and organisation of labour-processes is turned into an organised mode of crushing out the workman's individual vitality, freedom, and independence. The dispersion of the rural labourers over larger areas breaks their power of resistance while concentration increases that of the town operatives. In modern agriculture, as in the urban industries, the increased productiveness and quantity of the labour set in motion are bought at the cost of laying waste and consuming by disease labour-power itself. Moreover, all progress in capitalistic agriculture is a progress in the art, not only of robbing the labourer, but of robbing the soil; all progress in increasing the fertility of the soil for a given time, is a progress towards ruining the lasting sources of that fertility. The more a country starts its development on the foundation of modern industry, like the United States, for example, the more rapid is this process of destruction. Capitalist production, therefore, develops technology, and the combining together of various processes into a social whole, only by sapping the original sources of all wealth --- the soil and the labourer."^

  13. Note 12 in chapter 27 mocks William of Orange for awarding lands to a lady whose "endearing offices are supposed to have been --- foeda labiorum ministeria". Since royal land grants have usually recognized success in wholesale slaughter, using them to reward excellence in oral sex seems like a real advance for civilization and decency.^

  14. And, indeed, it's been pointed out to me that he basically does this, in Volume II (ch. 6, sec. 3).^

  15. Wakefield turns out to have been a remarkably colorful character, but that doesn't improve the quality of his ideas.^

  16. That the opportunity cost of one less day's labor power is also the worth of expanding the available labor power by one day is less obvious. However, it drops out of basic optimization. Lagrange multipliers are shadow prices, and give the value, here in terms of opportunity costs, of an incremental loosening of the constraint. When the constraint is that there is only so much labor power available, the claim follows.^

  17. You can find a very simple proof of the same proposition, along with some other interesting reflections on Marx's labor theory of value, in R. P. Wolff, "A Critique and Reinterpretation of Marx's Labor Theory of Value", Philosophy and Public Affairs 10 (1981): 89--120 [JSTOR].^

The Dismal Science; The Progressive Forces

Posted at September 08, 2018 22:10 | permanent link

September 07, 2018

Of Microfoundations (In Memoriam Gary Becker)

Attention conservation notice: 1200+ words about a nearly sixty-year-old paper on the foundations of microeconomics, maliciously re-interpreting it in ways its author explicitly disclaimed, by someone with no qualifications in economics. I wrote this in 2012 or 2013, re-titled in 2014, and never got around to properly finishing it. I dust it off now because I'm in the middle of reading Jason Smith's A Random Physicist Takes on Economics, and see that he had basically the same idea, but said it better and in public. This might, therefore, serve to amplify his message.

I recently re-read Gary Becker's "Irrational Behavior in Economic Theory" paper (Journal of Political Economy 70 (1962): 1--13), and want to talk about it. (Like most of what I know about economics, I was introduced to this paper by my father, many years ago now.) Becker was, to my mind, the embodiment of much of what is wrong with the discipline and practice of economics (see, e.g., John Emerson on Becker's work on families [1, 2]), but this is a genuinely brilliant paper. In many ways, the paper was wiser than its author.

Let me start with the basic observation of the paper. Suppose you have a monthly grocery budget. (Or book budget, or drug budget, or...) This budget, together with the prices of grocery items, dictates what you could buy in the way of groceries. Suppose that the price of onions goes up. Then, just as a matter of arithmetic, the maximum number of onions that you can buy goes down, and for any given level of consuming other goods, you can buy fewer onions. Even if you pick a totally random collection of groceries each week, having a fixed amount of money to spend will ensure that when the price of something go up, you will (on average) buy less of it.

If all the consumers in a market also have budget constraints, then the same reasoning applies to them, too, and so the total quantity of onions bought will tend to fall as their price rises. This is, of course, a basic idea of standard economics, but what Becker shows is that this has nothing to do with satisfying preferences or maximizing utility, or any other assumption about how individuals make decisions. Consumers' could make decisions by lots of different mechanisms --- indeed different consumers could be driven by different mechanisms --- and it wouldn't matter. What does matter is that they all have budget constraints.

Becker's treatment of the producer side of the market is subtly different, and doesn't rely on budget constraints. (Becker indeed insists producers aren't subject to budget constraints, which not only would be news to many small business owners I know, but would also mess up applying his reasoning about consumption to markets for capital goods.) Rather, the the insight is that a wider range of productive techniques, and of scales of production, become profitable at higher prices. This matters, says Becker, because producers cannot keep running losses forever. If they're not running at a loss, though, they can stay in business. So, again without any story about preferences or maximization, as prices rise more firms could produce for the market and stay in it, and as prices fall more firms will be driven out, reducing supply. Again, nothing about individual preferences enters into the argument. Production processes which are physically perfectly feasible but un-profitable get suppressed, because capitalism has institutions to make them go way. If you think this account of producers and supply is all less tidy and satisfactory than the treatment of consumers and demand, I agree with you, but it is, after all, far better than the usual microeconomic account of these matters.

Becker doesn't go in this direction, but of course this completely undermines welfare economics. The whole grasp of that sub-field on reality comes from assuming that individuals' choices reflect their judgments of utility. If that's true, we could read off a lot about people's (subjective) welfare from aggregated market prices and quantities. By breaking the link between microeconomic relations for such aggregated variables and individual utility, Becker breaks the link between observations and welfare economics.

Going further, Becker also removes the link between positive microeconomics, at the level of the market, and individual behavior. Though Becker doesn't put it this way, and would have been horrified by the thought, his argument is it really doesn't matter how individuals make decisions, or even whether economic behavior is usefully thought of as "making decisions" at all. Rather, everything depends on the constraints put on behavior by institutions. That consumers have budgets and can't spend more than that, and that producers can't keep running at a loss, these are both institutional facts about the social organization of capitalism, not facts about how individuals act. (Of course, institutions have to be implemented through individual actions, institutions aren't some sort of cloud hovering over society telling people what to do, but that's another story.) This implies that there are all sorts of different possible micro-foundations for our positive economics, since anything that preserved the institutions would serve. It also suggests that a different set of institutions would lead to a different positive economics. Since institutions change over space and time, then, positive economics would have to be historically relative.

In short, I am inclined to say that Becker fully deserves his Nobel Prize on the strength of this paper alone. The citation, though, should have read "For exploding the foundations of welfare economics, and re-constituting positive microeconomics on an institutionalist basis".

Now, this is not what Becker thought he was doing. From the opening and closing of the paper, it's clear that what he thought he was defending business-as-usual for the Chicago School. Nowadays, we can say that the assumptions of that school about individual behavior are experimentally refuted; science knows that's no more how humans behave than phlogiston is how stuff burns. Even back then, though, there was a lot of criticism for the lack of realism of these assumptions. For instance, Herbert Simon (pbuh) kept pointing out that human rationality is computationally bounded...

Chicago Schoolers had a range of response to such criticism. One was to ignore it and go ahead with the work. (This is part of why I am leery of "shut up and calculate" responses to conceptual criticisms; they're only as good as the basis for the calculations.) The prize for chutzpah went to Milton Friedman, here as elsewhere a profoundly malign influence, who claimed that this lack of realism was in fact a virtue, and actually what made economics extra scientific. (I am not exaggerating, and outsource further commentary to Simon, 1963.)

Compared to that, Becker must have thought he was holding out an olive branch. Look, he (in effect) said, I'll grant for the sake of argument that we're totally wrong about how people make decisions. At the market level, it wouldn't matter, things would still come out the way we say they should. So why not let us hold to those assumptions, just to have something definite to work with?

But of course this won't do. Whatever you feel about Occam's Razor, "This assumption is superfluous, therefore it should be retained" is a very strange rule of method. (Why not assume that people maximize utility, and that utility is purple?) A more sensible reaction to Becker's own argument would surely be to avoid relying on those assumptions for anything, at least until he could provide some independent support for them.

On a more constructive note, I suspect there is a connection to be made here with John Sutton's "bounds approach" to studying industrial organization, where the aim isn't to solve for the equilibrium of the one true model, but instead to find relationships, generally inequalities, which hold across a broad range of models. There might even be some sort of analogy to thermodynamics, which is what you get for nearly-any system built out of molecules when its parts can only interact through a restricted set of collective degrees of freedom (viz., exchange of heat, pressures, a few chemical species, etc.). But both of those would call for real thought and original work on my part, rather than mildly-malicious exposition.

The Dismal Science

Posted at September 07, 2018 17:55 | permanent link

September 06, 2018

Data over Space and Time, Lecture 4: Principal Components Analysis I

(R Markdown source file)

Corrupting the Young; Enigmas of Chance

Posted at September 06, 2018 20:40 | permanent link

September 04, 2018

Data over Space and Time, Lectures 2 and 3: Smoothing, Trends, Detrending

Lecture 2 was a chalk-talk, which proved not to work very well in the room. (I never thought I would miss Wean Hall, but there you are.) Lecture 3 went back to slides, which will probably have to be the way going forward.

Corrupting the Young; Enigmas of Chance

Posted at September 04, 2018 16:10 | permanent link

September 02, 2018

Some Blogospheric Navel-Gazing, or, Strange Memories of the Recent Past

Attention conservation notice: 2600 words about blogging for young scientists, last touched in 2013. Found five years later while looking for something else, they are too dated to be good career advice, too earnest and academic to be ironically amusing in retrospect, and too recent to inspire nostalgia. Also, most of the links have probably broken, I haven't had the heart to check.
Backstory: I was asked in 2011 to talk to the post-docs and students at SFI about the pros and cons of blogging for scientists. I didn't prepare anything, but what follows more or less reconstructs and tidies up what I said, with the benefit of feedback from Nathan "Explains Science" Collins.
As career advice now (2018), of course, this is laughable: the blogosphere limps along (every Internet medium still limps along...), but it's been largely displaced as a focus of attention by Twitter and other company-owned social-networking sites, which are (deliberately?) pessimized for articulating ideas and cumulative conversation.

I have been online for a long time, comparatively; I remember when Usenet was good for something, and we sent packets to each other by tying them to the backs of gophers. I have been writing something which could, with a little license, be called a proto-weblog since 1994, and a proper blog since 2003. Since I am, of course, aware of all Internet traditions, the blog has a silly sobriquet and Friday cat blogging (lapsed for a while), and ill-tempered political ranting, but mostly both of those sites are about the science I find interesting, including my own. I owe a large part of whatever professional reputation I have to blogging, so when Barbara asked me to talk about how you might use blogging yourselves as scientists, I couldn't think of a decent way to back out of it.

Good Reasons Not to Blog

The first thing to say is that blogging is not a good idea for many and perhaps most scientists, for several reasons, which can be summed up as time, content, disappointment and dissociation.

The binding constraint for most scientists is not having enough time; if you don't feel that way yet, you will soon enough. Every moment you spend writing a blog post is a moment you are not in the lab or at the blackboard, not writing a paper, not reading papers, not writing grants, not advising students, not having a normal and satisfying primate social life, and not sleeping. Even if you want to turn yourself into a "machine for converting amphetamines into proofs", you have only so many minutes to your life. Will you, in ten years, look back and say "You know, I wish I had spent more time writing for strangers on the Internet, and less time doing research or being with my friends and family"? It seems doubtful.

If you do decide to take the time, you need to have not just something to say, but something to say in public. Your public may be small and obscure, you may have to help create your public, but there does have to be a public, and your writing has to be accessible to any member of it. If not, then you are just sharing with your family and friends, and for that we have e-mail and social networking sites. (In fact a public, defined by shared concern with some external value or subject, is almost the polar opposite of a social network, defined by concrete ties of social interaction.) Scientific papers are a very pure form of public writing in this sense, but you are already going to be writing papers. To blog, you need to have something to say, publicly, which goes beyond your papers.

If you have something to say and take the time to say it, there is still no way to force anyone to pay attention, and most attempts at blogging do not succeed in attracting much attention. The distribution of readership is not a power law, but it is strongly heavy tailed. Worse, from the point of view of someone trying to start a blog and attract attention, lots of mechanisms reinforce the position of already-prominent blogs: search-engines, for instance, or the effects of attention from other websites. My impression is that it is now much harder to achieve a given degree of readership for a new blog than it was in say 2003, simply because lots of the niches have already been filled, and it is not easy to displace the encumbents. So even if you do everything well, there is a good chance nobody will notice, or very few.

If everything goes well, there are of course several down-sides or costs to running a blog, over and above the time. One of them which is somewhat subtle is that a successful blog tends to develop an authorial voice, or perhaps better yet a persona. This is a natural part of all forms of social interaction (go read Erving Goffman's The Presentation of Self in Everyday Life) but that persona is a literary creation, a sustained act of rhetorical self-fashioning, and not your total personality. Readers, however, are very apt to mistake an authorial persona for the personality of the author; they use your words to paint a picture of someone in their minds, and then they think they know you. (Novelists face similar issues.) This is something that can be quite weird and disturbing to experience, or figure out how to deal with. It is also possible to mistake your own narrative persona for your real personality, but is I think less common.

What Is It Good For?

Let us suppose that you find the time, that you find something to say, that you find readers, and that you make your peace with their confusing you with a fictional character. What then? What is all this good for?

There are of course a huge range of things people have found to do with blogs, but I will just discuss five which are especially useful for scientists: writing practice; the discussion of ideas; self-promotion; public communication; and teaching.

Writing practice Scientists live in a reputation economy. The reputation we want to have, with other scientists, is as original and reliable inquirers into interesting topics. We achieve this reputation by persuading them of our findings and their importance in writing. (We also do a little persuasion orally, in talks.) Findings which are not communicated, or communicated badly, are simply not part of the development of science, no matter how profound they might be. Scientists must therefore must write, and ought to write well.

Perversely, however, scientists are generally not good writers. The key reason is I think a lack of practice, especially a lack of practice with feedback. Like anything else, the only way to develop the skills of writing is to practice, see what worked and what failed, and try to do more of the former and less of the latter. Blogging can be an extremely good way to get this practice with feedback, if you can make yourself accept the negative, do-less-of-this signals. (Think of it as training for learning from negative referee reports.)

Discussing ideas Every scientific literature is, and has always been, surrounded by an area of discussion about those ideas, their implications and their possible developments. Contributions to this conversation do not need to be as sophisticated, novel, or rigorous as contributions to the literature, but they are vital to the way epistemic communities advance knowledge, and to the way people come to join such communities. These conversations have always gone on, as arguments at tea-time, or during breaks at conferences, etc., etc., but moving them into the form of blogging changes their character, because they become public and permanent. Publicity means that those who lack the social resources to be physically present and accepted at departmental tea-times can learn from the conversation, and potentially contribute to it. (This overlaps with the pedagogical function of blogging, particularly in the hands of people like Terry Tao, as I've talked about elsewhere.) Permanence and publicity makes it easier to hold people accountable for what they say, and in particular to force them to justify it by generally acceptable intellectual criteria.

Said slightly differently, moving this sort of conversation to blogs makes what sociologists call "invisible colleges" into visible publics. This has the potential to widen the circle of participation and to raise the level of discussion and thought. Some people find this uncomfortable; some economists, for instance, worry that "blogs are ruining economic debate". (In my own supremely arrogant opinion, this view owes much to the very strong tendency among economists to tell lies about over-simplify economics to lay-people and undergraduates "for their own good".) For my part I find this a Good Thing, for reasons expressed well by my friend and collaborator Henry Farrell, in a 2005 essay on "The Blogosphere as a Carnival of Ideas":

Academic blogs offer the kind of intellectual excitement and engagement that attracted many scholars to the academic life in the first place, but which often get lost in the hustle to secure positions, grants, and disciplinary recognition. Properly considered, the blogosphere represents the closest equivalent to the Republic of Letters that we have today. Academic blogs, like their 18th-century equivalent, are rife with feuds, displays of spleen, crotchets, fads, and nonsenses. As in the blogosphere more generally, there is a lot of dross. However, academic blogs also provide a carnival of ideas, a lively and exciting interchange of argument and debate that makes many scholarly conversations seem drab and desiccated in comparison. Over the next 10 years, blogs and bloglike forms of exchange are likely to transform how we think of ourselves as scholars. While blogging won't replace academic publishing, it builds a space for serious conversation around and between the more considered articles and monographs that we write.

Self-Promotion Of course, the ideas you discuss can be your own. You presumably think your finished papers, at least, are important and good and at least as interesting as any other papers, so why not talk about them? Yes, others in the field might see them in the journals or conferences, but they might not, or might not see why they should read them, so tell them why. You may target this at the general public, in which case you are acting as your own science journalist (an endeavor of which more below), or more or less narrowly at your professional peers. The latter calls for a little comment.

It is hard to over-emphasize the fact that science is a social process. (Discoveries which stay in your desk and are never communicated do not, properly speaking, form part of the body of scientific knowledge.) Making a career in science is building a reputation with your peers as a reliable, original and interesting investigator. In any half-way healthy field, the primary thing on which this depends is the quality of your research, but there is nothing wrong with thinking about other ways of developing your professional reputation, i.e., bringing your work and your insights to the attention of your peers. Blogging about your research, and that of other scientists which you find interesting, can be an excellent way of cultivating that reputation.

In this connection, let me recommend a resource which I found profoundly helpful when I was a graduate student, Phil Agre's "Networking on the Network". Ostensibly, Agre's article is about using electronic communications, circa 1996, for professional networking, but really it's about how to build a professional persona without being a manipulative creep. I will not repeat everything he says, but simply urge you to go read it, and the companion pieces "Find Your Voice" and "How to be a Leader in Your Field".

Public Communication I have been talking so far about participating in the conversation within your field or discipline. At the extreme, this shades off into blogging your research as you do it, or even having the research happen across multiple blogs building on each other. (The Polymath Project is perhaps the purest realization of this idea so far.) At the other end of the spectrum, as you make your writing more and more generally accessible, you approach science popularization, or acting as an amateur science journalist. Popularization and science journalism are tricky undertakings, and there is a reason there are training programs for them. Perhaps the hardest part of the endeavor is to find ways of explaining the science which are interesting and comprehensible to those who have not been immersed in the field, who (unlike students) do not have to pay attention, and who quite properly resent being patronized. Despite its difficulty, I think it is important for members of the scientific community to do this, to explain their science to the general public, both because science is important and worth knowing about, and because its pursuit on anything like its current scale — as anything more than a hobby for the eccentric rich — depends on public support. There are of course many excellent science journalists, some of whom are also good bloggers, such as Carl "The Loom" Zimmer and Tom "Inverse Square" Levenson, but it is I think important that scientists do it themselves, if only because even the most able journalist does not have the time to achieve the level of expertise that scientists possess.

Teaching Last among the uses of blogging for scientists that I want to mention is teaching. On the one hand, blogs can be a very good way of working through the material for classes --- what do you want to teach? how do you want to teach it? what do you actually think about the topics? On the other, they are a perfectly fine way of disseminating educational materials. Personally, I have gotten a lot of mileage out of using my blog to spread course notes on stochastic processes, data mining and data analysis. At a more exalted intellectual level there are things like Scott Aaronson's Quantum Computing Since Democritus, and Terry Tao's continuing series of lectures on pure math.

Mechancics

Actually Writing the Blessed Thing I have said nothing about the actual mechanics of writing and posting. This is the sort of thing where there are "nine-and-sixty ways, every single one of which is right" — for someone. I write everything as plain HTML files in Emacs, in a single flat directory, and publish with Blosxom, and write equations with MathJax. But my blog is boring to look at, and I wouldn't advocate my tool-set for anyone else.

I would suggest keeping a separate directory of drafts, well apart in your file system from posts which are ready to go, and a file listing ideas or notions that might go into the blog someday. As you go on, you'll find it easier to see the angles to things that could make them into posts; the list lets you capture those angles. When you start working on something, you move it from the list to its own draft file; when a draft is ready, you move it to accompany the finished posts.

Moving things from ideas to drafts to posts will require time spent on writing. If you are logorrhetic, like me, that might not be an issue, but some people find it helpful to budget time for this. The rule then is that the time must be spent either adding words or revising old ones, not fiddling with anything else, like software or social media. (This also helps for writing dissertations and papers.)

Comments Most blogs have comment sections. This can help for getting immediate feedback on your writing, and make for a much better experience for your readers, but there are very real downsides. The famous failure mode of unattended comments sections is to become vile. Avoiding this requires a lot of attention, either on your part or that of your readers, with ultimate responsibility resting with you — in the immortal words of Anil Dash, "if your website's full of assholes, it's your fault". The less heralded but perhaps even more common failure-mode is for the comments to be inane. There are, in my humble opinion, vanishingly few (though not quite zero) blogs whose comments are worth reading or participating in. I will not name them here, because I don't want them spoiled.

Manual trackback: Andrew Goldstone

Linkage; Learned Folly

Posted at September 02, 2018 16:40 | permanent link

August 31, 2018

Books to Read While the Algae Grow in Your Fur, August 2018

Attention conservation notice: I have no taste. I also have no qualifications to discuss folklore, structuralism, optics and painting in the early modern Netherlands, Aztec culture, or Cold War espionage.

Vladimir I. Propp, Morphology of the Folktale [as Morfologija skazki, Leningrad, 1928; translated by Svatava Pirkova-Jakobson, Indiana University Press, 1958; second edition, revised by Louis A. Wagner and with an introduction by Alan Dundes, Austin: University of Texas Press, 1968]
I'd known about this book for quite some time, and browsed in it long ago, but never actually read it until this year. It's a really incredible piece of work.
Propp set out to identify the basic elements of the plots of Russian fairy tales, working at a level of abstraction where "it does not matter whether a dragon kidnaps a princess or whether a devil makes off with either a priest's or a peasant's daughter". He came up with 31 such "functions". Just listing them (chapter 3) has a certain folkloric quality:
  1. One of the members of a family absents himself from home ($\beta$)
  2. An interdiction is addressed to the hero ($\gamma$)
  3. The interdiction is violated ($\delta$)
  4. The villain makes an attempt at reconnaissance ($\epsilon$)
  5. The villain receives information about his victim ($\zeta$)
  6. The villain attempts to deceive his victim in order to take possession of him or of his belongings ($\eta$)
  7. The victim submits to deception and thereby unwittingly helps his enemy ($\theta$)
  8. The villain causes harm or injury to a member of a family ($A$) or One member of a family either lacks something or desires to have something ($a$)
  9. Misfortune or lack is made known; the hero is approached with a request or command; he is allowed to go or he is dispatched ($B$)
  10. The seeker agrees to or decides upon counteraction ($C$)
  11. The hero leaves home ($\uparrow$)
  12. The hero is tested, interrogated, attacked, etc., which prepares the way for receiving either a magical agent or helper ($D$)
  13. The hero reacts to the actions of the future donor ($E$)
  14. The hero acquires the use of a magical agent ($F$)
  15. The hero is transferred, delivered, or led to the whereabouts of an object of search ($G$)
  16. The hero and the villain join in direct combat ($H$)
  17. The hero is branded or marked ($J$)
  18. The villain is defeated ($I$)
  19. The initial misfortune or lack is liquidated ($K$)
  20. The hero returns ($\downarrow$)
  21. The hero is pursued ($Pr$)
  22. Rescue of the hero from pursuit ($Rs$)

At this point, Propp observes, the tale can more or less begin over again, with the transition from the first "move" to the second being initiated by a new act of villainy, typically "Ivan's brothers steal his prize, and throw him into a chasm" ($\* A$). This leads to $C--G$ again.

  1. The hero, unrecognized, arrives home or in another country ($o$)
  2. A false hero presents unfounded claims ($L$)
  3. A difficult task is proposed to the hero ($M$)
  4. The task is resolved ($N$)
  5. The hero is recognized ($Q$)
  6. The false hero or villain is exposed ($Ex$)
  7. The hero is given a new appearance ($T$)
  8. The villain is punished ($U$)
  9. The hero is married and ascends the throne ($W$)

Each abstract function has, naturally, a great many more concrete sub-types (e.g., seven distinct variants of pursuit, ranging from $Pr^1$, "the pursuer flies after the hero", to $Pr^7$, "He tries to gnaw through the tree in which the hero is taking refuge".

Based on extensive study of the corpus of Russian fairytales, Propp claims that the initial functions, designated by Greek letters, are less essential than the ones designated by Roman letters. In fact, in what I take to be the central finding of the book (ch. IX, sec. D, pp. 104--105), he claims that all the tales in the corpus belong to four, and only four, categories:

  1. Tales with a struggle and victory, but no difficult tasks, following the scheme $ABC\uparrow DEFGHJIK\downarrow Pr Rs o LQ Ex TUW$.
  2. Tales with difficult tasks, but no combat, following the scheme $ABC\uparrow DEFG o LMJNK\downarrow Pr Rs Q Ex TUW$.
  3. Tales which include struggle-victory and difficult tasks, under the scheme $ABC\uparrow FH-IK\downarrow LM-NQ Ex U W$. (That is, the struggle always comes before the difficult tasks.)
  4. Tales which include neither struggle nor difficult tasks, under the scheme $ABC\uparrow DEFGK\downarrow Pr Rs Q Ex T U W$.
As he remarks after making these claims,
To the variable scheme \[ ABC\uparrow DEFG \frac{HJIK\downarrow Pr-Rs o L}{LMJNK\downarrow Pr-Rs} Q Ex TUW \] are subject all the tales of our material: moves with $H-I$ develop according to the upper branch; moves with $M-N$ develop according to the lower branch; moves with both pairs first follow the upper part and then, without coming to an end, develop following the lower offshoot; moves without either $H-I$ or $M-N$ develop by bypassing the distinctive elements of each. [p. 105]

What I find so astonishing here is that this is a formal grammar, though propounded many years before that notion emerged in linguistics, logic and computer science. Specifically, it is a formal grammar which generates fairytale plots. Propp realized this, and used the schema to create new fairy tales [1] (unfortunately, not recorded). A basic principle of formal language theory is that a schema which generates all and only the valid strings of a language can also be used to recognize whether a string belongs to that language; Propp implicitly grasped this, and argued on this basis that some non-fairy-tales in his corpus were more properly classed with the fairy tales.

It's especially noteworthy to me that Propp's schema is a regular grammar, i.e., at the lowest level of the Chomsky hierarchy. These correspond to the regular expressions familiar to programmers, to finite-state machines, and to (functions of) Markov chains. The production rules would be something like \[ \begin{eqnarray*} Story & \rightarrow & ActI Act2 Act3\\ Act1 & \rightarrow & ABC\uparrow DEFG\\ Act2 &\rightarrow & (Struggle | 0) (Task | 0)\\ Act3 & \rightarrow & Q Ex TUW\\ Struggle & \rightarrow & HJIK\downarrow Pr-Rs o L\\ Task & \rightarrow & LMJNK\downarrow Pr-Rs\\ \end{eqnarray*} \] using $|$ as usual to represent alternatives, and $0$ to represent a null story element. There would, then, have to be further production rules where abstract villainy, pursuit, marking of the hero, etc., are differentiated into their more concrete types.

In a pure regular grammar, which choice gets made at each application of a production rule is totally independent of the choices made at every other application of a rule. (This is because regular languages are a sub-type of "context free" languages, and is what gives both kinds of language their madlibs flavor.) Propp is at some pains to argue (pp. 109--113) that this is very, very nearly true of fairytales. The exceptions are few enough that they could, I think, be handled within the finite-state, regular-grammar framework, by expanding the set of non-terminal symbols a little.

To sum up, Propp did grammatical induction on fairytales by hand, in 1928, and came up with a regular language.

Naturally, I have questions.

  • How reliably can people identify his plot functions in the text of tales?
  • Do all tales in his corpus in fact fit his grammar?
  • How much interpretive violence is necessary to make an arbitrary story seem to fit his schema? If it's easy to warp anything so it seems to fit, this becomes much less interesting.
  • Do other story corpora show the same set of functions? If so, do they follow the same story grammar? (My sense is that the overwhelming majority of fiction I read either conforms exactly, or is very close.)

I am sure that folklorists must have tackled questions like this, and I would very much appreciate pointers to the literature.

[1] Propp, pp. 111--112, on how his conclusions "may also be verified experimentally":

It is possible to artificially create new plots of an unlimited number. All of these plots will reflect the basic scheme, while they may not resemble one another. In order to create a tale artificially, one may take any $A$, then one of the possible $B$'s then a $C\uparrow$, followed by absolutely any $D$, than an $E$, then one of the possible $F$'s, then any $G$, and so on. In doing this, any element may be dropped (except possibly for $A$ or $a$), or repeated three times, or repeated in various forms. If one then distributes functions according to the dramatis personae of the tale's supply or by following one's own taste, these schemes come alive and become tales. Of course, one must also keep motivations, connections, and other auxiliary elements in mind.
Unfortunately, Propp provides no samples of tales generated in this manner. If they have survived, it would be very interesting to read them. (Dundes, in his introduction to the 2nd English edition, mentions "programm[ing] a computer" to do this, but I haven't tracked down that reference (Alan Dundes, "On Computers and Folklore", Western Folklore 24 (1965): 185--189).

He immediately goes on:

The application of these conclusions to folk creation naturally requires great caution. The psychology of the storyteller and the psychology of his creative work as a part of the over-all psychology of creation must be studied independently. But it is possible to assume that the basic, vivid moments of our essentially very simple scheme also play the psychological role of a kind of root.
Again, this cries out for follow-up study, which may well have been done. ^.
Indra Das, The Devourers
Mind candy, historical fantasy/horror division: European shapechangers (not werewolves, exactly) in Mughal India, and modern Calcutta. Angst ensues. (I was very disappointed that the narrator's deep dark secret, at the very end, proves to be something as mundane as cross-dressing.)
Laura J. Snyder, Eye of the Beholder: Johannes Vermeer, Antoni van Leeuwenhoek, and the Reinvention of Seeing
Parallel lives of two 17th century inhabitants of Delft --- not revelatory as either art history or history of science, but deftly done, even to the explanations of some fairly involved optics.
Garrett Birkhoff, Hydrodynamics: A Study in Logic, Fact, and Similitude
This is Birkhoff surveying the state of hydrodynamics in 1950, and in particular looking at why some theoretical results so conspicuously fail to match observations (the "logic" part), and when the uses of physical scale models can be justified (the "similtude" part). For the former, his diagnosis is not just mathematical sloppiness on the part of physicists, or making inappropriate approximations, but taking inconsistent assumptions. The latter part largely turns on the method of dimensional analysis, its limitations, and how it can be seen as a special case of more general group-theoretic approaches to finding similar solutions to partial differential equations.
These are, of course, more general morals about mathematical modeling, but nobody who isn't pretty familiar with hydrodynamics and group theory will get anything out of this book.
An incidental observation: It's striking to me that Birkhoff cites many much (relatively) older works than a contemporary writer would, and that he cites plenty of French- and German- language publications. (I can't remember if he cites any Italian or Russian works in the original, rather than translations.) There is a little lesson here about the transformation of post-war science...
Donald B. Rubin, Multiple Imputation for Nonresponse in Surveys
"Imputation" is the more dignified name statistics gives for "making stuff up to fill in missing observations". (To be fair, that's a mouthful.) Rubin was, back in the day, a very forceful and necessary advocate for multiple imputation, i.e., for making up a whole bunch of different things to fill in the missing observations, and trying them all out, to make sure that your results aren't just creatures of accidents in your imputation. While this is clearly almost always a better idea than single imputation, there are also, clearly, some details that will need to be pinned down. This short (258 pp.) book does a remarkably good job of pinning down those details in an easy-to-follow way. It also includes a summary of Rubin's equally-influential work on missing data, i.e., when exactly it's a problem for what kinds of inferences. Some of the computational advice is antiquated, and I could have wished it was less Bayesian, but it's still a very nice piece of work.
Ob. "Why Oh Why Can't We Have a Better Academic Publishing System": Wiley has kept the book in print, as a "classic", but the list price is \$ 158, or over sixty cents per page.
Inga Clendinnen, Aztecs: An Interpretation
This is a brilliant book, a learned, intelligent and sympathetic attempt to try understand something of how things must have felt to what must stirke most readers as a very strange and unsympathetic society. I also cannot help but feel that huge chunks of it are massive speculations, starting from the first substantive chapter on Aztec notions of "the sacred" and going on from there. The sources seem to me just too thin, and too peculiar*, to support the very elaborate interpretations Clendinnen erects upon them. (Which doesn't mean she was wrong.) But she was the expert who was immersed in the source material, not me.
[*]: Largely, they are accounts given many decades after the conquest, and it seems unclear how much of them was the sources recalling what happened, as opposed to giving their views about what ought to have happened, or what they wanted Spanish missionaries and their helpers to think happened. Even if they were doing their best to stick to their memories of the facts, they couldn't possible have experienced, say, the new fire ceremony which took place every 52 years more than once, and that, perhaps, when they were quite young. ^
David E. Hoffman, The Billion Dollar Spy: A True Story of Cold War Espionage and Betrayal
In which the poisonous legacy of the Stalinist purges inspires a Soviet engineer to volunteer to spy for the US, extremely successfully.

Books to Read While the Algae Grow in Your Fur; Writing for Antiquity; The Commonwealth of Letters; Physics; Scientifiction and Fantastica; Enigmas of Chance

Posted at August 31, 2018 23:59 | permanent link

August 29, 2018

Data over Space and Time, Lecture 1: Introduction to the Course

Any lesson plan that survives first contact with the students is a good one. Have some slides:


(R Markdown source file).

Not un-related:

  1. I appear to be the last carbon-based life-form actually running Blosxom. I'd appreciate hearing about replacements with not too much of a learning curve.
  2. How does anyone ever finish a book? Do books even exist? Both Advanced Data Analysis from an Elementary Point of View and The Truth About Linear Regression seem to have entered a Zeno's paradox of revision...

Corrupting the Young; Enigmas of Chance

Posted at August 29, 2018 14:45 | permanent link

April 27, 2018

Course Announcement: Data over Space and Time (36-467/667), Fall 2018

Attention conservation notice: Notice of an advanced statistics class at a university you probably don't attend, covering abstruse topics you probably don't care about. Also, it's the first time the class is being offered, so those who do take it will have the fun of helping me debug it.

This course is an introduction to the opportunities and challenges of analyzing data from processes unfolding over space and time. It will cover basic descriptive statistics for spatial and temporal patterns; linear methods for interpolating, extrapolating, and smoothing spatio-temporal data; basic nonlinear modeling; and statistical inference with dependent observations. Class work will combine practical exercises in R, some mathematics of the underlying theory, and case studies analyzing real problems from various fields (economics, history, meteorology, ecology, etc.). Depending on available time and class interest, additional topics may include: statistics of Markov and hidden-Markov (state-space) models; statistics of point processes; simulation and simulation-based inference; agent-based modeling; dynamical systems theory.

Co-requisite: For undergraduates taking the course as 36-467, 36-401. For graduate students taking the course as 36-667, consent of the professor.

Course materials will be posted publicly on the class website (once that's up).

Corrupting the Young; Enigmas of Chance

Posted at April 27, 2018 09:15 | permanent link

April 12, 2018

Major depression, qu'est-ce que c'est?

Attention conservation notice: 1100+ words on a speculative scientific paper, proposing yet another reformation of psychopathology. The post contains equations and amateur philosophy of science. Reading it will not make you feel better. — Largely written in 2011 and then forgotten in my drafts folder, dusted off now because I chanced across one of the authors making related points.

As long-time readers may recall, I am a big fan of Denny Borsboom's work on psychometrics, and measurement problems more generally, so I am very pleased to be able to plug this paper:

Denny Borsboom, Angélique O. J. Cramer, Verena D. Schmittmann, Sacha Epskamp and Lourens J. Waldorp, "The Small World of Psychopathology", PLOS ONE 6 (2011): e27407 [Data, code, etc., not verified by me]
Abstract: Mental disorders are highly comorbid: people having one disorder are likely to have another as well. We explain empirical comorbidity patterns based on a network model of psychiatric symptoms, derived from an analysis of symptom overlap in the Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV).
We show that a) half of the symptoms in the DSM-IV network are connected, b) the architecture of these connections conforms to a small world structure, featuring a high degree of clustering but a short average path length, and c) distances between disorders in this structure predict empirical comorbidity rates. Network simulations of Major Depressive Episode and Generalized Anxiety Disorder show that the model faithfully reproduces empirical population statistics for these disorders.
In the network model, mental disorders are inherently complex. This explains the limited successes of genetic, neuroscientific, and etiological approaches to unravel their causes. We outline a psychosystems approach to investigate the structure and dynamics of mental disorders.

In the initial construction of the graph here, two symptoms are linked if they are mentioned in the DSM as criteria for the same disorder. That is, Borsboom et al. think of the DSM as a bipartite graph of symptoms and disorders, and project down to just symptoms. (There is some data-tidying involved in distinguishing symptoms and disorder.)

The small-world stuff leaves me cold — by this point it might be more interesting to run across a large-world network — but the model is intriguing. Each node (i.e., symptom) is a binary variable. The probability that node $i$ gets activated at time $t$, $p_{it}$, is a function of the number of activated neighbors, $A_{i(t-1)}$: \[ p_{it} = a + (1-a) \frac{e^{b_i A_{i(t-1)}-c_i}}{(1-a)+e^{b_i A_{i(t-1)}-c_i}} \] In words, the more linked symptoms are present, the more likely it is for symptom $i$ to be present to, but symptoms can just appear out of nowhere.

Statistically, this is a logistic regression: $b_i$ is how much symptom $i$ is activated by its neighbors in the graph, $c_i$ is a threshold specific to that symptom, and $a$ controls the over-all rate of spontaneous symptom appearance and disappearance. Using a very interesting data set (the National Comorbidity Survey Replication of about 9200 US adults), Borsboom et al. in fact fixed the $b_i$ and $c_i$ parameters by running logistic regressions. The $a$ parameter, which was kept the same across symptoms, was tweaked to make the rate of spontaneous occurrence not too unreasonable.

What Borsboom et al. did with this model was to run it forward for 365 steps (i.e., a year), and then look at whether, in the course of the previous year, it would have met the DSM criteria for major depression, and for generalized anxiety disorder, and then repeat across multiple people. It did a pretty good job of matching the prevalence of both disorders, and got their co-morbidity a bit too high but not crazily so.

Now, as a realistic model, this is rubbish, for a host of reasons. Lots of the edges have to be wrong; the edges should be directed rather than undirected; the edges should be weighted; the logistic form owes more to what psychologists are used to than any scientific plausibility. (Why should psychopathology be a spin glass?) The homogeneity of parameters across people could easily fail. And yet even so it comes within spitting distance of reproducing the observed frequencies of different conditions, and their co-morbidities.

Notice that despite this, there are no underlying disease variables in this network, just symptoms. So why do we believe that there are unitary disease entities? I can see at least three routes to that:

  1. Perhaps this symptom-network model simply fails to match the detailed statistics of the data, while latent-disease-entity models can. This might be a bit boring, perhaps, but it would be persuasive if one could show that no model without the disease entities could work. (I find that dubious, but my doubt is not evidence.)
  2. Alternately, one might appeal to causal autonomy. The temperature of a gas, in a strong sense, amounts to the average kinetic energy of its molecules, and one can accurately simulate gases at a molecular level without ever invoking the notion of temperature. But if I manipulate the gas to have a certain temperature, then, very quickly, the effects on pressure and volume, and even the velocity distribution of individual molecules, is always (pretty much) the same, no matter how I bring the temperature about. This is what lets us give sensible causal, counter-factual accounts at the level of temperature, and thermodynamics more generally. (Cf. Glymour.)
    Now, in the network model, we can imagine "giving someone" generalized anxiety disorder, by activating some set of nodes which meets the DSM criteria for that condition. There are actually multiple, only partially-overlapping symptom sets which will do. In the network model, these different instantiations of generalized anxiety disorder will have similar but not identical consequences (for other symptoms, duration of the condition, response to treatments, etc). If, in reality, it makes no difference how someone comes to meet the criteria for generalized anxiety disorder, the implications for the future are always the same, that would be a powerful argument that the disorder is something real.
    More medically: think how we distinguish diabetes into type 1 (the body doesn't make enough insulin) and type 2 (the body doesn't respond properly to insulin). This is, I'd say, because they differ greatly in their causal implications, but once you find yourself in one of these classes, it makes little difference how you got there.
  3. It could be that a description in terms of higher-level entities like depression allows for a higher efficiency of prediction than just sticking with symptoms. This notion could even be made fairly precise; it may also end up being the same as the second route.

Of course, it might be that to make any of these three defenses (or others which haven't occurred to me) work properly, we'd have to junk our current set of disorders and come up with others...

Minds, Brains, and Neurons; Networks; Enigmas of Chance

Posted at April 12, 2018 14:30 | permanent link

April 01, 2018

An _Ad Hominid_ Argument for Animism

Attention conservation notice: Note the date.

A straight-forward argument from widely-accepted premises of evolutionary psychology shows that humans evolved in an environment featuring invisible beings with minds and the ability to affect the material world, especially through what we'd call natural forces.

  1. (Premise) Humans have evolved psychological modules, which carry out specific sorts of computations on very specific sorts of representations, as triggered by environmental conditions. These modules are in fact adaptations to the "environment of evolutionary adaptation", or, rather, environments.
  2. (Premise) Indeed, when we encounter a human cognitive module, we should presume that it is an evolved adaptation.
  3. (Premise) Humans have modules for theory-of-mind, social exchange, and otherwise dealing with intentional agents by reckoning with their beliefs, desires, intentions, and (crucially) capacities to act on those intentions.
  4. Therefore, the human modules for theory-of-mind, social exchange, and dealing with intentional agents are evolved adaptations to our ancestral environment.
  5. (Premise) Humans often engage those modules when dealing with invisible beings, often manifesting as (what scientists categorize as) natural forces.
    (In fact, such engagement of those modules was near-universal up to the emergence of WEIRD societies. The historical record shows aberrant individuals who did not do this, but it's plain even from texts those individuals authored, when they have come down to us, that their bizarre behavior had absolutely no traction on the vast, neurotypical majorities of their societies. [One is reminded of the militantly color-blind trying to convince others that colors do not exist.] Moreover, treating natural forces as manifestations of invisible beings who are intentional agents, amenable to bargaining, threats, supplication, etc., etc., is still very common in WEIRD societies, perhaps even modal.)
  6. (Premise) Engaging a wrong or inappropriate module is expensive, even potentially dangerous, and thus mal-adaptive, and so should be selected against.
  7. If natural forces are mindless and invisible beings did not exist in the EEA, then engaging theory-of-mind and social-exchange modules to deal with natural forces and invisible beings would be mal-adaptive.
    (Occasionally, people suggest that it's so dangerous to ignore another intentional agent that it was adaptive for our ancestors to suspect intentionality everywhere, on "better safe than sorry" grounds. I have never seen this supported by a concrete calculation of the costs, benefits and frequencies of the relevant false-positive and false-negative errors. I have also never seen it supported by a design analysis of why our ancestors could not have evolved to realize that storms, earthquakes, droughts, diseases, etc., were no more intentional agents than, say, fruit, or stone tools.)
  8. Since those modules are adaptive, we must conclude that invisible beings with beliefs, desires, intentions, and the power to act on them, especially through "natural" forces were a common, recurring, predictable feature of the environments of evolutionary adaptation.

Of course, none of this implies that those invisible beings aren't as extinct as mammoths.

To spoil the [not very funny] joke: even if the relevant modules exist, they are engaged not by intentional-agent-detectors, but by human mental representations of intentional agents. Once the idea starts that storms are the wrath of some invisible being, that can be self-propagating. For further details, I refer to the works of Dan Sperber, especially Explaining Culture (and to some extent Rethinking Symbolism). Credit for the phrase "ad hominid argument" goes, I believe, to Belle Waring, back in the Early Classic period of blogging.

Learned Folly; Minds, Brains, and Neurons

Posted at April 01, 2018 22:59 | permanent link

March 31, 2018

Books to Read While the Algae Grow in Your Fur, March 2018

Attention conservation notice: I have no taste. I also have no qualifications to opine on the sociology of experts and anti-intellectualism, or on the history of Islamic civilization.

Andrea Camilleri, A Nest of Vipers and Pyramid of Mud
Mind candy mysteries, the nth and (n+1)st in the series (previous entries to be found here), with all the usual pleasures. But Nest was first one I can remember where I worked out whodunnit before Montalbano. Whether this means I am getting better, or Camilleri (or Montalbano?) is slipping, is not for me to say.
Ruth Downie, Memento Mori
Mind candy historical mystery: latest (previously) in her series about an ex-legionary doctor and his British (i.e., barbarian native) wife, and their misadventures in Roman Britain --- this time, Aquæ Sulis (i.e., Bath). It continues to be really good, though I am a bit worried about where the end of this book leaves our protagonists.
Ausma Zehanat Khan, The Bloodprint
Mind-candy epic fantasy. On those terms, it's enjoyable enough, though unremarkable. What makes it really notable is that it's entirely based on the recent history of Afghanistan and environs, with a magic system based on Quranic recitation. Many reviewers have noticed that "the Talisman" = the Taliban, but none I've seen have remarked on the other correspondences. (This does not pretend to be a complete list.)
  • Khorasan = Afghanistan
    • West Khorasan = eastern Iran
    • North Khorasan = Trans-Oxiana, ex-Soviet Central Asia
  • Candor = Kandahar, Hira = Herat (I did not notice counter-parts for Kabul, Mazar-i-Sharif, Jalalabad, or Ghazni)
  • Sailing Pass = Salang Pass
  • Zergil tribe = Gilzai tribe
  • Valley of the Five Lions = Panjshir
  • the Claim = the Qur'an (I'm not sure which translation Khan used, but I am pretty sure every verse of the Claim is in fact from the Qur'an)
  • the Hazaras and Hazarajat = the Hazaras and Hazarajat
  • Turquoise mountain = Turquoise mountain
  • the Wall = the border with the former Soviet Union
  • The Authoritan = Karimov
  • basmachi = basmachi
  • Jaslyk prison and boiling dissidents alive = Jaslyk prison and boiling dissidents alive

My only surprise is that Khan's family are evidently Pashtuns from Pakistan, rather than from Afghanistan.

Tom Nichols, The Death of Expertise: The Campaign Against Established Knowledge and Why It Matters
This is an expansion of a blog post that went viral. It's easy enough to find summaries (e.g.), so I will skip giving one. It's good, as far as it goes, but I don't think it really goes far enough, or probes deeply enough. I will offer some critical remarks, but I want to emphasize that I do think this is a good book, by a skilled and well-intentioned writer, and worth reading. (Also, I realize that some of my complaints amount to wishing he'd written an academic treatise rather than a popular essay.)
  1. Nichols starts with a sensible definition (p. 29): "I will use the words 'professionals,' 'intellectuals,' and 'experts' interchangeably, in the broader sense of people who have mastered particular skills or bodies of knowledge and who practice those skills or use that knowledge as their main occupation in life". Unfortunately, he does not really wrestle with how this opens up a line of attack on expertise, which is that a profession is really a group of people who make their livings by claiming to have mastered what they say is a shared, unusual body of knowledge. But maybe all they have really mastered (whatever they might think) is a shared body of jargon, mistake, fallacy and delusion --- their purported knowledge might really be bogus. Haruspicy, or even homeopathy, would be too easy as targets. Take something more mainstream: there are many people who claim to be experts in personality testing, and sell their services to schools, corporations, law courts, etc. But the methods they use are well-established to be scientifically worthless, even by the not-exactly-demanding standards of social psychology. Or, perhaps more consequentially, we have created a profession of "forensics" ("forensic science", even), which we use to help make literally life-or-death decisions in the courts. It turns out that most of the methods employed (and taught, and certified) are unscientific and untrustworthy, and that's before we get to admissions of massive failure stretching over decades. (Beside things like this, my pet peeve of statsiness hardly matters, but it's in the mix, too.) The personality testers and the forensic scientists, while organized professionals, aren't really experts, just specialists. And if that's true of those professions, how many others might also be mere bogus specialisms? What about, say, the law?
    Yes, we live in a vastly elaborated division of labor and responsibility, where different professions claim jurisdiction over different specialized tasks, but how much of that is jargon and bullshit, rather than genuine knowledge? Maybe that's inevitable, maybe deferring to specialists who are just (collectively) making shit up is the price we pay for our prosperity, but can you really blame anyone for resenting that, or for refusing to acknowledge the authority of non-existent knowledge? (Proverbially: "Don't piss on my leg and tell me that it's raining.") Most people are, after all, much more likely to be told by a boss to take a personality test than to interact with, say, a rocket engineer.
    Or, worse: I have been writing as though some profession just are bogus, no question, because that's what I happen to think. But a more neutral description would be that these are conflicts between rival groups of professionals, where one group (personality testers, forensic technicians) claim expertise and the authority that goes with it, while another group (social psychologists, statisticians) attacks the first group's claims. If you are unable or unwilling to enter into those specific debates on their merits, you might well think that all the claims to expert knowledge here are bogus, perhaps merely rationalizing professional interests. At the very least, you should be willing to entertain it as a possibility.
    Now, I do not think this nightmare vision is correct. (I also do not believe in my neutral model of inquiry.) And I know that the overwhelming majority of the people driving Nichols to despair aren't making an argument this coherent. But this is kind of idea many of them are fumbling towards, and I wish this book had addressed it. Instead, the chapter on "when the experts are wrong" limits itself to specific mistakes (e.g., is it healthy to eat eggs?), rather than the possibility of global, albeit sincere quackery (do "dieticians" actually know anything?).
  2. This brings me, indirectly, to a curious omission. The book never mentions the well-funded and well-organized campaigns by vested corporate interests to cast doubt on expert opinions when those are inconvenient. The paradigm case is tobacco companies trying to obfuscate the dangers of smoking, but of course the same tactics, and even the same personnel, got recycled for global warming. (Merchants of Doubt is the essential reading here, but good journalists following the public relations industry have been warning about this stuff for decades. [I don't think Nichols ever mentions PR; it's not in his index.]) Some of this has been, pretty explicitly, about attacking scientific expertise as such. Even more of it has been about creating confusion about who is an expert and what the expert consensus is. The goal is to get lay people to see dueling experts who can't agree on anything. To the extent this has been successful, sensible lay people should show less deference to self-described expert opinion. More precisely, once lay people are persuaded that experts are always in deep disagreement, and that there is even no consensus on who the experts are, they should show less deference to opinions which are presented to them as those of experts. [0]
    (Relatedly: if many people are now telling their doctors what drugs to prescribe them, might this have something to do with drug makers spending massive amounts on advertising at patients, aiming to achieve just this?)
  3. Nichols has his political and ideological commitments, which show through in places. This is fine (at least, I hope it is, for my own sake), except that I think it may have led him in places to pass over facts and cases which would complicate (though perhaps not refute) his argument. For instance: as an example of journalists failing to abide by the principles of their calling, he mentions Sabrina Ederley and Rolling Stone (pp. 163--4). He does not mention, say, Stephen Glass and The New Republic; that comparison would have been instructive, since, after all, Glass's fabrications happened decades before what Nichols sees as the recent corruption of journalism.
    Or, again, Nichols is very unhappy about the way many college students these days seem to be very resistant to hearing opposing viewpoints (chapter 3). Well, so am I [1]. Here, though, are some things which are notable by their absence from this chapter:
    1. evidence that this failing of undergraduates is worse now than it has been in the past;
    2. evidence that contemporary undergraduates are worse about this than their age-mates who don't go to college;
    3. evidence that college makes this worse among those who attend;
    4. argument, and supporting evidence, connecting this failing to the way colleges and universities increasingly treat undergraduates as customers to be pleased, rather than students to be taught, something the book righteously, and rightly, denounces [2].
    None of this means that he's wrong, but it is a failing, by the standards of argument and inquiry which Nichols, very properly, espouses, and which I'm sure he'd adhere to in his more strictly-scholarly work.
    As it happens, since the book was published we have had some evidence about (a)--(c), which would at the very least complicate the argument of the chapter. Of course, no one study is definitive, and there are specific ways one might attack this one. (E.g., perhaps different cohorts, or different educational groups, use the word "racist" very differently, so that apparently small variations in tolerance for "racist" speech conceals huge actual differences [cf. Becker].) But this is the kind of thing I wish Nichols had engaged in, or at least marked out as a very clear area of relevant uncertainty, rather than simply editorializing.
  4. That last point brings me to a larger failing of the book, which is its failing to make the necessary historical case [3]. The book wants to argue both that the relationship between experts and the lay public is bad now, and that it has gotten worse, that it is in a uniquely bad state. For the first part, examples of contemporary dysfunction are persuasive, though I would have liked those to have been more systematic (if not necessarily quantitative, though that would have been nice too).
    But for the second part, the claim that things are worse now than they've ever been, there needs to be comparative, historical evidence, and that's just not there. Hofstadter's Anti-Intellectualism in American Life gets mentioned (pp. 18--19; oddly calling Hofstadter a "political scientist", rather than a historian). But this book never really wrestles with Hofstadter's, or with the larger historical literature of which the latter is a part. Those histories at least seem to show that there has always been a very strong current in American life which despises intellectuals, experts, and education (even if it's still fascinated with technology). One could even make out a counter-scenario that Nichols's "death of expertise" is just America reverting to normal, after a transient, out-of-character Cold War episode, when competing with the USSR over the Space Race and the Third World lent value to all sorts of obscure skills. That wouldn't make our current plight any better, but it would change the diagnosis! N.B., I am not not claiming that's the right interpretation; I don't know, but I wish Nichols had considered such alternatives, or at least made his necessarily-historical case better.
    To put it more briefly: Chapter 3 ends (p. 104) with the heart-felt plaint that
    [I]f college graduates can no longer be counted on to lead reasoned debate and discussion in American life, and to know the difference between knowledge and feeling, then we're indeed in the kind of deep trouble no expert can fix.
    I'm a bit skeptical that we ever could count on college graduates to play that role in American life, not least because many generations of earlier writers lamented that they didn't [4].
Again, just to recap: I liked the book, and I share a lot of Nichols's exasperations and anxieties. I think the book is worth reading. But I wish, in the name of the standards it very rightly advocates, it had gone much deeper.
[0]: There is another, related issue which should have made it into the book, but didn't. This is the trend, over the last few decades, towards removing expertise from public bodies, in favor of relying on private industry and its lobbyists. The last chapter of this book is about the role of experts in democratic policy-making. It emphasizes, correctly, that in a modern, representative democracy, people delegate power and authority to elected representatives, who are expected to know more about issues than most citizens. At the same time, even those representatives cannot master all of the intricacies of every issue, and so must consult experts. (Such consultation isn't, or shouldn't be, just doing what experts say to do.) At most, some areas of policy might be further delegated to appointees, but even that is revocable. All this is basic and right as far as it goes, but the disturbing development over recent decades is the on-going erosion of the government's own capacity for expertise, whether in the legislative or the executive branch, in favor of relying on private industry and its lobbyists, sometimes laundred through think-tanks. To be fair, this continues to trade on the appearance of disinterested expertise, rather than de-legitimizing it. ^
[1]: For the record, as threats to academic freedom, I rank ranty undergraduates, however appalling, below administrative branding and fund-raising practices, interfering state legislatures and/or trustees and donors, and the increasing use of un-tenured teachers. The actions of administrators, legislators, and donors are, arguably, relevant to the topic of decreasing deference to expertise, but I also think it is defensible for the book to not generally explore the threats to academia. ^
[2]: The book doesn't say why schools have come to see undergraduates that way, though it leaves the impression that this was a sort of unforced error on the part of university faculty and administrators. It might be that, but it might also have something do to with a shift in how we support the bulk of higher education, away from state governments directly funding their universities and towards the federal government underwriting student loans. (This would seem to be worth investigating, if it hasn't been already.) ^
[3]: I realize that I made a similar complaint about Chris Hayes's Twilight of the Elites. I would pay good money to read a dialogue between Hayes and Nichols. ^
[4]: Another encounter I would pay good money to observe would be Nichols's grappling with the late Jacques Barzun's 1959 The House of Intellect. ^
Richard W. Bulliet, The Case for Islamo-Christian Civilization
This is in some-ways a period piece, the period being 2002--2003. It is nonetheless full of good sense, especially in the title essay.
(I cannot decide whether the definite description of Edward Said on p. 59 --- "Elegantly attired Palestinian professors at renowned universities write cutting-edge works that command worldwide respect" --- is affectionate or catty.)

Books to Read While the Algae Grow in Your Fur; Pleasures of Detection, Portraits of Crime; Writing for Antiquity; Afghanistan and Central Asia; Scientifiction and Fantastica; The Beloved Republic; Learned Folly; Islam; Tales of Our Ancestors

Posted at March 31, 2018 23:59 | permanent link

February 28, 2018

Books to Read While the Algae Grow in Your Fur, February 2018

Attention conservation notice: I have no taste. I also have no qualifications to opine on the theory of measurement in psychology, the philosophy of science, the law, race relations, or criminology.

Joel Michell, Measurement in Psychology: A Critical History of a Methodological Concept
Comments having passed the 1500 word mark, including long quotations, this will have to be a separate review.
H. P. Lovecraft, At the Mountains of Madness
This is an umpteenth re-read, of course. (I tend to do them in the winter.) This one made me want to read a history of subsequent Elder Thing archaeology, where the mountains and the city were revisited during the International Geophysical Year, and it's become obvious that 99% of this is as much a product of the discoverers' imagination and preconceptions as, say, Arthur Evans's views of the Minoans. (But that 1%...)
ObLinkage: The Lovecraft Reread tackles AtMoM in three parts.
Lauren Willig, The English Wife
Mind candy historical mystery, set in New York and London just a bit before 1900. An interesting aspect of the writing is that here, as in her historical romance novels, Willig uses two time-lines, where the characters in one time-line are trying to discover what happened in the other. But in the romances the time-lines are parallel, whereas here they converge. What this signifies, I couldn't say.
Peter Godfrey-Smith, Theory and Reality: An Introduction to the Philosophy of Science
I can easily say that this is the one of the best modern introductory books on the philosophy of science I've ever read. (Another, of a very different sort, is William Poundstone's Labyrinths of Reason.) It's presented roughly historically, beginning with Logical Positivism and moving forward, through Popper, Kuhn, such post-Kuhnians as Lakatos, Feyerabend and Laudan, and classic 1970s/1980s "sociology of scientific knowledge", before ending with a range of contemporary topics. Throughout, Godfrey-Smith strikes a good balance between persuading the reader that there are problems worth wrestling with, and that they're not hopeless.
To the former: too many scientists, encountering issues from the philosophy of science, find them pointless, or at most things which could be cleared up in an afternoon with a little clear thinking and maybe some algebra. (Occasionally this results in weird little cults like self-styled "strong inference", which is firmly put in its place here.) Godfrey-Smith is very good at conveying how there are real issues here, which very smart people have wrestled with, without coming to any truly satisfactory answers.
This then raises the possibility that the exercise is futile, not because it's unimportant but because it's doomed, that the problems are just too hard for us. Against this, Godfrey-Smith is good at conveying how, if we're still confused about questions like "When does observing something that a theory predicts confirm the theory?", or "How can the social organization of a scientific community support its cognitive goals?", we're at least understanding the issues much better. (For example, it's become very clear that social organization does matter.)
This book is worthwhile reading for any scientist interested in philosophical issues. It might be even more worthwhile for those who aren't interested, but...
--- Two thoughts which occurred to me while reading Godfrey-Smith's discussion of how "naturalistic" philosophy of science is anti-foundationalist, in the sense of eschewing the search for philosophical foundations for the sciences which are somehow prior to the sciences themselves.
  1. Strong forms of this would say that such foundations are impossible or undesirable. A weaker form, however, would compare the track-records of philosophy and science, and say that it's rash to expect philosophy to be more secure than (say) neurophysiology any time soon. (Where this would leave, say, social psychology is a nice question.) I am not sure whether anyone has taken this position within the philosophical literature, or even what it would be called.
  2. Saying that we will use the results of scientific inquiry to understand the process of scientific inquiry can sound like a vicious circle, but can also, more reasonably, be just a self-consistency check. If our best scientific understanding of the world and ourselves implied that scientific inquiry was unreliable, we would have a real problem. Worries about science being self-undermining are a a long-running theme in the history of the sort of philosophy of science that Godfrey-Smith writes about, going back before the Logical Positivists into the nineteenth century (see, e.g., Leszek Kolakowski's The Alienation of Reason / History of Positivist Thought from Hume to the Vienna Circle and his Husserl and the Search for Certitude), and continues on today (naturally in meme format). Even if all naturalistic philosophy of science achieves is showing that science doesn't undermine itself the way that the more ambitious and outrageous forms of sociology of knowledge do, this would be a real accomplishment.)
Richard Thompson Ford, The Race Card: How Bluffing about Bias Makes Race Relations Worse
Let me spoil the ending:
No doubt some readers will wish to ask whether I really think playing the race card is now the biggest racial justice issue this society faces. No, I don't. I hope it's clear that I believe old-school bigotry remains a severe social problem and that subtler and systemic racial disadvantages --- even when they can't be blamed on "racists" --- are profound social evils that demand redress. These are bigger problems than playing the race card. But the race card is an impediment to dealing with these problems. It distracts attention from larger social injustices. It encourages vindictiveness and provokes defensiveness when open-mindedness and sympathy are needed. It leads to an adversarial, tit-for-tat mind-set ("You're a bigot!" "No, you're just playing the race card!") when a cooperative spirit of dialogue is required.
The race card is symptomatic of a real crisis in the way we currently think and talk about race: a crisis borne of our failure to keep up with a changing social world, a crisis of social change and of intellectual stasis. We need new intellectual tools and new language to deal with the new realities of American racism. Thus far we've failed to develop them, so we find ourselves increasingly unable to discuss issues of race intelligently and convincingly. We find ourselves listening to and repeating the slogans and catch-phrases of the past, whether or not they apply, like a catechism that's long since lost its power to invoke or inspire, or like a curse that damns guilty perpetrator and innocent bystander with indiscriminate contempt. [p. 349]
And this was in 2008! (Ford's skepticism about the Implicit Association Test is looking pretty good these days. His confidence that open expressions of outright racism have been driven to the fringes of American public life, maybe not so much.)
More constructively, I found chapter 2's discussion of "racism by analogy" thought-provoking, and chapter 3 on legal criteria for discrimination and disparate impact quite eye-opening.
John Pfaff, Locked In: The True Causes of Mass Incarceration --- and How to Achieve Real Reform
This is a thoughtful book about the causes of mass incarceration, and what can and should be done to reverse it. I should say at the beginning that Pfaff is as outraged as anyone about how many people we have in prison (or otherwise subject to "corrections"), so that when Pfaff challenges elements of what he calls the "standard story", it's not to minimize the disaster and disgrace, it's to help efforts at reform actually improve things. I found a lot of it convincing, but I should say up-front that I haven't tried to independently check any of Pfaff's figures or calculations.
The most convincing parts of the preliminary de-bunking are as follows:
  1. Private prisons are awful, but they are quantitatively too small to account for mass incarceration. Also, the lobbying efforts of private prison corporations are too small, and come too late in the surge in incarceration, to explain it.
  2. Most of our prison population isn't there for drug offenses, or non-violent offenses in general, but for violent crimes, and so undoing mass incarceration will mean changing how we deal with those convicted of violence. Pfaff presents this as a refutation of the idea that mass incarceration is due to the war on drugs, which I think is a bit too hasty (as I will explain below).
  3. Maximum legal prison sentences have gone way up, and longer prison terms would naturally lead to more people being in prison. But this can't explain most of the growth in incarceration, because the actual average length of time served hasn't increase very much.
It then behooves Pfaff to explain why, in his view, we have so many more people in prison than we used to, even adjusting for population. Implicitly --- this is a popular book and he does no explicit models here --- he works with a "compartment" model, where the compartments or stages are something like: \[ [\text{Commit crime}] \Rightarrow [\text{Arrested}] \Rightarrow [\text{Charged}] \Rightarrow [\text{Convicted}] \Rightarrow [\text{Prison}] \Rightarrow [\text{Release/Parole}] \] where at each stage before prison one might be diverted away (e.g., arrested but not charged), and prison is of course of variable duration. The advantage of approaching the question "why are so many people in prison?" this way is that if you can track the number of people in each stage, and the flows of people from one stage to the next, they have to add up: the number of people in prison on 1 July 2018 will be the number who were in prison on 1 July 2017, plus those convicted and sentenced over the year, minus those released over the year. (At the risk of being dis-respectful, I am counting deaths in prison under "release".) Changing the proportions who go on from one stage to the next changes the flows, and hence will accumulate over time to the number of those in prison.
Pfaff claims that the big change which drove up the number of people in prison wasn't at the stage of being arrested, or convicted, or even the length of time spent in prison, but rather in the proportion of those arrested who are actually charged with a crime. This is a decision made by local public prosecutors. If we believe Pfaff's numbers, this locates a key source of the problem.
Unfortunately, as he is at pains to say, we have very little systematic information on prosecutors' offices and how they make their decisions. We do know that they face a somewhat perverse set of incentives, in that declining to charge someone who goes on to do something bad is electoral poison, but charging someone who's really harmless has almost no downside (for the prosecutor; it has plenty for the person charged, and their family and community). Prosecutors also face little opposition from public defenders, which is a big part of why almost all criminal charges are settled by plea deals, not brought to trial. The whole business is a mess, with almost no accountability (either to hierarchical superiors or to the democratic public), and scarcely any systematic reporting. Pfaff does not attempt to say why any of these issues should have gotten worse during this period, however.
Popular books about policy or social problems usually have a last chapter which talks about what to do about the issue. Pfaff follows this practice, and, as usual, it's the weakest part of the book, because his proposals are so much smaller than his own account of the scale of the problems. (Whether this is better or worse than the alternative tradition, of proposing measures which would solve the problem but also be totally unworkable, is a nice question.) In no small part this is because he has fairly convincingly localized the problem, but he's localized it not so much to a black box as to a mob of 3,000-odd ill-coordinated black boxes.
--- I said above that I am not sure Pfaff is entirely fair to the blame-the-drug-war camp; in particular, I think he ignores a fairly obvious counter-argument. He attacks the idea that the growth in incarceration is a result of the war on drugs, by pointing out that only a minority of those in prison are there for non-violent drug offenses, while the majority are there for violent crimes. Grant that this is true (as I said, I haven't checked his figures*.) How much of that violence is due to the war on drugs? Legal businesses get robbed, of course, and from time to time one even reads of, say, dentists conspiring to assassinate rival dentists, but this sort of thing is rare in trades where the law is available to settle disputes and protect property. A criminalized but lucrative drug trade, on the other hand, seems conducive to violence. Localizing the trade to specific neighborhoods make those dangerous, law-less places, further inciting violence (cf. Allen and Leovy). Effects like these are hard to quantify --- we can't just read them off from administrative data, as Pfaff likes to do --- but they could be very important. I'm not sure where this leaves us.
*: One point which would be good to check is how possessing of a firearm while committing another crime gets coded in these records. If every drug-dealer who gets busted while also carrying a gun counts as "violent", for example, that might make a substantial difference. (Or it might not; that's why someone should check.) ^
Danielle Allen, Cuz: The Life and Times of Michael A.
A memoir of the life, imprisonment and death of Allen's cousin Michael. It's at once the specific story of a unique person and their family, and a slice through what's gone wrong with our country*, that someone could be thrown in prison for eleven years for some stupid crimes committed at fifteen (where Michael was the only one hurt), ultimately setting his life on a path where, at age 29, his corpse was found in a shot-up car on the street. Michael made bad choices, which Danielle never shies from, but he made them in a foolishly, evilly un-forgiving context, in a society which essentially threw his life away for no good reason, and that is messed up. It's horribly, horribly sad, but beautifully told.
Disclaimer: I know Prof. Allen, and have participated in a series of workshops she organized and contributed to a book she edited, but I feel under no obligation to write a positive notice of her books.
*: One of the things which makes this a complicated book is that it is also, implicitly and in glimpses, the story of what has gone right with our country that it now creates people like its author. ^

Books to Read While the Algae Grow in Your Fur; The Beloved Republic; Cthulhiana; Minds, Brains, and Neurons; Philosophy; The Collective Use and Evolution of Concepts; Pleasures of Detection, Portraits of Crime; Tales of Our Ancestors; Commit a Social Science

Posted at February 28, 2018 23:59 | permanent link

January 31, 2018

Books to Read While the Algae Grow in Your Fur, January 2018

Attention conservation notice: I have no taste. Also, I have no qualifications to opine on criminology, or the history of millenarian movements and the Russian Revolution.

Maria Konnikova, The Confidence Game: And Why We Fall for It... Every Time
An engaging popular-science look at confidence games, their players and their marks. (Konnikova references a lot of the social psychology literature, which is certainly better than ignoring it, but I haven't had the heart to check how many of those studies have failed to replicate.)
Yuri Slezkine, The House of Government: A Saga of the Russian Revolution
TL;DR: It's about some Russians.
This book is a lot of things: at barest bones, a look at the history of the Bolshevik party, the Russian Revolution and the USSR from, say, the 1880s down to about the out-break of World War II. But it is also a kind of collective biography of the Old Bolsheviks, which particularly emphasizes their imaginative lives as readers and as writers of literature, and their family lives. It is also an analysis of Bolshevism as a millenarian sect, closely following Norman Cohn's Cosmos, Chaos, and the World to Come and (less crucially) Mircea Eliade. (On the one hand, this point is kind of obvious to any non-Bolshevik from the definitions; on the other, I know of nobody else who has (i) worked it through in detail, without (ii) being a propagandistic right-wing hack-job.) This leads to looking closely through the Bolshevik's literary output for mythological themes and symbols, especially re-workings of Exodus and of creation out of the primeval swamp. It is an account of the up-bringing and youth of the children of the Old Bolsheviks, and of how they became patriotic Soviet citizens without really getting Marxism. It examines architecture, winter holidays, witch-hunts from early modern Germany to 1980s America, and window curtains. It is the story of the building, life and decay of a particular building in Moscow, the eponymous House of Government. Finally, it is the story of the many awful things which the Old Bolsheviks did and suffered. It is vast, detailed, humorous, learned, intensely arguable (*), and over-all magnificent.
One comment seems worth making: it is striking to me how modestly the occupants of the House of Government lived, for the unchecked rulers of a huge country. A four-room apartment, a nanny, the shared use of a vacation home --- this put them near the pinnacle, which is to say, on a par with moderately successful big-city professionals and executives in the contemporary west. (Some of the provincial managers seem to have been more ostentatious.) I think this really does indicate that whatever else might be said about them, they weren't in it for personal gain. Of course, living like the western upper-middle class in a country where millions of people were literally starving to death indicates incredible relative inequality...
Finally, I feel compelled to mention that I actually "read" this by listening to the audiobook, read by Stefan Rudnicki, who did an absolutely magnificent job at delivering the text, and in particular capturing Slezkine's use of repetition as a deliberate rhetorical device. (I can't judge Rudnicki's pronounciation of Russian.)
*: When I was in college, under the spell of Eliade and (less defensibly; but I was an adolescent) Joseph Campbell, I tormented my humanities teachers with analyses of literary works along the same lines as what Slezkine does here. They were very patient with me, and eventually got me to see that this mode of interpretation is just too flexible, that there is basically nothing it couldn't seem to account for, hence uninformative. (As I would now put it, the Rademacher complexity is too high.) I am not saying that Slezkine's efforts are on a par with my undergraduate effusions, but I do wonder, once he's decided that such-and-such a period's novels are variants on Exodus, how hard is it for him to find examples? how hard would it be for him to find Exodus stories from other periods, if he wanted to? how hard would it be for another critic to take the same text and read it as a variant on Genesis?
David N. Schwartz, The Last Man Who Knew Everything: The Life and Times of Enrico Fermi, Father of the Nuclear Age
This is a nice biography of Fermi, who wasn't, of course, the last man who knew everything (Schwartz says as much!), but was the last great physicist to be both a great theorist and a great experimentalist, and whose work helped create the world we live in. It's not ground-breaking (Schwartz has no pretensions in that direction), but it is very readable, and especially good at explaining the physics, with the imagined reader being an intelligent non-scientist, albeit one who sort of remembers what atoms and electrons are.
The one complaint I have is that I wish Schwartz had taken the space to explain and work through at least one of the canonical "Fermi problems". This would have made his descriptions of how Fermi worked much more concrete. As it is, those passages come across as quite abstract, and perhaps unconvincing. (After all, what who wouldn't prefer to ignore the irrelevant aspects of a problem?)
Jim C. Hines, Terminal Alliance
Mind candy: comic science fiction from a post-apocalyptic future, told from the view-point of military janitors. In addition to being funny, Hines has done a much better job of world-building than many writers of ostensibly more serious SF.
Mira Grant, Into the Drowning Deep
Mind candy techno-thriller / predator porn, set just a few years into the science-fictional future, featuring carnivorous mermaids. Grant has clearly given a lot of loving attention to their biology, and I look forward to the nigh-inevitable sequel.

Books to Read While the Algae Grow in Your Fur; Scientifiction and Fantastica; Physics; Writing for Antiquity; The Progressive Forces; Commit a Social Science; Minds, Brains, and Neurons; Pleasures of Detection, Portraits of Crime;

Posted at January 31, 2018 23:59 | permanent link

Three-Toed Sloth