Attention conservation notice: I have no taste, and no qualifications to opine on anti-discrimination law, early 20th century shock art movements, early 20th century science fiction, or the Renaissance reception of classical mythology. Also, most of my reading this month was done at odd hours and/or while bottle-feeding a baby, so I'm less reliable and more cranky than usual.
Books to Read While the Algae Grow in Your Fur; Writing for Antiquity; Scientifiction and Fantastica; Teaching: Statistics of Inequality and Discrimination; The Beloved Republic; The Commonwealth of Letters
Posted at December 31, 2022 23:59 | permanent link
Attention conservation notice: 1800+ words of academic self-promotion, boosting a paper in which statisticians say mean things about some economists' favored toys. They're not even peer-reviewed mean things (yet). Contains abundant unexplained jargon, and cringe-worthy humor on the level of using a decades-old reference for a title.
Entirely seriously: Daniel is in no way responsible for this post.
Update, December 2022: Irritatingly, there are some small but real bugs, glitching all our numerical results. This is an even stronger reason for you to direct your attention elsewhere. (Details at the end.)
I am very happy that after many years, this preprint is loosed upon the world:
To expand a little: DSGE models are models of macroeconomic aggregate quantities, like levels of unemployment and production in a national economy. As economic models, they're a sort of origin story for where the data comes from. Some people find DSGE-style origin stories completely compelling, others think they reach truly mythic levels of absurdity, with very little in between. While settling that is something I will leave to the professional economists (cough obviously they're absurd myths cough), we can also view them as statistical models, specifically multivariate time series models, and ask about their properties as such.
Now, long enough ago that blogging was still a thing and Daniel was doing his dissertation on statistical learning for time series with Mark Schervish and myself, he convinced us that DSGEs were an interesting and important target for the theory we were working on. One important question within that was trying to figure out just how flexible these models really were. The standard learning-theoretic principle is that the more flexible model classes learn slower than less flexible ones. (If you are willing and able to reproduce really complicated patterns, it's hard for you to distinguish between signal and noise in limited data. There are important qualifications to this idea, but it's a good start.) We thus began by thinking about trying to get the DSGEs to fit random binary noise, because that'd tell us about their Rademacher complexity, but that seemed unlikely to go well. That led to thinking about trying to get the models to fit the original time series, but with the series randomly scrambled, a sort of permutation test of just how flexible the models were.
At some point, one of us had the idea of leaving the internal order of each time series alone, but swapping the labels on the series. If you have a merely-statistical multivariate model, like a vector autoregression, the different variables are so to speak exchangeable --- if you swap series 1 and series 2, you'll get a different coefficient matrix out, but it'll be a permutation of the original. (The parameters will be "covariant" with the permutations.) It'll fit as well as the original order of the variables. But if you have a properly scientific, structural model, each variable will have its own meaning and its own role in the model, and swapping variables around should lead to nonsense, and grossly degraded fits. (Good luck telling the Lotka-Volterra model that hares are predators and lynxes are prey.) There might be a few weird symmetries of some models which leave the fit alone (*), but for the most part, randomly swapping variables around should lead to drastically worse fits, if your models really are structural.
Daniel did some initial trials with the classic "real business cycle" DSGE of Kydland and Prescott (1982), and found, rather astonishingly, that the model fit the swapped data better a large fraction of the time. Exactly how often, and how much better, depended on the details of measuring the fit, but the general result was clear.
The reason we'd gotten in to all this was wanting to apply statistical learning theory to macroeconomic forecasting, to put bounds on how bad the forecasts would be. Inverting those bounds would tell us how much data would be needed to achieve a given level of accuracy. Our results were pretty pessimistic, suggesting that thousands of years of stationary data might be needed. But those bounds were "distribution-free", using just the capacity or flexibility of the model class, and the rate at which new points in the time series become independent of its past. This could be pessimistic about how well this very particular model class can learn to predict this very particular data source.
We therefore turned to another exercise: estimate the model on real data (or take published estimates); simulate increasingly long series from the model; and re-estimate the model on the simulation. That is, bend over backwards to be fair to the model: if it's entirely right about the data-generating process, how well can it predict? how well can it learn the parameters? how much data would it need for accurate prediction? With, again, the Kydland-Prescott model, the answer was... hundreds if not thousands of years worth of data.
Of course, even in the far-off days of 2012, the Kydland-Prescott model was obsolete, so we knew that if we wanted anyone to take this seriously, we'd need to use a more up-to-date model. Also, since this was all numerical, we didn't know if this was a general problem with DSGEs, or just (more) evidence that Prescott and data analysis were a bad combination. So we knew we should look at a more recent, and more widely-endorsed, DSGE model...
Daniel graduated; the workhorse Smets and Wouters (2007) DSGE is a more complicated creature, and needed both a lot of programming time and a lot of computing time to churn through thousands of variable swaps and tens of thousands of fits to simulations. We both got busy with other things. Grants came and (regrettably) went. But what we can tell you now, with great assurance, is that:
Series swapping is something we dreamed up, so I'm not surprised we couldn't find anyone doing it. But "let's try out the estimator on simulation output" is, or ought to be, an utterly standard diagnostic, and it too seems to be lacking, despite the immense controversial literature about DSGEs. (Of course, it is an immense literature --- if we've missed precedents for either, please let me know.) We have some thoughts about what might be leading to both forms of bad behavior, which I'll let you read about in the paper, but the main thing to take away, I think, is the fact that this widely-used DSGE works so badly, and the methods. Those methods are, to repeat, "simulate the model to see how well it could be estimated / how well it would predict if it was totally right about how the economy works" and "see whether the model fits better when you swap variables around so you're feeding it nonsense". If you want to say those are too simple to rise to the dignity of "methods", I won't fight you, but I will insist all the more on their importance.
It might be that we just so happened to have tried the only two
DSGEs with these pathologies. (It'd be a weird coincidence, but it's
possible.) We also don't look at any non-DSGE models, which might be as bad on
these scores or even worse. (Maybe time series macroeconometrics is inherently
doomed.) But anyone who is curious about how whether their favorite
macroeconomic model meets these very basic criteria can check, ideally
before they publish and rack up thousands of citations lead
the community of inquirers down false trails. Doing so is conceptually simple,
if perhaps labor-intensive and painstaking, but that's science.
After posting the preprint, people helpfully found some bugs in our code. These glitch up all our numerical results. Since this is primarily a paper about our numerical results, this is obviously bad. The preprint needs to be revised after we've fixed our code and re-run everything. I am pretty confident, however, about the general shape of the numbers, because as I said we got the same kind of behavior from the Kydland-Prescott model and (importantly, in this context) off-the-shelf code. Of course, you being less confident in my confidence after this would be entirely sensible. In any event, I'll update this again when we're done with re-running the code and have updated the preprint.
*: E.g., in Hamiltonian mechanics, with generalized positions \( q_1, \ldots q_k \) and corresponding momenta \( p_1, \ldots p_k \) going into the Hamiltonian \( H \), we have \( \frac{dq_i}{dt} = \frac{\partial H}{\partial p_i} \) and \( \frac{dp_i}{dt} = -\frac{\partial H}{\partial q_i} \). A little work shows then that we can exchange the roles of \( q_i \) and \( -p_i \) with the same Hamiltonian. But you can't (in general) swap position variables for each other, or momenta for each other, or \( q_1 \) for \( -p_2 \), or even \( q_i \) for \( p_i \), etc.
Posted at November 02, 2022 14:51 | permanent link
Attention conservation notice: I have no taste, and no qualifications to opine on public administration, political philosophy, social epistemology, or the aims and methods of sociology. Also, most of my reading this month was done at odd hours and/or while bottle-feeding a baby, so I'm less reliable and more cranky than usual.
Books to Read While the Algae Grow in Your Fur; The Progressive Forces; Teaching: Statistics of Inequality and Discrimination; Scientifiction and Fantastica; Commit a Social Science; The Collective Use and Evolution of Concepts; Networks; Philosophy; Cthulhiana; Actually, "Dr. Internet" Is the Name of the Monsters' Creator
Posted at October 31, 2022 23:59 | permanent link
Attention conservation notice: I have no taste, and no qualifications to opine on political sociology or the history of the Second International. Also, most of my reading this month was done at odd hours and/or while bottle-feeding a baby, so I'm less reliable and more cranky than usual.
By a universally applicable social law, every organ of the collectivity, brought into existence through the need for the division of labor, creates for itself, as soon as it becomes consolidated, interests peculiar to itself. The existence of these special interests involves a necessary conflict with the interests of the collectivity. Nay, more, social strata fulfiling peculiar functions tend to become isolated, to produce organs fitted for the defense of their own peculiar interests. In the long run they tend to undergo transformation into distinct classes. [Part 6, ch. 2]This is the source for the famous "iron law of oligarchy". Paraphrasing: Effective and efficient social groups must employ a division of labor, and must create specialists. Those specialists genuinely know more about how to make the group work than most of its members, and to be effective they need to stay in their roles for extended periods of time, and to be replaced by other specialists. They inevitably become leaders, with different interests than others. Michels: "Who says organization, says oligarchy" [pt. 6, ch. 4].
Books to Read While the Algae Grow in Your Fur; The Progressive Forces; Physics; Mathematics; Scientifiction and Fantastica; Commit a Social Science
Posted at September 30, 2022 23:59 | permanent link
Attention conservation notice: I have no qualifications to opine on early 20th century Russian and Mongolian history, or even on crackpots.
Constant readers (if I have any left) will notice that this was not a lot of books. This is because I am now engaged in a very time- and attention- consuming project which will occupy me for the foreseeable future. My collaborator in this endeavor requests that I not blog about it, but I am allowed to describe it by linking to an emblematic image. I like to imagine that the satyr is playing the pipes because he and the nymph have learned that it is, paradoxically, actually the only way to get their baby to sleep.
Books to Read While the Algae Grow in Your Fur; Scientifiction and Fantastica; Pleasures of Detection, Portraits of Crime; Afghanistan and Central Asia; Psychoceramica; Writing for Antiquity; The Running-Dogs of Reaction
Posted at August 31, 2022 23:59 | permanent link
Attention conservation notice:: I have no taste, and no qualifications to opine on the Italian Renaissance, political philosophy, intellectual history, or even game theory.
Books to Read While the Algae Grow in Your Fur; Writing for Antiquity; Philosophy; Enigmas of Chance; The Dismal Science; Physics
Posted at July 31, 2022 23:59 | permanent link
Attention conservation notice: I have no taste, and no qualifications to opine on the (linked) decay of our infrastructure and our institutions, or to evaluate books on pregnancy (but then neither does that author).
Some years ago I was trying to decide whether or not to move to Harvard from Stanford. I had bored my friends silly with endless discussions. Finally, one of them said, "You're one of our leading decision theorists. Maybe you should make a list of costs and benefits and try to roughly calculate your expected utility." Without thinking, I blurted out, "Come on, Sandy, this is serious."That said, I did appreciate Oster's efforts at providing actual estimates of various probabilities, however imperfect. §
Books to Read While the Algae Grow in Your Fur; Natural Science of the Human Species; The Beloved Republic; The Continuing Crises; The Great Transformation; Scientifiction and Fantastica
Posted at June 30, 2022 23:59 | permanent link
Attention conservation notice: You have better things to do with an hour of your precious, finite life than staring at a screen while an academic tries to give a hand-wavy summary and advertisement for technical work on abstruse problem you don't care about.
I will be talking on Random-Feature Matching to the One World Approximate Bayesian Computation Seminar at 8:30 am Eastern time (=1:30 pm UK time) on Thursday, 23 June. If you are interested in simulation-based inference but have not (oddly) read my paper, or if you just want to marvel at how bad someone can be at giving a Zoom talk, two years on, please join. (Details on getting access to the Zoom session can be had by following that last link.)
Let me take this opportunity to thank the organizer both for the invitation, and for not insisting on the usual seminar time of 9:30 am UK time.
Posted at June 21, 2022 14:11 | permanent link
Attention conservation notice: Advertisement for a course you won't take, at a university you don't attend, in which very human and passionately contentious topics deliberately have all the life sucked from them, leaving only the husk of abstractions and the dry bones of methodology.
In the fall I will, again, be teaching my class on inequality
36-313, Statistics of Inequality and Discrimination
9 units
Time and place: Tuesdays and Thursdays, 1:25 -- 2:45 pm, in Wean Hall (WEH) 6403 (tentatively)
Description: Many social questions about inequality, injustice and unfairness are, in part, questions about evidence, data, and statistics. This class lays out the statistical methods which let us answer questions like Does this employer discriminate against members of that group?, Is this standardized test biased against that group?, Is this decision-making algorithm biased, and what does that even mean? and Did this policy which was supposed to reduce this inequality actually help? We will also look at inequality within groups, and at different ideas about how to explain inequalities between groups. The class will interweave discussion of concrete social issues with the relevant statistical concepts.
Prerequisites: 36-202 ("Methods for Statistics and Data Science") (and so also 36-200, "Reasoning with Data"), or similar with permission of the instructor
Last year was the first time I got to teach it, and it was a mixed experience. The students who stuck with it were, gratifyingly, uniformly very happy with it (and I am pretty sure they learned a lot!). But it also had the biggest "melt" of any class I've taught, with fully half of those who initially signed up for it eventually dropping it. The most consistent reason why --- at least, the one they felt comfortable telling me! --- was that they were expecting something with a lot more arguing about politics, and a lot less math and data analysis. I have taken this feedback to heart, and decided to do even more math and data analysis.
There will be no exams.
My usual policy is to drop a certain number of homeworks, and a certain number of lecture/reading questions, no questions asked. The number of automatic drops isn't something I'll commit to here and now (similarly, I won't make any promises here about the relative weight of homework vs. lecture-related questions).
Posted at June 21, 2022 13:45 | permanent link
Attention conservation notice: I have no taste, and no qualifications to opine on the archaeology of the Southwest, the pre-history of diversity training, or trends in American economic inequality.
Books to Read While the Algae Grow in Your Fur; Scientifiction and Inequality; The Dismal Science; Writing for Antiquity; Commit a Social Science; The Progressive Forces; Teaching: Statistics of Inequality and Discrimination; Pleasures of Detection, Portraits of Crime; Tales of Our Ancestors; Physics; The Great Transformation; Biology
Posted at May 31, 2022 23:59 | permanent link
Attention conservation notice: Rationalizing my gut-level dislike of a social medium as Objectively Correct. First drafted in mid-2017, left to rest in my drafts folder because, while sincere, it feels a bit mean. Posted now because I found myself re-writing the next-to-last paragraph.
If, as Leibniz has prophesied, libraries one day become cities, there will still be dark and dismal streets and alleyways as there are now. --- LichtenbergI mentioned, some years ago, that in response to reader requests I have a Twitter account. I use this only for announcing new posts here. Messages sent to it will go unread; attempts to communicate through it will be fruitless.
I have, nonetheless, put some time over the years into observing Twitter; I wish I had it back again. There are, so far I can see, only four good uses for Twitter:
For everything else, well, if someone had deliberately tried to combine the worst features of comments sections and Usenet, they could hardly have done better --- except by first imposing silly length restrictions, followed by kludged-on threads that make Usenet seem a model of clear organization, plus of course an interface that channels people towards the outrage (or main character) of the moment.
I don't know whether it makes people unhappy and angry, or whether only unhappy, angry people persist in using it, but I am not joking when I say that we would all be better off if it disappeared immediately.
--- One of my long-held semi-crank notions is this: all online communication, being through writing, reproduces the social dynamics of literary communities, especially print-literary communities. This law holds independent of the educational level or even intellectual seriousness of the participants. Thus flame-wars, sock-puppets, selective quotation, trawling through the archive for discreditable episodes, "the lurkers support me in e-mail", creating isolated fora to incubate increasingly weird ideas, recycling from supposedly-authoritative source texts long after they're debunked (if they were ever bunked in the first place), spastic attention cascades in which "all fandom was plunged into war", etc., escape from the pages of the little magazines (such as the Philosophical Transactions of the Royal Society), to become part of everyone's life. Twitter has raised this to a new level of awfulness, by making it very hard to actually contribute anything of value, or, having done so, for others to find it and build on it, while still preserving the affordances for weirdness, meanness, and spasm-proneness.
That is my opinion; and it is further my opinion that you people should get off my lawn.
Update, 28 May 2022, further to the theme, in no particular order:
*: Some comments on Frost's review, without having read the book being reviewed. (1) I am, unsurprisingly, extremely sympathetic to the position that hashtag activism is basically futile. (If the authors really neglect Tufekci's empirical and theoretical work as much as Frost says they do, it's pretty damning.) (2) Not examining right-wing hashtag activism seems like an obvious analytical flaw. (Even if your primary interest is in left-wing movements, the comparisons are essential.) (3) It's true that Twitter isn't accountable to its users, or to the people-as-represented-by-government, but Frost for her part never makes clear which of the flaws she identifies would be remedied by such accountability. (4) Doing something about the opioid epidemic by tinkering with drug policy seems a hell of a lot more practical to me that doing something about it by overthrowing American capitalism, or even reversing the trends in inequality over the last half-century. (I would like to see those trends reversed.) ^
Actually, "Dr. Internet" Is the Name of the Monsters' Creator; Linkage; Modest Proposals; The Collective Use and Evolution of Concepts
Posted at May 28, 2022 12:56 | permanent link
Attention conservation notice: I have no taste, and no qualifications to opine on U.S. politics, or the lives and works of 20th century Marxist intellectuals.
Books to Read While the Algae Grow in Your Fur; Scientifiction and Fantastica; Pleasures of Detection, Portraits of Crime; The Progressive Forces; Writing for Antiquity; Commit a Social Science; The Beloved Republic
Posted at April 30, 2022 23:59 | permanent link
Attention conservation notice: Links to forbiddingly-technical scientific papers and lecture notes, about obscure corners of academia you don't care about, and whose only connecting logic is having come to the attention of someone with all the discernment and taste of a magpie (who's been taught elementary probability theory).Or whatever the heck it is I study these days. (I did promise that this series would be intermittent.) In no particular order.
I incorporate computational constraints into decision theory in order to capture how cognitive limitations affect behavior. I impose an axiom of computational tractability that only rules out behaviors that are thought to be fundamentally hard. I use this framework to better understand common behavioral heuristics: if choices are tractable and consistent with the expected utility axioms, then they are observationally equivalent to forms of choice bracketing. Then I show that a computationally-constrained decisionmaker can be objectively better off if she is willing to use heuristics that would not appear rational to an outside observer.
Complexity;
Enigmas of Chance;
Networks;
Physics;
The Dismal Science;
Constant Conjunction Necessary Connexion;
Automata and Calculating Machines
Posted at April 25, 2022 10:41 | permanent link
Attention conservation notice: A link-dump piece, where some of the links were first opened in 2015.
Tabs I have closed recently, which are of a positive and/or constructive and/or cheerful nature:
Linkage; Psychoceramica; Physics; Natural Science of the Human Species; Mathematics; Automata and Calculating Machines; The Commonwealth of Letters; Biology
Posted at April 25, 2022 10:40 | permanent link
Attention conservation notice: I have no taste, and no credentials to opine on the sociology of education, political and moral philosophy, medieval Islamic science, or even, strictly speaking, pure mathematics.
I don't think that the people questioning the evidence are bad people, but they are reluctant to let go of the dominant narrative about schools. It would be one thing if the reason was because they had issues with whether the ECLS-K item-response theory scales of reading can be considered truly interval, or if they questioned whether nonschool investments in children are constant across seasons, or if they thought that the approach scholars use to model the overlap days between test dates and the beginnings and ends of school years was insufficient. ... But while many have resisted the empirical patterns in chapters 1--4 and remain committed to The Assumption, the quality of evidence doesn't seem to be the obstacle. [p. 97]
Books to Read While the Algae Grow in Your Fur; Pleasures of Detection, Portraits of Crime; Philosophy; Commit a Social Science; Scientifiction and Fantastica; Islam and Islamic Civilization; Afghanistan and Central Asia; Writing for Antiquity; Mathematics; Teaching: Statistics of Inequality and Discrimination
Posted at March 31, 2022 23:59 | permanent link
Attention conservation notice: I have no taste, and no qualifications to opine on the history of Central Asia, the philosophy of science, the anthropology of New Guinea and/or cultural creativity, archaeology, Antarctic exploration, or the philosophy of Spinoza.
Books to Read While the Algae Grow in Your Fur; Writing for Antiquity; Philosophy Enigmas of Chance; Afghanistan and Central Asia; The Great Transformation; Minds, Brains, and Neurons; Commit a Social Science; Psychoceramics
Posted at February 28, 2022 23:59 | permanent link
Attention conservation notice: I have no taste, and no qualifications to opine on the history and geopolitical context of Antarctic exploration, the social structure of medieval China, or philosophy of any kind.
Books to Read While the Algae Grow in Your Fur; Writing for Antiquity; Networks; Scientifiction and Fantastica; Philosophy
Posted at January 31, 2022 23:59 | permanent link