Three-Toed Sloth

Lyric Poetry in Pluto's Republic

Attention conservation notice: 3200 words on a silly academic paper about popular music and narcissism. Contains complaints about bad data analysis, firm statements about writing poetry from someone who can't, and largely unsupported gloomy reflections about the condition of the house of intellect.

Let me begin with a quotation from one of my favorite books:

A good many years ago a neighbour whose sex chivalry forbids me to disclose exclaimed upon learning of my interest in philosophy: `Don't you just adore Pluto's Republic?'

Pluto's Republic has remained in my mind ever since as a superlatively apt description of [the] intellectual underworld.... We each populate Pluto's Republic according to our own prejudices: for me its most prominent citizens are IQ psychologists.... Other prominent citizens include all practitioners of `scientism', especially those who apply what they mistakenly believe to be the methods of science to the investigation of matters upon which science has no bearing whatsoever...

— Peter Medawar, Pluto's Republic, p. 1

I have no taste, and so a large part of my reading consists of what is frankly mind candy, and a large part of the candy consists of mystery series in which murders are solved by amateur sleuths. It is part of the norms of this genre or tradition that the heroine (and it is almost always a heroine) is a not-too-old woman who pursues a more or less genteel occupation in a small town or not-too-large city, which has an (unremarked-on) rate of violent death comparable to post-invasion Iraq. It is equally a norm of the genre that the novels are narrated in the first person (singular). Two of my other addictions are secondary-world fantasies and space opera science fiction, which use first-person narration far more sparingly*. While one might come up with functional-rhetorical rationales for this contrast (perhaps based on some of the experiments in Bortolussi and Dixon), the proximate explanation is simply genre norms. To argue on this basis that writers, or readers, of amateur-sleuth mysteries are less narcissistic than writers or readers of space opera and lap-breaker fantasies would be stupid.

More specifically, it would be to ignore the fact that, while mind-candy genre novels are perhaps very humble works of art, they are works of art, and, as the poet says, all art is artifice. They are things which are made to achieve certain ends (which may be vague), employing skills and traditions and what one might call internal norms. Even when writers pour their hearts out on to the page, to treat works of art as direct, unmediated expressions of their makers' personalities trembles on the border between utter philistinism and not-safe-to-be-let-outdoors-without-grownup-supervision naivete. And this is true not just of popular novels, but also of poems which demand musical accompaniment and take about three minutes to recite, bringing us to today's reading.

C. Nathan DeWall, Richard S. Pond, Jr., W. Keith Campbell, and Jean M. Twenge, "Tuning in to Psychological Change: Linguistic Markers of Psychological Traits and Emotions Over Time in Popular U.S. Song Lyrics", Psychology of Aesthetics, Creativity, and the Arts online-first (2011)

Abstract: American culture is filled with cultural products. Yet few studies have investigated how changes in cultural products correspond to changes in psychological traits and emotions. The current research fills this gap by testing the hypothesis that one cultural product — word use in popular song lyrics — changes over time in harmony with cultural changes in individualistic traits. Linguistic analyses of the most popular songs from 1980--2007 demonstrated changes in word use that mirror psychological change. Over time, use of words related to self-focus and antisocial behavior increased, whereas words related to other-focus, social interactions, and positive emotion decreased. These findings offer novel evidence regarding the need to investigate how changes in the tangible artifacts of the sociocultural environment can provide a window into understanding cultural changes in psychological processes.

I want to add a few remarks to what Mark Liberman has already said ("Lyrical Narcissism?", "Vampirical Hypotheses", "Pop-culture narcissism again"), first about the methodological inadequacies, then about the statistics, and finally on the larger lessons.

The empirical basis for inferring narcissism from using first person singular pronouns appears to be Robert Raskin and Robert Shaw, "Narcissism and the Use of Personal Pronouns", Journal of Personality 56 (1988): 393--404. This shows that, over twenty years ago, there was a modest positive correlation (+0.26) between scores on a quiz intended to measure narcissism, and how often 48 UC Santa Cruz undergrads used first-person singular pronouns in extemporized five minute monologues. Top 100 songs are not spontaneous monologues by undergrads looking for a painless way to get \$5 and/or check off a Psych. 1 requirement, and DeWall et al. offer no evidence that this correlation generalizes to any other context. In particular they offer no reason to think that differences over time, as language and culture changes, should be explained in the same way as these differences across people, at a single time and in a single school.

Let me sketch an analogy. You can measure the height of a building from the length of its shadow, using trigonometry. If you gather a data set of many building heights and shadow lengths taken at nearly the same time of day on the same day of the year, there will in fact be an excellent correlation between the two, and a genuinely linear relationship. (Indeed, the only reason the correlation would be even slightly less than 1 would be measurement noise.) But the relationship between the height of buildings and the length of their shadows depends on where the sun is in the sky. At a different time of day or a different day of the year, you will get a different linear relationship. If you just plug in to your formula blindly, you will get bad estimates of the height. If you were a morning person, and precisely operationalized your initial data as "length of shadow to the west of the building", you would get negative estimated heights in the afternoon, when shadows point to the east. (Sure, it's counterintuitive that buildings are actually sunk below the ground, but are you going to argue with the numbers?) On cloudy days, whatever you measured in place of shadows would just be noise.

To draw the moral explicitly, even if there is such a thing as a one-dimensional personality trait of narcissism**, and even if that was correlated with pronoun use in one particular historical population, in one particular social/rhetorical context, that tells us nothing at all about the correlation in other situations. I don't assert that it can't be true, but there is no psychological or statistical reason to presume that it is true, and so it needs to be established. In more psychological terms, thinking otherwise is not so much slipping into the fundamental attribution error as wallowing in it.

Composing a popular song is not coming up with a five-minute off-the-cuff monologue. Lyrics are in fact composed. They are deliberately made to achieve certain effects on the audience, including meshing in certain ways with the music (which is also being composed), they are stylized, and their composition is guided by inherited traditions and formulas of the genre and by individual habits of writing. Those guides and constraints are at once cognitive — it is computationally necessary to cut down the search space (see e.g. Lord or Simon or Boden) — and aesthetic — they are norms (see e.g. Wellek and Warren). The persona of the song or poem is not the personality of the song-writer or poet. (David Byrne is not actually a psycho-killer.) This is true no matter how strong the emotions which motivate the song-writer are, or how lacking the writer may be in self-conscious artistry.

Commercially successful popular songs are artistic compositions which have been filtered through a rather byzantine industry of gate-keepers and intermediaries. The songs which survive this filtration must then be bought by many thousands of people, for their own reasons. The song might succeed by appealing to a single very popular taste; or simultaneously appealing to many different tastes; or, indeed, merely by already being popular.

If the question is whether musicians have become more narcissistic, we need to ask whether more narcissistic musicians compose songs which use first person singular pronouns more often, and, if so, whether this signal survives the filtering process of the music industry. If the question is whether audiences have become more narcissistic, we need to ask whether more narcissistic people prefer songs which use such pronouns more often, and, if so, whether this signal survives the filtering process of the music industry. (Anyone who thinks individual preferences simply translate into aggregate outcomes has simply not been paying attention, and for a very long time at that.) We are so far from the laboratory situation of Raskin and Shaw that it's not even funny.

Let me turn to more specific weaknesses in the logic of the paper.

1. Should the analysis adjust for genres at all? If consumption or production of songs reflects personality traits, and some genres are inherently more narcissistic, then switching to those genres is itself indicative of changing distributions of personality.
2. That said, p. 5 provided the following head-desk moment: "Because our multiple regression analysis controlled for genre, any changes in genre over time did not significantly account for our effects". I hope that any one of my undergraduate students in data analysis could, by this point in the semester, pick at least three holes in this without having to think too hard***.
3. Whose narcissism is being measured here? Is it the consumers of the songs, or their producers? This point is never addressed, and it is hard to see how it could be from this data.
4. Why does an increase in the use of first-person-singular pronouns in popular songs not indicate increasing empathy, and decreasing self-centeredness, on the part of the audience? (That is, they are better able and more willing to identify with the persona of the song's narrator.)
5. Why attribute these changes to a modification in the distribution of personality types among the general population, rather than changes in the demographic characteristics of people who buy music?
6. Why attribute changes to the general population, as opposed to changing personality types among producers? One might sub-divide this further into composers, performers, gate-keepers in music companies, etc.
7. Why attribute these to changes to changing personality types, as opposed to changing audience expectations about musicians, and suitable modes of expression for them? Maybe neither audiences nor musicians are a whit more narcissistic, but audiences expect their musicians to be.
8. Why attribute these to an actual change in personality types, as opposed to a change in the perception by producers of the tastes of the commercially-effective audience? (Such a perception might or might not be right.)
9. Salganik and Watts have shown, experimentally, that song popularity is incredibly stochastic and path-dependent. Musicians learn from each other, and the gate-keepers of the music industry try to copy what works, so there is a large amount of reinforcement or cascading added to an initially extremely noisy process. This is going to be very apt to produce cascades, and "hitchhiking", as well as changing distributions of songs when nothing at all about audience or musician personalities is changing. (Genre norms create an extra mechanism for hitchhiking.)
10. The relationship between personality traits and language use may just have changed, either in the general culture or in the highly specific and formalized context of song lyrics. As Pennebaker will explain to you, there are many ways in which first-person singular pronouns can be used, carrying many different implications. (Pennebaker is actually cited in this paper, but not at all intelligently.)

For all I know, it's possible to rule out all of these alternative explanations, but they would have to first be recognized and then investigated. (Distinguishing changes in audiences' demand for narcissism from changes in musicians' ability to supply it will be especially tricky.) DeWall et al., however, are oblivious to any possibility other than lyrics reflecting personality, while being vague about whose personality is on display.

So, to sum up, we have basically no reason to think that changes in the use of first-person singular pronouns measure changes in narcissism (certainly not over time or in this context), and a slew of alternative explanations for any changes which might be found, other than "Americans are becoming narcissistic and this is reflected in their popular songs". One might, perhaps, write these off as the excessive scruples which come from over-indulging in skeptical philosophy. Let's have the courage to assume away all these inconvenient possibilities, and look at what the data show.

The centerpiece Figure 1 from DeWall et al. goes like so:

Many journalists seem to have found this very convincing. Fortunately, however, DeWall et al. also provide a table with the mean and standard deviation of the first person pronoun use for each year, and a 95% confidence interval. (They don't say how they calculated the latter, but I'll take them at their word and presume they did that properly.) This lets me plot the actual data, which looks like this:

(My code, in R.) The black dots, joined by lines to guide the eye, are the actual percentages. The dashed lines are the 95% confidence limits. The horizontal grey line is the over-all mean percentage, over the whole data set. The two colored lines are two smoothing spline fits, one with (purple) and one without (blue) giving extra weight to years with smaller standard deviations. Making the smoothing splines requires a little knowledge of statistics; everything else just needs the ability to draw the numbers DeWall et al. provide.

The flat horizontal line is inside the confidence limits in 27 of the 28 years. This is exactly what we would expect if there was no signal here whatsoever, and all fluctuations from year to year were just noise****. (95% coverage per year and 28 years yields 1.4 expected non-coverage events.) There is nothing here to explain; the appearance that there is something in their Figure 1 is one part bad data analysis to one part How to Lie with Statistics-level bad graphing*****.

While perhaps not a truly epic fail, this is not a creditable performance. The paper probes a hugely complex tangle of issues relating individual minds, communication, social norms, artistic expression, social change and cultural transformation. There is no shame in not unraveling the whole snarl at once, but between the incompetent data analysis, the failure of logical imagination, and the deep misunderstanding of how works of art are made and used, it does nothing to advance our knowledge of anything. I am not sure which is more needed here, remedial reading in Richard Berk and Denny Borsboom, or in Wellek and Warren and Erving Goffman, but they need something. Psychology of Aesthetics, Creativity, and the Arts does not seem to be a very highly ranked journal within psychology, but the authors of this paper have plenty of papers elsewhere, so this says something about the intellectual standards of the discipline.

Looking at the reception of the paper (see Liberman, again, for linkage), one finds dreary moralizing about how kids these days are selfish brutes and nobody makes decent music any more, given an unearned air of authority by the pretense to science. It should not, by this point, come as a surprise that many science journalists and pundits lack the numeracy, imagination and skepticism to avoid being taken in by such foolishness.

Public trust in scientists — that we generally know what we are talking about — is an extremely valuable resource for the scientific community. It is, I think, ultimately why people are willing to devote such vast resources to the scientific enterprise, to letting us gratify our curiosity. This trust has been painfully built up over many long years and generations and even centuries, by, among other things, taking great pains to be trustworthy. This trust is even a valuable resource for the public, when it is not misplaced. The more I see of this kind of thing, the more I wonder how well-founded that trust really is. This specific myth — that it has been scientifically proven that pop songs reflect increasing American narcissism — will persist as a minor vampirical hypothesis, occasionally draining the blood from graduate students in psychology. This kind of pointless myth-making and perversion of science will continue as long as the implicit goal of our institutions for cultivating knowledge is in fact to realize Pluto's Republic.

Update, later that day: Mark Liberman points out, by e-mail, that the famous 23rd psalm ("The Lord is my shepherd; I shall not want") clocks in at 14.3% first person pronouns in the King James Version, above the DeWall et al. confidence limits for all but seven years. I would add that "Rock of Ages" is a lower but still well-above-average 13.3%. On the other hand, "Rock of Ages" by Def Leppard (a top 100 song in 1983, and so part of the data) is between 4.6% and 6.2% first person singular pronouns (depending on how you want to count "gimme"). Clearly, the only thing saving American popular culture from epidemic narcissism in the early 1980s was preferring heavy metal to hymns.

Update, next day: John Emerson points out that "Like a Rolling Stone" contains plenty of instances of second person singular pronouns, and no first person singular pronouns, fully consistent with Bob Dylan's famed selflessness.

Manual trackback: Language Log (so I should probably really tag this post "incestuous amplification"); English, Jack; Permutations (with links to earlier posts)

*: Cue fannish nit-picking.

**: My skepticism about the "constructs" of correlational psychology is not limited to IQ/g, but that's another story for another time. For the present, I am willing to stipulate that narcissism exists and can be measured by the psychometric instruments which purport to do so.

***: Hole one: their four genre categories are incredibly crude, and have huge amounts of internal diversity. (Likewise, the genre norms of amateur-sleuth mysteries are rather different from private-eye detective stories, police procedurals, and serial-killer thrillers. Calling them all "mysteries", with one dummy variable, would not answer.) Hole two: "controlling for" variables this way only gets you an all-else-being-equal prediction if the regression model is actually well specified, which they hadn't the wit to check. Hole three: the counterfactual issue. (This is the only evenly slightly tricky one.) We have a certain distribution of the regressor variables in the training data, and so certain correlations among them. These correlations mean that each regressor can, to some extent, be linearly predicted from the others. The regression coefficients are basically the correlation between the response and the distinct, linearly-unpredictable part of each regressor. This means that when you change the distribution of regressors, the regression coefficients will, in general, change too. The regression coefficients can only be used to answer counterfactual questions ("what would the proportion of first-person pronouns be, if genre composition had stayed constant?") under very special assumptions, which we have no reason to think hold here. (See the notes for lectures 2, 22 and 23 for more.)

****: More exactly, this is what we would expect if the causes producing year-to-year shifts were so many, so various, and had such hard-to-describe inter-relations with each other that they cannot be effectively compressed or predicted from the past of the time series, and must simply be described in all their unique historical detail. As I tell my undergrads, "any signal distinguishable from noise is insufficiently complicated". If you want the full technical version of this idea, read Li and Vitanyi.

*****: Abbreviated scale on the vertical axis, visually exaggerating the change; inappropriate use of a linear model, which is guaranteed to give the impression of a steady and relentlessly one-directional march; no indication of uncertainty. I also can't figure out why they binned the values on the horizontal axis.

Learned Folly; Minds, Brains, and Neurons; Enigmas of Chance; The Commonwealth of Letters; Commit a Social Science