Measurement, Especially in the Social and Behavioral Sciences

08 May 2024 22:07

Yet another inadequate placeholder.

Factor Models (often advocated as a crucial tool here, I think too often without an appreciation of the intrinsic limits)
Mixture Models (ditto)
Optimal Linear Prediction and Estimation
Scientific Method and Philosophy of Science
Social Science Methodology
Statistics
The Thomson Ability-Sampling Model

Howard Becker, Evidence
Denny Borsboom, Measuring the Mind: Conceptual Issues in Contemporary Psychometrics
Joel Michell, Measurement in Psychology: A Critical History of a Methodological Concept
Oskar Morgenstern, On the Accuracy of Economic Observations

Denny Borsboom, "The Attack of the Psychometricians", Psychometrika 71 (2006): 425--440 [PDF reprint]
Denny Borsboom and Gideon J. Mellenbergh, "Test validity in cognitive assessment", pp. 85--115 in Jacqueline Leighton and Mark Gierl (eds.), Cognitive diagnostic assessment for education: Theory and applications (Cambridge University Press, 2007) [The points made here apply much more broadly than just to cognitive assessments, as the authors make clear]
Denny Borsboom, Gideon J. Mellenbergh and Jaap van Heerden
- "The Theoretical Status of Latent Variables", Psychological Review 110 (2003): 203--219 [PDF reprint]
- "The Concept of Validity", Psychological Review 111 (2004); 1061--1071 [PDF reprint]
Lee J. Cronbach and Paul E. Meehl, "Construct Validity in Psychological Tests", Psychological Bulletin 52 (1955): 281--302 [Though I think there is a lot here that's questionable... PDF reprint.]
Peter Fishburn and Bernard Monjardet, "Norbert Wiener on the theory of measurement, 1914, 1915, 1921", Journal of Mathematical Psychology 36 (1992): 165--184
R. D. Luce and J. W. Tukey, "Simultaneous conjoint measurement: a new type of fundamental measurement", Journal of Mathematical Psychology 1 (1964): 1--27
Andrew Maul, "Rethinking Traditional Methods of Survey Validation", Measurement: Interdisciplinary Research and Perspectives 15 (2017): 51--69
Norbert Wiener, "A new theory of measurement: A study in the logic of mathematics", Proceedings of the London Mathematical Society 19 (1919): 181--205 [Wiener's notation and terminology here is very archaic, so I strongly recommend Fishburn and Monjardet's expository paper first, or instead]

Jessica Flake and Eiko Fried, "Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them", psyarxiv/hs7wm (2019)
Jake Westfall and Tal Yarkoni, "Statistically controlling for confounding constructs is harder than you think", PLoS ONE 11 (2016): e0152719

Riley K. Carney and Ryan D. Enos, "Conservatism and Fairness in Contemporary Politics: Unpacking the Psychological Underpinnings of Modern Racism" [PDF preprint via Prof. Enos. This is a remarkable paper; if their conclusions generalize to further samples, it makes it very hard to see how "modern racism" scales are actually measuring racism in contemporary (c. 2020) American populations, whatever they may have done when introduced in the 1970s]
Brandon del Pozo, Peter Moskos, John Donohue, John Hall, "Registering a Proposed Business Reduces Police Stops of Innocent People? Reconsidering the Effects of Sex Work on Sex Crimes Found in Ciacci & Sviatschi’s Study of New York City", SSRN/4528229 (2023). This is a good case study which I will never teach, for obvious reasons. Cf.]
Klaus Fiedler, Claude Messner and Matthias Bluemke, "Unresolved problems with the ``I'', the ``A'', and the ``T'': A logical and psychometric critique of the Implicit Association Test (IAT)", European Review of Social Psychology 17 (2006): 74--147 [Ungated copy via Dr. Messner]
Seth J. Hill and Margaret E. Roberts, "Acquiescence Bias Inflates Estimates of Conspiratorial Beliefs and Political Misperceptions" [PDF preprint via Prof. Roberts]
Dean Knox, Will Lowe and Jonathan Mummolo, "Administrative Records Mask Racially Biased Policing", American Political Science Review 114 (2020): 6199-637
Marian F. MacDorman, Eugene Declercq, Howard Cabral, and Christine Morton, "Is the United States Maternal Mortality Rate Increasing? Disentangling trends from measurement issues Short title: U.S. Maternal Mortality Trends", Obstetrics and Gynecology 128 (2016): 447--455 [Comments]
Amy Orben and Andrew K. Przybylski, "The association between adolescent well-being and digital technology use", Nature Human Behaviour 3 (2019): 173--182 [Recommended here because it gives an excellent account of how different choices of how to measure vague, verbal ideas like "digital technology use" and "well-being" lead to different quantitative results]

Zach Weinersmith, "How Social Science Metrics Work", Saturday Morning Breakfast Cereal 2 July 2023

Can't really recommend:

Richard A. Zeller and Edward G. Carmines, Measurement in the Social Sciences: The Link between Theory and Data

René Baston, "From old-fashioned to offensive racism: How social norms determine the measurement object of prejudice questionnaires", Philosophical Psychology 36 (2022): 247--269
Nicholas Binney, "Meno's paradox and medicine", Synthese 196 (2019): 4253--4278 [From a quick scan, this doesn't really seem to consider the possibility of a virtuous circle here, i.e., iterative refinement]
Hubert M. Blalock, Conceptualization and Measurement in the Social Sciences
Marcel Boumans, Science Outside the Laboratory: Measurement in Field Science and Economics
Hasok Chang, Inventing Temperature: Measurement and Scientific Progress
Jan De Houwer, Sarah Teige-Mocigemba, Adriaan Spruyt, and Agnes Moors, "Implicit measures: A normative analysis and review", Psychological Bulletin 135 (2009): 347--368
Roberto Franzosi, From Words to Numbers: Narrative, Data, and Social Science [I think the first half of this book is brilliant. The second half also looks terrific, but it's written in an extremely different style that strikes me as very self-indulgent, and has put me off reading it. (I'm one to talk, I realize.) So I should buckle down and make my way through it one of these years.]
Gary Goertz, Social Science Concepts and Measurement
Ian Hussey and Sean Hughes, "Hidden Invalidity Among 15 Commonly Used Measures in Social and Personality Psychology", Advances in Methods and Practices in Psychological Science 3 (2020): 166--184
Ruth H. Keogh, Jonathan W. Bartlett, "Measurement error as a missing data problem", arxiv:1910.06443
John I. Kitsue and Aaron V. Cicourel, "A Note on the Uses of Official Statistics", Social Problems 11 (1963): 131--139
Dean Knox, Christopher Lucas, and Wendy K. Tam Cho, "Testing Causal Theories with Learned Proxies", Annual Review of Political Science 25 (2022): forthcoming
Krantz, Luce, Suppes and Tversky, Foundations of Measurement
Andrew T. Little and Anne Meng, "Measuring Democratic Backsliding", osf/n32zk (2023)
Emanuele Lugli, The Making of Measure and the Promise of Sameness
Rory McVeigh, Michael R. Welch, Thoroddur Bjarnason, "Hate Crime Reporting as a Successful Social Movement Outcome", American Sociological Review 68 (2003): 843--867
John W. Mohr, Christopher A. Bail, Margaret Frye, Jennifer C. Lena, Omar Lizardo, Terence E. McDonnell, Ann Mische, Iddo Tavory and Frederick F. Wherry, Measuring Culture
Jerry Z. Muller, The Tyranny of Metrics
Martin Niss, "Brownian Motion as a Limit to Physical Measuring Processes: A Chapter in the History of Noise from the Physicists' Point of View", Perspectives on Science 24 (2016): 29--44
Clayton Peterson, "Accommodation, prediction and replication: model selection in scale construction", Synthese 196 (2019): 4329--4350
Dirk Philipsen, The Little Big Number: How GDP Came to Rule the World and What to Do about It
Theodore M. Porter, Trust in Numbers: The Pursuit of Objectivity in Science and Public Life
Ryan Saylor, "Concepts, Measures, and Measuring Well: An Alternative Outlook", Sociological Methods & Research 42 (2013): 354--391
Melissa Gordon Wolf, Elliott Ihm, Andrew Maul, and Ann Taves, "The Response Process Evaluation Method", psyarxiv/rbd2x