How you measure something determines what you find — and often determines what you can find. The gender wage gap looks different depending on whether you control for occupation. Segregation looks different depending on whether you measure it relatively or absolutely. Ideology looks different depending on whether you use roll calls or surveys. Posts here take the act of measurement seriously as a theoretical and political problem, not just a technical one.
In a 2015 symposium on big data and measurement, Maggie Penn and I argued that social choice theory is central to the analysis of complex data precisely because any reduction of high-dimensional data into a usable measure involves aggregation, and aggregation involves choices about what to preserve and what to discard.1 We used the Florentine … Read more
Scientific American ran a piece yesterday on Simpson’s paradox — the phenomenon in which a trend that holds within every subgroup of a dataset reverses, or vanishes entirely, when those subgroups are combined. Regular readers of this blog will have encountered it before. In 2012, it showed up inside No Child Left Behind: a school … Read more
Yesterday’s post argued that a national aggregate can accurately represent no one’s actual experience when the underlying data are structured the right way — or, more precisely, the wrong way. The incarceration example was instructive precisely because it was symmetrically uncomfortable: the same Simpson’s paradox that embarrasses the reform-is-working narrative also embarrasses the cities-are-dangerous one. … Read more
A friend and colleague gave a talk today about incarceration trends in the United States (she knows who she is, and I owe her a coffee). The importance of the problem — and the genuine messiness of the data — got me thinking about how difficult it is to convey the right lessons from debates … Read more
The Republican Party is in crisis. This year’s presidential campaign is arguably evidence enough for this conclusion, but it is important to remember that there are really (at least) two “Republican Parties”: one composed of voters and another composed of Members of Congress. A split in the broader GOP is troublesome for Republican elites because, … Read more
This is a guest post by David Epstein. Picture this: you are on a committee to hire a new CEO for a large, multinational firm. There are a number of qualified candidates, you are told, each of whom has many years of experience in the relevant field, and then you are handed a background folder … Read more
The Democratic and Republican Parties each use a weighted voting system to choose their presidential nominees. This only matters when no candidate has a majority of the delegates, and the details are complicated because the weight a particular candidate has is actually a number of (possibly independent) delegates. Leaving those details to the side, let’s consider how much … Read more
This is math, but it isn’t politics. This is serious business. This is the NFL. The New England Patriots won the coin toss to begin today’s AFC championship game against the Denver Broncos. With that, the Patriots have won 28 out of their last 38 coin tosses. To flip a fair coin 38 times and have … Read more
FOX News just announced the 10 candidates who will participate in the first primetime Republican presidential primary debate on August 6, 2015. The top 10 were decided by these procedures. Given that FOX is arguably playing a huge role in the free-for-all-for-the-GOP’s-Soul that is that race for the 2016 GOP presidential nomination, it is important to consider whether, and to what … Read more
You know where you are? You’re down in the jungle baby, you’re gonna die… In the jungle…welcome to the jungle…. Watch it bring you to your knees, knees… – Guns N’ Roses, “Welcome to the Jungle” It’s a jungle out there, and … Read more