Thinking Causally in Social Science

This post adapts the content and structure of Carolina Torreblanca’s tutorial session on causal inference for DDDI’s Summer Hangouts 2026. Carolina, a Penn AI Fellow in the Department of Political Science at Penn, walks through the subject from first principles. Starting from a misleading news headline, she builds up the Potential Outcomes Framework step by step and explains why isolating cause from correlation is genuinely hard, and how modern social science has developed rigorous tools to get there. No statistics background required.

Motivation

A Causal Headline

Carolina opened the session with a headline from the Medical Journal of Australia: “High social media use increases mental health risk in adolescents.” The paper was substantial — it followed 1,200 teenagers over a decade and found that teens using social media more than two hours a day had a higher risk of depression than lighter users.

Sounds conclusive. But notice the structure of the claim: heavy users are more likely to develop depression than lighter users. More likely compared to what? That question — what is the right comparison? — is the core of causal inference. The entire tutorial is an attempt to make it precise.

Causality in Social Science

Different disciplines conceptualize causality differently. In the hard sciences, causality is often framed in terms of necessary conditions (oxygen is necessary for combustion — remove it and fire stops) or sufficient conditions (exceeding 100°C is sufficient for water to boil — do it and boiling follows). Social media passes neither test: many heavy users never become depressed (not sufficient), and many depressed people never used social media heavily (not necessary).

In the social sciences, causality is a probabilistic statement: treatment shifts the probability of an outcome. Social media use makes depression more likely. This is an inherently comparative claim — more likely than the counterfactual world in which the same person didn’t use social media heavily. Nailing down that counterfactual is the challenge.

Potential Outcomes Framework

Potential Outcomes

The rigorous answer to “compared to what?” comes from the Potential Outcomes Framework, rooted in Neyman’s 1920s dissertation on fertilizer use and developed further by Rubin in the 1970s.

The idea is simple: for any unit — a plot of land, a person, a country — imagine two potential outcomes existing simultaneously in parallel worlds:

• $Y_i(1)$: the outcome if the unit receives treatment (fertilizer applied; teen uses social media heavily)
• $Y_i(0)$: the outcome if the unit does not receive treatment (no fertilizer; teen is a light user)

The causal effect of treatment on unit i is the difference:

This is what Carolina called the multiverse definition of causality: $\tau_i$ is the gap between two parallel worlds — the one that happened, and the one that didn’t. Simple to write. Impossible to observe.

Fundamental Problem

Here is the core obstacle. When the farmer applies fertilizer, they observe $Y_i(1)$ for that plot. When they don’t, they observe $Y_i(0)$. They can never observe both for the same plot at the same time. Only one branch of the multiverse is ever accessible.

Carolina named this Holland’s Fundamental Problem of Causal Inference and reframed it as a missing data problem. Imagine a table with one row per unit and two columns: $Y_i(1)$ and $Y_i(0)$. For every row, exactly one cell is filled — whichever potential outcome was realized by the treatment actually received. The other cell is permanently blank. Individual causal effects are row-level subtractions; with half the table missing, they cannot be computed.

Average Treatment Effect

The move that makes progress possible is to shift the target. Instead of individual causal effects $\tau_i$, aim for the Average Treatment Effect (ATE) — the average of $\tau_i$ across all units:

By linearity of expectations, this is the difference between two population averages. And now there’s a tempting shortcut: compute the observed average outcome among treated units ($\bar{Y}_1$) and the observed average among untreated units ($\bar{Y}_0$), and use $\bar{Y}_1 - \bar{Y}_0$ as the estimate.

This plug-in estimator is valid — but only under a critical condition. The treated and untreated groups must be identical in expectation on every characteristic that matters: they must look the same, on average, in the counterfactual world where neither received treatment. In other words, untreated units must be a plausible stand-in for what treated units would have looked like had they not been treated — and vice versa.

This is a counterfactual claim. It cannot be directly tested. Making it credible is the entire job of causal inference.

Identification Strategies

Randomization

The most powerful tool for making that claim credible is randomization. If treatment is assigned by a coin flip, then by the Central Limit Theorem and the law of large numbers, treated and untreated groups will be identical in expectation on every pre-treatment characteristic — observed and unobserved. The coin doesn’t know which plots are sunnier, which teenagers are genetically predisposed to depression, or which farmers are more skilled. It allocates blindly, producing groups that are comparable on everything.

This is why randomized controlled trials are the gold standard. Randomization doesn’t just control for the confounders you thought of — it controls for the ones you didn’t.

The problem is that many of the most important questions in social science cannot be randomized. You can’t randomly assign victimization to study its effect on political participation. You can’t randomize wars to study their effect on national identity. You can’t randomly relocate immigrants to study wage effects without raising serious ethical concerns. Carolina flagged the debate this creates: some researchers argue it’s better to answer narrow questions credibly than to attempt big questions without the machinery to answer them properly. Her own view is that this is a false choice — rigorous and ambitious work is possible, but it requires being honest about what your design can and cannot establish.

When randomization isn’t available, two threats to the plug-in estimator become salient.

Confounding arises when a hidden third variable causes both the treatment and the outcome, creating a spurious association. Wine drinkers live 21% longer than beer and spirits drinkers in observational data — but wine is expensive, and class affects both beverage choice and longevity. The correlation is real; the causal story is wrong. Museum visitors appear to have lower mortality — but visiting museums requires time and disposable income, both of which also predict better health. In both cases, plugging $\bar{Y}_1 - \bar{Y}_0$ into the ATE formula gives a biased answer because the two groups weren’t alike to begin with.

Selection bias is subtler. It arises when the sample is conditioned on a variable that is itself a consequence of the treatment, distorting the observed relationship. Carolina’s example: in NBA data, taller players score fewer points per game on average. This seems absurd — height obviously helps in basketball. But NBA players aren’t a random sample of tall and short people. Short players only survive the selection process if they compensate with extraordinary skill. Conditioning on “reached the NBA” creates a sample where the height–skill relationship is inverted, making height appear to hurt performance.

Quasi-Experimental Methods

When randomization is off the table, the empirical strategy is to find settings where nature, history, or policy introduced as-if random variation in treatment assignment — and exploit that variation to justify a credible comparison.

The main toolkit, with one canonical example each:

• Lottery/Draft designs: the 1969 Vietnam draft lottery assigned induction order by birthday — a genuinely random process. Researchers use lottery number as an instrument to estimate the causal effect of military service on later earnings, sidestepping the selection problem that volunteers and non-volunteers differ in ways correlated with earnings.
• Regression Discontinuity (RDD): compares units just above and just below an arbitrary threshold — a scholarship cutoff, a voting age, a policy income limit. Near the threshold, which side of the line you fall on is effectively accidental, making the two groups credible counterfactuals for each other. The cost: estimates are local to the threshold and may not generalize.
• Difference-in-Differences (DiD): compares the before/after change in a treated group against the before/after change in an untreated control group. By differencing twice, stable pre-existing differences between groups cancel out. The core assumption — that trends would have been parallel in the absence of treatment — is untestable but often defensible.
• Instrumental Variables (IV): finds a variable that affects treatment but has no direct effect on the outcome except through treatment. Using the instrument as a lever isolates the portion of treatment variation that is as-if random.

What all these methods share: they are different arguments for why the comparison being drawn is plausible — why the untreated group is a reasonable stand-in for the counterfactual treated group. None of them prove it. They make the case.

Returning to the opening headline: do heavy social media users provide a valid counterfactual for light users? Almost certainly not. Heavy and light users likely differ in unobserved ways — genetic predisposition to depression, family stability, peer networks, sleep habits — that independently affect mental health. Worse, the relationship may run in the opposite direction: struggling teens may turn to social media because they are struggling, not the other way around. The researchers’ longitudinal design is richer than a simple cross-section, but without a credible source of exogenous variation in usage, the causal story remains contested.

The lesson Carolina left the room with is not that causal inference is hopeless — it’s that it is demanding. Every causal claim rests on an untestable assumption about what the world would have looked like had treatment not occurred. The researcher’s job is to make that assumption as explicit, as narrow, and as defensible as possible. The entire toolkit of modern empirical social science — from RCTs to RDDs to structural models — is machinery for that one task.