I’ve been reading several papers lately that examine the effects of various government policies on various social and economic outcomes. Increasingly, I find myself wondering what these studies actually conclude with “null” results. (By the way, I am sure that this issue has been raised before, but I’ve been thinking a lot about it lately, and I figured that’s what a blog is for.)

A (justifiably) standard approach in these literatures is as follows:

1. Describe why the outcome variable, *y*, is important, how it is measured, acknowledge weaknesses in the data, *etc.*

2. Describe the vector (list) of *K* independent variables, *X*, acknowledge they are imperfect, describe why they are still arguably useful, and perhaps link these with a theory explaining why they might affect *y*.

3. Apply a statistical model to generate estimates of the effect of the various variables in *X* on *y.*

For a lot of very good reasons, the standard approach in thinking about (or “modeling”) the effect of *X* on *y* is as based on some equation that essentially boils down to the following:

,

so that essentially measures the linear impact of variable on the outcome variable, . (The function captures nonlinearities, particular for situations in which is meaningfully bounded, like a proportion or probability.)

Then, typically, if the researcher is unable to reject the hypothesis that the estimated value of , is equal to 0, the conclusion is that there is little or no evidence that affects . This is usually followed by a puzzled expression and an awkward pause.

In many respects, this is perfectly reasonable: this approach is a classical way to model/uncover the relationship between the outcome variable and independent variables. And, particularly in modern social science, it is broadly and well-understood as a means to conceptualize/present results. So, **I’m not saying we shouldn’t do this.** That said, I am saying that we should think about the political relationship between the outcome and independent variables.

Now, for the sake of argument, suppose that , to focus the discussion. Then, suppose that is a politically important variable that voters “like” (i.e., want higher levels of), such as per capita income in a state and that represents a policy controlled/set by political actors. Now, suppose that political actors are responsive to voter demands, so that they set so as to maximize .

The first order condition for maximization of with respect to is . In general, $f$ is a strictly increasing function, so that implies that .

We have reached this conclusion without presuming *anything* about the true relationship between and . Thus, if one is unable to reject the null hypothesis that , isn’t it arguably better to conclude that the *marginal effect* of on is zero, *given the observed data and behaviors underlying them* than that has no apparent effect on ?

Put another way, if we find *in observed, real-world data* that the effect of on is unambiguously non-zero, shouldn’t we be *more* surprised than if we fail to uncover a systematic, non-zero (linear) effect of on ?

With that, I leave you with this.