Freedom in the 50 States Teaser #3: Weighting the Variables

The new, book-length edition of Freedom in the 50 States: Index of Personal and Economic Freedom will be released on March 28 by the Mercatus Center at George Mason University. In the days leading up to release, I will be “teasing” a few of the novel findings and methods from the study. Here at Pileus, I’ve already posted a couple of teasers over the past few months, linked here:

This post will explain the logic and method behind the weighting scheme in the new edition. Every index of freedom has to use some way of weighting its variables to come up with an aggregate measure of freedom. The Heritage Foundation’s “Index of Economic Freedom” and Fraser Institute’s “Economic Freedom of the World” and “Economic Freedom of North America” essentially weight each variable equally, either within categories that are themselves weighted equally in the overall index (Fraser) or across the index as a whole (Heritage). The most commonly used international indices of democracy, Polity IV and Freedom House, and the first two editions of Freedom in the 50 States use “arbitrary” weights, that is, the researchers weight the categories according to their own judgment using general criteria.

We were unsatisfied with all of these approaches, as well as with inductive statistical alternatives known as “principal component analysis” and “factor analysis.” Here is how we put the case in the book:

Because we want to score states on composite indices of freedom, we need some way of “weighting” and aggregating individual policies. One popular method for aggregating policies is “factor” or “principal component” analysis, which weights variables according to how much they contribute to the common variance—that is, how well they correlate with other variables.

Factor analysis is equivalent to letting politicians weight the variables, because correlations among variables across states will reflect the ways that lawmakers systematically prioritize certain policies. Of course, partisan politics is not always consistent with freedom (e.g., states strong on gun rights tend to be weak on gay rights). The index resulting from factor analysis would be an index of “policy ideology,” not freedom.

Another approach, employed in the Fraser Institute’s “Economic Freedom of North America,” is to weight each category equally, and then to weight variables within each category equally. Of course, this approach assumes that the variance observed within each category and each variable is equally important. In the large dataset used for the freedom index, such an assumption would be wildly implausible. We feel confident that, for instance, tax burden should be weighted more heavily than court decisions mandating that private malls or universities allow political speech.

Previous versions of this index used a subjective weighting system, based on a rough assessment of the importance of each policy in terms of the number of people affected and the value they were likely to place on their infringed freedom. We were dissatisfied with the imprecise and subjective manner in which we constructed those weights, and for this edition we have tried to use a much more objective and independent measure of the “value” of each freedom.

The new, “objective” method of weighting variables is what we call the “freedom value” approach. Here is how we describe it:

In this edition, variables are weighted according to the value of the freedom affected by a particular policy to those people whose freedoms are at stake. Each variable receives a dollar estimate, representing the financial, psychological, and welfare benefits of a standardized shift of the variable in a pro-freedom direction to those people who enjoy more freedom. We base these values on estimates derived from the scholarly literature in economics and public policy that quantifies the effects of policies on behavior.

The “freedom value” of each variable represents the benefits only to those people whose freedoms have been respected. We do not include the benefits to those who wish to take away freedoms. For instance, private companies may benefit from receiving eminent domain transfers, but we count only the costs to those whose property has been taken away. [emphasis original]

We do not include the benefits of taking away freedom because we do not want the index of freedom to be an index of social welfare. If we are to compare freedoms in any way, the logical way of doing so seems to be to look at how the people enjoying those freedoms themselves value them. We cite John Rawls in support of our approach, from his early paper, “Justice as Fairness“:

As an interpretation of the basis of the principles of justice, classical utilitarianism is mistaken. It permits one to argue, for example, that slavery is unjust on the grounds that the advantages to the slaveholder as slaveholder do not counterbalance the disadvantages to the slave and to society at large burdened by a comparatively inefficient system of labor. Now the conception of justice as fairness, when applied to the practice of slavery with its offices of slaveholder and slave, would not allow one to consider the advantages of the slaveholder in the first place. . . . The gains accruing to the slaveholder, assuming them to exist, cannot be counted as in any way mitigating the injustice of the practice. (87-88)

Calculating the “freedom value” of each variable was a Herculean endeavor, to be perfectly honest, but we did it because we wanted to make the investment in a dramatic improvement to the study’s methodology. For some variables, precious little scholarly investigation revealed the information we wanted, and our calculations end up being more like “guesstimates” than precise estimates.

But in general, we found the new approach disciplining. It forced us to reconsider cases in which variables seemed to “double count” an underlying concept. It also forced us to think about how the policies we include really affect people’s lives. For us, education had been an extremely important category in previous editions, and regulations on private and home schools made a big difference in the index. But under the new methodology, education is now worth significantly less, in part because states’ reforms for school choice have not been very far-reaching and have not seriously affected many students. On the other hand, tobacco and gambling laws are worth a lot more than they used to. We weren’t much interested in these freedoms ourselves, and so in previous editions we had underestimated their value to Americans.

Despite this significant methodological shift, however, most states’ scores for equivalent years do not change dramatically in this edition compared to prior editions. Still, the improvements were significant enough that one substantive statistical finding changed: our old measure of regulatory policy had never been significantly associated with in-migration or economic growth, but our new measure of regulatory policy is strongly associated with both, and is even more important than fiscal policy for economic growth. We are looking forward to seeing how the new methodology is received and how it can be improved in future editions.

2 thoughts on “Freedom in the 50 States Teaser #3: Weighting the Variables

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s