A few months back our friends at CityLab published the results of a survey looking at differences in attitudes about cities and suburbs under the provocative headline, “Overall, Americans in the suburbs are still the happiest.”

Their claim is buttressed with a reported finding that 84 percent of all the respondents in suburbs said that they were “satisfied with their communities”, while only 75 percent of those who reported living in cities felt the same.

While at first glance, this seems to be pretty cut and dried, a closer look at the data suggests that the answer is far less clear.

As with all surveys, it’s worth paying very close attention to the actual question asked, the size of the survey’s margin of error, and the other factors that determine how respondents answer particular questions.

When we consider each of these factors, it actually turns out that it is difficult to make a strong claim that suburban residents are happier than their urban kin.

First, consider the question asked in the State of the City survey.  It isn’t about “happiness”—it’s actually about satisfaction.  This is more akin to a consumer satisfaction.   There’s actually a well-developed happiness literature that asks people about their overall level of happiness. The conventional question is very internally focused, and doesn’t refer to place. The Pew Center has a good introduction to this subject here.   So when we interpret these data, we should think of them not showing whether people are more or less happy than others living elsewhere, but whether they are satisfied or dissatisfied with their communities.

Second, in interpreting survey results, it’s important to consider the sample size and the sampling distribution of error.  The overall survey included 1,656 respondents and the reported margin of error for the survey was plus or minus 3.4 percentage points.  But that margin of error holds only for the entire sample—subgroups of the population (like just the one-third or so of respondents living in cities) are fewer in number and therefore have a larger percentage point margin of error.  That number isn’t reported.  But differences of less than four or five percentage points between sub-groups of the sample are likely to be borderline significant at best.  When differences between sub-groups are small, we shouldn’t make too much out of them.

Third, we know that happiness (or in this case, satisfaction) is correlated with income.  Higher-income people are more likely to say they are happy; lower-income people less likely.  So if suburbs have more higher-income people and cities have more lower-income people, the apparent difference in reported satisfaction could be the product of income, rather than location.  This appears to be the case for the data reported here.

satisfaction

 

Like published happiness research, the State of the City survey shows that reported satisfaction is highly correlated with income. Some 88 percent of those with incomes over $75,000 said their community was excellent or good; only 66 percent of those with incomes of less than $30,000 said the same. It’s worth noting here that the impact of income is larger than the impact of location in influencing satisfaction (a 9 percent difference between city and suburb as opposed to a 22 percent difference between high and low-income groups.  These data suggest that the real headline finding of this survey should be “higher income people are more satisfied with the places they live.”  The unsurprising takeaway here is that more income enables you to afford to live in a community that makes you happy. We could get a more direct answer to this question by looking directly at income data.  While the CityLab article didn’t publish the findings on income by city and suburb, we can observe the differential effect of income on satisfaction levels in cities and suburbs by looking at two other variables—education and home ownership—which tend to be correlated with income.If we look just at the college educated, we find that the differences between cities and suburbs shrink by about half:  College-educated urban residents are almost exactly as satisfied as college educated suburban residents (80 percent v. 85 percent).

If we look just at homeowners—who in general have higher incomes than renters—we find that the differences between cities and suburbs almost entirely disappear.  Urban homeowners, for example, are almost exactly as satisfied as suburban homeowners (84 percent v. 87 percent).

Finally, we might want to consider how the race of respondents influences community satisfaction.  When you dig into the data by race and ethnicity, the entire difference in reported levels of satisfaction appears to be the result of the differential racial and ethnic composition of cities and suburbs.  Non-Hispanic whites living in cities were almost exactly as likely to report being satisfied with their communities (84 percent) as non-Hispanic whites living in suburbs (83 percent).

It’s definitely worth looking hard at data on personal happiness and community satisfaction, but in doing so, it’s critical that we take care to understand what the data are—and aren’t telling us.