Let’s stop with the absurd surveys masquerading as serious research
By Joe Cortright
No: Eighty percent of today’s 8 to 23 year-olds won’t be buying houses in the next five years
At City Observatory, we get a regular stream of press releases and media advisories about the results of surveys and other market research, purporting to tell us the preferences of the American people. Some of what we get is carefully researched, thoughtfully presented, and genuinely useful–we really appreciate the insights we get from the likes of Zillow, Redfin, ApartmentList, Indeed, and BuildZoom. These companies have very smart researchers, undertake careful, disciplined research efforts, and are transparent about their data and method. Thanks to them, we have more detailed and more timely data on a range of markets, especially housing markets.
But some of it is abysmal, with sloppy and questionable “research” which is actually thinly-veiled self-promotion and click bait. We get entreaties from their media relations staff inviting us to share their surveys with our audience. The latest of these is a survey commissioned by real estate firm “Property Shark,” which purports to reveal generational trends in home-buying patterns. Ordinarily, we’d just click “delete” and look the other way. But this particular survey is so genuinely awful and demonstrably wrong, than we can’t help but call it out.
The study’s headline finding is that “Gen Z is keen on entering the real estate market as soon as possible – a whopping 83% of Gen Z plan to purchase a home within the next 5 years.”
Survey says . . . Absolutely nothing.
Let’s unpack that claim, shall we?
The definitions people use for different generations are a bit slippery, but this study says GenZ is persons born between 1995 and 2010. That means that these people are between the ages of 8 and 23 today, and will be between the ages of 13 and 28 in 2023, i.e. five years from now.
There are several problems with this assertion. First, it’s unclear how many 8 year-olds , or 12 year-olds or even 18 year-olds, actually have given serious thought to homeownership. (And this is the group that the survey claims to have reached via Amazon’s mechanical turk; the press materials claim, in our view preposterously, that “Our survey sample size has a 3% margin of error and a 99% confidence level.” That implies that they’ve got a statistically valid random sample of that entire age group.
Second, assuming that virtually no one under age 18 is financially or legally able to own a home, in order for this prediction to become true the homeownership rate for persons 18 and older would be 100 percent in just five years. (Even that wouldn’t get you to an 83 percent homeownership rate, because about 30 percent of the persons in this age group would be under 18 in 2023.)
Third, and perhaps most importantly, the current homeownership rate for persons aged under 25 to 34 is about 37 percent. So if you believe this survey, you think that the homeownership rate for 13 to 28 year olds will be something like 80 percent in just five years. This would be a pretty stupendous reversal: homeownership among 25 to 34 year olds has fallen 8 percentage points in the past decade. It would also mean that these very young people would have a higher homeownership rate than any other demographic group: Millennials, GenX or BabyBoomers.
If its headline finding is that wrong, it’s clear that we can safely ignore all of the survey’s other claims about what size house, neighborhood characteristics and home amenities these GenZers prefer. If a 16 year-old claims they’re going to be a homeowner in the next five years, you can be their estimate of the number of square feet that they’ll buy is even more unreliable that their ability to know whether they could even afford to own a home.
While this is a far more egregious example than most, it is emblematic of more common surveys that ask people whether they would “like” to own a home. Obviously, and correctly, most people associate homeownership with wealth. Generalized questions about the desirability of homeownership may simply be surrogates for asking “would you like to be wealthy and successful.” These kind of open-ended, aspirational questions seldom test this in any meaningful way by asking about trade-offs between alternatives.
We realize, of course, that we’ve played right into the hands of Property Shark’s media team. They’ll no doubt duly mark down our mention–and your eyeballs–as “earned media” for their website. But if they’re this cavalier about what they put out as “research,” then you would be well advised to discount heavily whatever else they might be telling you about real estate markets (and we suspect that they actually have some information of value to share). In the future, though, if they’re going to undertake the effort to do research and share it–and they actually want to enhance their reputation, rather than diminish it–they might want to take a few notes from the Zillows, Redfins and other’s we mentioned early on. If they do, we’ll be happy to share–and praise–their work.
Let’s stop with the absurd surveys masquerading as serious research
No: Eighty percent of today’s 8 to 23 year-olds won’t be buying houses in the next five years
At City Observatory, we get a regular stream of press releases and media advisories about the results of surveys and other market research, purporting to tell us the preferences of the American people. Some of what we get is carefully researched, thoughtfully presented, and genuinely useful–we really appreciate the insights we get from the likes of Zillow, Redfin, ApartmentList, Indeed, and BuildZoom. These companies have very smart researchers, undertake careful, disciplined research efforts, and are transparent about their data and method. Thanks to them, we have more detailed and more timely data on a range of markets, especially housing markets.
But some of it is abysmal, with sloppy and questionable “research” which is actually thinly-veiled self-promotion and click bait. We get entreaties from their media relations staff inviting us to share their surveys with our audience. The latest of these is a survey commissioned by real estate firm “Property Shark,” which purports to reveal generational trends in home-buying patterns. Ordinarily, we’d just click “delete” and look the other way. But this particular survey is so genuinely awful and demonstrably wrong, than we can’t help but call it out.
The study’s headline finding is that “Gen Z is keen on entering the real estate market as soon as possible – a whopping 83% of Gen Z plan to purchase a home within the next 5 years.”
Let’s unpack that claim, shall we?
The definitions people use for different generations are a bit slippery, but this study says GenZ is persons born between 1995 and 2010. That means that these people are between the ages of 8 and 23 today, and will be between the ages of 13 and 28 in 2023, i.e. five years from now.
There are several problems with this assertion. First, it’s unclear how many 8 year-olds , or 12 year-olds or even 18 year-olds, actually have given serious thought to homeownership. (And this is the group that the survey claims to have reached via Amazon’s mechanical turk; the press materials claim, in our view preposterously, that “Our survey sample size has a 3% margin of error and a 99% confidence level.” That implies that they’ve got a statistically valid random sample of that entire age group.
Second, assuming that virtually no one under age 18 is financially or legally able to own a home, in order for this prediction to become true the homeownership rate for persons 18 and older would be 100 percent in just five years. (Even that wouldn’t get you to an 83 percent homeownership rate, because about 30 percent of the persons in this age group would be under 18 in 2023.)
Third, and perhaps most importantly, the current homeownership rate for persons aged under 25 to 34 is about 37 percent. So if you believe this survey, you think that the homeownership rate for 13 to 28 year olds will be something like 80 percent in just five years. This would be a pretty stupendous reversal: homeownership among 25 to 34 year olds has fallen 8 percentage points in the past decade. It would also mean that these very young people would have a higher homeownership rate than any other demographic group: Millennials, GenX or BabyBoomers.
If its headline finding is that wrong, it’s clear that we can safely ignore all of the survey’s other claims about what size house, neighborhood characteristics and home amenities these GenZers prefer. If a 16 year-old claims they’re going to be a homeowner in the next five years, you can be their estimate of the number of square feet that they’ll buy is even more unreliable that their ability to know whether they could even afford to own a home.
While this is a far more egregious example than most, it is emblematic of more common surveys that ask people whether they would “like” to own a home. Obviously, and correctly, most people associate homeownership with wealth. Generalized questions about the desirability of homeownership may simply be surrogates for asking “would you like to be wealthy and successful.” These kind of open-ended, aspirational questions seldom test this in any meaningful way by asking about trade-offs between alternatives.
We realize, of course, that we’ve played right into the hands of Property Shark’s media team. They’ll no doubt duly mark down our mention–and your eyeballs–as “earned media” for their website. But if they’re this cavalier about what they put out as “research,” then you would be well advised to discount heavily whatever else they might be telling you about real estate markets (and we suspect that they actually have some information of value to share). In the future, though, if they’re going to undertake the effort to do research and share it–and they actually want to enhance their reputation, rather than diminish it–they might want to take a few notes from the Zillows, Redfins and other’s we mentioned early on. If they do, we’ll be happy to share–and praise–their work.
Related Commentary