Lies, damn lies, and (on-line shopping) statistics.
By Joe Cortright
Here’s an eye-catching statistic: “people in the US buying more things online than in brick-and-mortar stores.” This appears in the lead of a story published this week by Next City.
There’s one problem with this claim: it’s not remotely close to true.
One of the things we pay taxes for is the Census Bureau, which gathers copious amounts of data about us and our economy. For a decade now, they’ve been tracking e-commerce sales. According to their latest report, e-commerce retail sales in the second quarter of 2016 were $97 billion, equal to about 8.1 percent of total retail sales in the U.S.
So how could anyone come up with the claim that we’re buying more on-line than in physical stores? Everyone knows the classic game of telephone, where a series of people are asked to repeat a simple phrase in succession. Between the first telling and the sixth or seventh repetition, some key words have changed, and often the meaning has become garbled or unintelligible. That’s what’s seemed to have happened here.
If you click through to the source cited by Next City, you are taken to a short write-up by Fortune magazine, of a proprietary study conducted for UPS by comScore, a web marketing research firm. Their headline: “Consumers are now doing most of their shopping online.” The article leads with the claim “For the first time ever, shoppers are going to the web for most of their purchases.” While it quotes the comScore and UPS as its source, Fortune doesn’t bother to provide a link to either company’s research (although it does show their stock tickers).
But that’s no problem, we can easily google “comScore, UPS, shopping, survey.” What we then find is comScore’s press release describing the study, and a link to the UPS website with a PDF of the white paper reporting the survey’s results. (Thanks comScore & UPS: That’s good form!)
The headline of the press release makes it clear that its authors are making a much more limited claim: “UPS Study: Avid Online Shoppers Making More Than Half of Their Purchases With E-commerce.” The key word here is “avid.” When you read the UPS study, conducted by Com Score, you discover exactly what they mean by avid online shoppers. First, it’s a survey sample is drawn from ComScore’s proprietary panel, which is composed of heavy internet users. Second, UPS and ComScore restricted survey participation to those panel members who shopped on line at least two times in the previous three months. Third, UPS and ComScore weighted the sample of participants so that at least 40 percent of participants were those who had shopped on line at least seven times in the past three months. So, right of the bat, you have to recognize that this survey is only of tech savvy people, and has been skewed to exclude people who never or rarely shop on line, and weighted heavily toward respondents who do a lot of on line shopping.
What’s more, the white paper is impressively vague about the scope of what’s being measured. Is it the dollar value of purchases? Is the number of different products purchased? Is it only goods, or large goods, or durable goods? Unfortunately the white paper doesn’t include the survey instrument or basic tabulations of the answers to its questions, so there’s no way to tell for sure. Its pretty clear from the text that the survey excluded grocery purchases, and its highly likely that it also excluded purchases of things like gasoline and cars. There’s no breakdown by product category (i.e. apparel, books, electronics, etc), so its just impossible to tell what “a majority of purchases” means, even for this very select group.
There’s nothing wrong with that, if you’re trying to tell merchants about the behaviors of heavy on-line shoppers. But this kind of survey tells you nothing about whether this select sample is typical of all Americans, and there’s no way you can make a claim about whether this reflects what a majority of Americans are or aren’t doing. And, on top of this, we have clear evidence from the Census Bureau that on-line shopping accounts for less than a twelfth of all retail sales.
The growth of on-line shopping is likely to have major impacts on travel behavior, on urban form, and on employment. There’s no question that on line shopping is growing (Census data show it growing nearly 16 percent annually, compared to about 2.3 percent for all retail sales). As we wrote last year, the growth of e-commerce could have a major positive effect on urban congestion, as it eliminates consumer shopping trips (typically by single occupancy vehicle) and improves the efficiency of delivery services (by increasing the number of deliveries per mile traveled). So its really important that we have a good understanding of what’s changing and where (and we’ll have more to say about that in the weeks ahead at City Observatory).
In a simpler time, Paul Krugman once wrote that one ought not to publish statements that can be disproven in five minutes with a pocket calculator and a copy of the Statistical Abstract of the United States. Anybody who publishes an sweeping statistical claim like this one ought to have the ability to click through to the original source or just google a few statistics to verify they’ve got it right. But sadly, that’s not the standard here.
Lies, damn lies, and (on-line shopping) statistics.
Here’s an eye-catching statistic: “people in the US buying more things online than in brick-and-mortar stores.” This appears in the lead of a story published this week by Next City.
There’s one problem with this claim: it’s not remotely close to true.
One of the things we pay taxes for is the Census Bureau, which gathers copious amounts of data about us and our economy. For a decade now, they’ve been tracking e-commerce sales. According to their latest report, e-commerce retail sales in the second quarter of 2016 were $97 billion, equal to about 8.1 percent of total retail sales in the U.S.
So how could anyone come up with the claim that we’re buying more on-line than in physical stores? Everyone knows the classic game of telephone, where a series of people are asked to repeat a simple phrase in succession. Between the first telling and the sixth or seventh repetition, some key words have changed, and often the meaning has become garbled or unintelligible. That’s what’s seemed to have happened here.
If you click through to the source cited by Next City, you are taken to a short write-up by Fortune magazine, of a proprietary study conducted for UPS by comScore, a web marketing research firm. Their headline: “Consumers are now doing most of their shopping online.” The article leads with the claim “For the first time ever, shoppers are going to the web for most of their purchases.” While it quotes the comScore and UPS as its source, Fortune doesn’t bother to provide a link to either company’s research (although it does show their stock tickers).
But that’s no problem, we can easily google “comScore, UPS, shopping, survey.” What we then find is comScore’s press release describing the study, and a link to the UPS website with a PDF of the white paper reporting the survey’s results. (Thanks comScore & UPS: That’s good form!)
The headline of the press release makes it clear that its authors are making a much more limited claim: “UPS Study: Avid Online Shoppers Making More Than Half of Their Purchases With E-commerce.” The key word here is “avid.” When you read the UPS study, conducted by Com Score, you discover exactly what they mean by avid online shoppers. First, it’s a survey sample is drawn from ComScore’s proprietary panel, which is composed of heavy internet users. Second, UPS and ComScore restricted survey participation to those panel members who shopped on line at least two times in the previous three months. Third, UPS and ComScore weighted the sample of participants so that at least 40 percent of participants were those who had shopped on line at least seven times in the past three months. So, right of the bat, you have to recognize that this survey is only of tech savvy people, and has been skewed to exclude people who never or rarely shop on line, and weighted heavily toward respondents who do a lot of on line shopping.
What’s more, the white paper is impressively vague about the scope of what’s being measured. Is it the dollar value of purchases? Is the number of different products purchased? Is it only goods, or large goods, or durable goods? Unfortunately the white paper doesn’t include the survey instrument or basic tabulations of the answers to its questions, so there’s no way to tell for sure. Its pretty clear from the text that the survey excluded grocery purchases, and its highly likely that it also excluded purchases of things like gasoline and cars. There’s no breakdown by product category (i.e. apparel, books, electronics, etc), so its just impossible to tell what “a majority of purchases” means, even for this very select group.
There’s nothing wrong with that, if you’re trying to tell merchants about the behaviors of heavy on-line shoppers. But this kind of survey tells you nothing about whether this select sample is typical of all Americans, and there’s no way you can make a claim about whether this reflects what a majority of Americans are or aren’t doing. And, on top of this, we have clear evidence from the Census Bureau that on-line shopping accounts for less than a twelfth of all retail sales.
The growth of on-line shopping is likely to have major impacts on travel behavior, on urban form, and on employment. There’s no question that on line shopping is growing (Census data show it growing nearly 16 percent annually, compared to about 2.3 percent for all retail sales). As we wrote last year, the growth of e-commerce could have a major positive effect on urban congestion, as it eliminates consumer shopping trips (typically by single occupancy vehicle) and improves the efficiency of delivery services (by increasing the number of deliveries per mile traveled). So its really important that we have a good understanding of what’s changing and where (and we’ll have more to say about that in the weeks ahead at City Observatory).
In a simpler time, Paul Krugman once wrote that one ought not to publish statements that can be disproven in five minutes with a pocket calculator and a copy of the Statistical Abstract of the United States. Anybody who publishes an sweeping statistical claim like this one ought to have the ability to click through to the original source or just google a few statistics to verify they’ve got it right. But sadly, that’s not the standard here.
Related Commentary