REAL ESTATE ISSUES Volume 39, Number 1, 2014
FEATURE
Accuracy of Zillow’s Home
Value Estimates
BY CHARLES CORCORAN, PH.D., CFA, AND FEI LIU
INTRODUCTION
Z
tremendous name recognition. Buyers use it to search for
homes; sellers type in their addresses and get what they
believe to be a value of their homes. But is the site accurate
and should consumers rely upon it?
LITERATURE REVIEW
In recent years, home value estimates have been subject
to heightened scrutiny, with a housing price bubble
followed by a sharp downturn. Interested parties such as
appraisers, tax assessors, buyers and sellers seek reliable
data from which they can derive an unbiased estimate of
value. e real estate industry is based on “information
asymmetry,” which means that one party (typically the
seller) knows more about a product than the other (the
buyer). It’s an opaque market that encourages obfuscation
and leads to awed pricing. A motivation behind the
founding of Zillow.com in 2006 was to make real estate
more like a stock exchange, a transparent market where all
information about every property is readily available and,
as a result, pricing is less imperfect.
1
Zillow provides an estimate of market value for more
than 100 million homes based on a proprietary formula.
In general, it o ers free value estimates, or “Zestimates,”
using data from appraisal districts and from multiple
listing services (MLSs), depending on availability. Zillow
uses a “static” formula employing tax information, and
applies it uniformly across the country. eir stated
mission is “to empower consumers with information and
tools to make smart decisions about homes, real estate and
mortgages.”
2
Zillow is a home and real estate marketplace
created to help homeowners, homebuyers, sellers, renters,
real estate agents, mortgage professionals, landlords and
property managers nd and share vital information about
homes, real estate, mortgages and home improvement.
ey assert to be “transforming the way consumers make
home-related decisions and connect with professionals.”
Zillow partnered with Yahoo! in 2011 to provide the vast
majority of Yahoo’s real estate listings online, cementing
their place as the largest real estate network on the Web
according to several online measurement agencies.
3
e focus of this article is to determine whether Zillow’s
Zestimates re ect actual sale prices. Realtors generally
have been critical of the values produced by Zillow,
claiming the data are secondhand, not locally sourced and
out of date. Realtors with speci c market knowledge are
more likely to know speci c factors a ecting the sale of
a home such as the overall condition of the home, room
ow, landscaping, views, tra c noise and privacy. ese
factors have been called unzillowable.
4
Hagerty
5
studied the accuracy of Zillow’s estimates and
found that they “o en are very good, frequently within
a few percentage points of the actual price paid. But
Charles P. Corcoran, Ph.D., CFA,
is a professor and chair of the Accounting
and Finance Department at the University of
Wisconsin/River Falls. His recent publications
have appeared in Asset International’s CIO,
Global Journal of Business Research,
Journal of International Business and
Economics, The Journal of Accounting
and Finance Research, the Journal of
Instructional Pedagogy, among others. Corcoran teaches Real
Estate Finance. He received his Ph.D. from the University of Minnesota.
Fei Liu is a visiting scholar at the
University of Wisconsin/River Falls.
Fei is pursuing a Ph.D. in Trade and
Finance from Central China Agricultural
University, Wuhon, China.
About the Authors
45
REAL ESTATE ISSUES Volume 39, Number 1, 2014
FEATURE
Accuracy of Zillow’s Home Value Estimates
when Zillow is bad, it can be terrible.” O’Brien
6
asserts
that “Zillow has Zestimated the value of 57 percent of
U.S. housing stock, but only 65 percent of that could be
considered ‘accurate’—by its de nition, within 10 percent
of the actual selling price. And even that accuracy isn’t
equally distributed.” e article cites the state of Louisiana
as an example, where “the site is just about worthless.”
e National Community Reinvestment Coalition led
a complaint with the Federal Trade Commission stating
that Zillow was “intentionally misleading consumers
and real-estate professionals to rely upon the accuracy
of its valuation services, despite the full knowledge of
the company o cials that their valuation Automated
Valuation Model (AVM) mechanism is highly inaccurate
and misleading.”
7
Zillow o en overestimates home values, much as
homeowners themselves do. Goodman and Ittner
8
compare owners’ estimates of value with subsequent sale
prices; their results indicate that homeowners overestimate
value by approximately six percent. Riel and Zabel
9
nd
an 8.4 percent overestimate compared to sale prices.
ese ndings suggest that Zillow estimates are not as
accurate as homeowners’ estimates. Hollas, Rutherford
and omson
10
nd that Zillow estimates overvalue
homes by 10 percent compared to the sale price. Zillow
also overestimates values for approximately 80 percent of
the houses in their sample by at least one percent. ey
conclude that homeowners’ estimates of value may be
more accurate than Zillow’s estimates. e coe cients
on a Zillow model compared to the coe cients on a sale
price model indicate that Zillow prices some housing
characteristics di erently than the market. Speci cally,
vacant properties are overvalued. It appears that Zillow
does not track the occupancy of a property, yet vacancy
is known to a ect value. Moreover, Doshan
11
asserts that
Zestimates are “gamed.” Zillow uses the Zestimate “on
or before the sales date.” In other words, they use the
Zestimate a er the listing price becomes public. at
makes their Zestimate look more accurate than it really
is since the Zestimate can be drastically a ected by the
listing price.
In response to homeowners’ complaints about the quality
of the data Zillow extracts from public archives across
the United States, in 2011 Zillow added tools that enable
homeowners to edit facts and add information about
their properties. Zillow also o ers listing services for
homeowners and real estate agents, which enable these
users to edit and add information, both manually and
through automated data feeds. ese tools are becoming
increasingly popular. At present, nearly 20 percent of
archived properties have been edited through such tools.
By default, Zillow shows the facts that are supplied by
the owner or agent, and these facts are supplemented by
public data. Zillow also uses the user-contributed facts
when computing Zestimates. Zillow’s website declares:
“we’ve made it easier for our users to help us improve
accuracy by incorporating edited home facts into our
Zestimate calculations.”
12
Zillow asserts that the improved
algorithm models have improved the Zestimate median
margin of error to 8.5 percent from 12.7 percent. However,
Gelman and Wu
13
nd that edited facts improve the
completeness of the information that Zillow has in store,
but the “accuracy of Zillow’s edited facts is not high.”
An inherent shortcoming in Zillow’s AVM formulation
is its reliance on assessed valuation. If a property
happens to be in a Proposition irteen (California)
type of jurisdiction, with limited periodic assessment
increases, over time its assessed valuation could be well
below market value. Recent sales and reassessments of
valuation impact the Zestimate. So Zestimate values can
be “o ” signi cantly for a property with no sales history,
in a jurisdiction where assessed value is not signi cantly
increased until a sale occurs.
Zillow’s no-cost, no-hassle model seems to stand apart
from most competitors. Red n
14
o ers a free, no-strings-
attached service but its model is rudimentary, considering
only comparables in deriving value. Trulia.com and
HomeValues.com require a return contact from a realtor;
RealEstate.com requires registration, including disclosure
of phone number and email address; RealEstateABC.
com relies on Zillow’s Zestimates. FreddieMac o ers
its Home Value Explorer. is AVM tool generates an
estimate of property value quickly, relying on a proprietary
algorithm that blends model estimates, a repeat sales
model and a hedonic model. is product is licensed and
serviced through a distributor network. Each distributor
adds services and charges fees.
15
LexisNexis provides a
seemingly sophisticated AVM model incorporating price
indexing, tax assessment values, and a hedonic model that
utilizes comparables sold in the previous year. ere is a
fee for this service.
16
METHODOLOGY
e objective of this research is to compare di erences
between Zillow’s Zestimates and actual sale prices in
di erent markets and at di erent price ranges for single-
46
REAL ESTATE ISSUES Volume 39, Number 1, 2014
FEATURE
Accuracy of Zillow’s Home Value Estimates
family homes. For 2,005 transactions, the following model
was developed for measuring mean error:
(Zestimate value – sale price) / sale price.
To measure for signi cant di erences between the two
markets, and within ve price ranges in each market,
a one-way analysis of variance (ANOVA) was used.
e ANOVA is used to determine whether there are
signi cant di erences among the means of three or
more independent groups. In this study there are ten
groups altogether, ve price ranges within two markets—
suburban St. Louis, Missouri, and St. Paul, Minnesota.
ANOVA compares the variance (or variation) between any
two markets’ data sets to variation within each particular
market sample. If the between variation is much larger
than the within variation, as measured by the F-ratio
17
,
the means of di erent samples will not be equal. If the
between and within variations are approximately the same
size, then there will be no signi cant di erence between
means. Tukey’s test is a post-hoc test, meaning that it is
performed a er an ANOVA test. e purpose of Tukey’s
test is to determine which groups in the sample di er. e
ANOVA measures only whether groups in the sample
di er; it does not measure which groups di er.
is study seeks to measure Zestimate accuracy along two
dimensions. First, measuring accuracy between markets.
Is the Zestimate value more accurate in markets with
better data inputs? And second, between price ranges.
Is Zestimate accuracy between the markets a ected by
property price?
For comparison purposes, a Zillow one-star market
(suburban St. Louis) and a Zillow four-star market
(suburban St. Paul), segregated into ve price ranges, are
analyzed. ese are both large suburban markets in the
Midwest, for which the quality of valuation information
di ers considerably, according to Zillow’s four-star
rating scheme. Four-star markets supposedly provide
the most accurate, “best” Zestimates, followed by three-
star markets, noted as “good,” “fair” two-star markets
and, nally, one-star markets where estimates cannot be
computed accurately or are simply the tax assessor’s value.
Zestimate accuracy is computed by comparing a property’s
nal sale price to the Zestimate on or before the sale date.
Ratings are based on accumulated data over the previous
three months. Zillow promotes the star-rating scheme
from an implied presumption that a four-star rating must
be good, as it exceeds the other three-star categories and
is termed “best.” A Tukey post-hoc test was conducted on
multiple price range comparisons between the
two markets.
Of the 2,005 properties analyzed, 849 were in the St.
Paul market and 1,156 were in the St. Louis market.
Five price ranges were employed: (1) < $103,000; (2)
$103,000–$203,000; (3) >$203,000–$253,000; (4)
>$253,000–$353,000; and (5) > $353,000. e $203,000
price benchmark was based on the median existing single-
family home price for the second quarter of 2013.
18
FINDINGS
In aggregate, for both markets and for all prices ranges,
the mean error is 24.8 percent. Mean error rates in the
four-star (St. Paul) market compared with the one-star
(St. Louis) market are signi cantly di erent, with a mean
error rate of 17.15 percent in the four-star market and
30.48 percent in the one-star market. e signi cance level
is 0.000 (p = .000), which is below 0.05. Note the large
F-ratio. See Figure 1 and bottom of Figure 2.
Even though Zestimate values are signi cantly closer
to sale prices in the four-star market compared with
the one-star market, the di erences are most prevalent
among properties with sale prices under $203,000, the
benchmark price level used in this study. For homes under
$103,000, four-star market data may not have signi cantly
better information value than the one-star market, given
mean error rates of 52.43 percent and 64.23 percent,
respectively. Further, overestimates are far more common
on the lower-priced homes. Zestimates exceed actual
market values in 63.44 percent of all transactions, but for
properties with sale prices under $103,000, 93.08 percent
(121/130) of properties in the four-star market and 95.14
percent (333/350) of properties in the one-star market are
associated with overestimated Zillow values.
Figure 1
One-Way ANOVA
Di erence Sum of
Squares
df Mean
Square
F Sig.
Between
Groups
Within
Groups
Tot al
85.976
137.958
233.934
9
1995
2004
9.553
.069
138.143 .000
Signifi cance at .05 level
Source: SPSS statistical package
47
REAL ESTATE ISSUES Volume 39, Number 1, 2014
FEATURE
Accuracy of Zillow’s Home Value Estimates
For homes priced between $103,000 and $203,000, the
four-star market does provide an outcome signi cantly
di erent from the one-star market, with mean error rates
of 10.77 percent and 19.68 percent, respectively.
Within higher price ranges, above $203,000, di erences
between the two markets are not signi cant, with mean
error rates ranging from 9.53 percent to 14.63 percent.
See Figure 2.
CONCLUSION
e four-star market had a signi cantly lower mean error
rate than the one-star market, 17.15 percent versus 30.48
percent. High mean error rates are concentrated among
lower-priced homes. At prices above the median home
price of $203,000, di erences between the four-star and
one-star markets are not signi cant.
While di erences between the two markets are signi cant
for homes selling for less than $103,000, the mean error
rates are so great that they are of little value in either the
four-star or one-star markets. A four-star’s mean error of
52.43 percent indicates little more credibility than a one-
star’s 64.23 percent. While di erences at all price levels in
both markets are usually overestimates, at this lowest price
level they are almost always overestimates.
Di erences between the two markets are also signi cant in
the $103,000–$203,000 price range. But with a mean error
in the four-star market of 10.77 percent, this is close to the
10 percent error level noted by O’Brien as an acceptable
threshold. So for properties in this price range, a four-star
rating may be meaningful.
For the three price ranges beginning with the national
median of $203,000 and above, di erences between the
four-star and one-star markets are not signi cant. With
the exception of the $203,000–$253,000 price range, this
does not imply improved outcomes in the four-star market
for the top two price ranges. Di erences in both markets,
while not statistically signi cant, are quite large, with
mean error rates ranging from 11.54 percent to
14.63 percent.
Within the middle price range, $203,000–$253,000, the
smallest di erences are found within both markets. In the
four-star market, the mean error rate is 9.53 percent, while
in the one-star market it is 12.38 percent. is di erence
is, again, statistically insigni cant.
Zillow’s value as a pricing tool is questionable. With the
possible exception of the $203,000–$253,000 price range,
the four-star designation is of little value. Even the best
results in the four-star market produce mean error rates
approaching 10 percent. In both markets and for all
other price levels, mean error rates are above the
10 percent level. Accuracy of 10 percent still implies an
error of more than $20,000 for an average price property.
While Zillow may be a useful tool, providing an ever-
changing snapshot of home prices, don’t bet the ranch
on it.
■
ENDNOTES
1. For details about Zillow’s estimation methods and models, see
http://www.zillow.com/zestimate/#what.
2. http://www.zillow.com/corp/About.htm.
3. http://websearch.about.com/od/Alternative-Search-Engines/p/
Zillow-Com-Real-Estate-Search-Made-Simple.htm.
4. http://forsalebylocals.wordpress.com/2006/08/18/unzillowable-the-
perfect-term/
5. Hagerty, James R., “How Good Are Zillow’s Estimates?”
e Wall Street Journal, Feb. 14, 2007, sec. D.
6. O’Brien, Je rey, “What’s Your House Really Worth?”, Fortune,
Feb. 15, 2007.
Figure 2
Tukey Post-Hoc Test for Multiple Comparisons
Price (x1000)
<103
103-203
<203-253
<253-353
<353
All
Di erences
between
markets
(mean values)
SP - SL
-.11793235*
-.08910191*
.02845627
.008355306
.02245725
Signi cance
.001
.000
.997
1.000
1.000
SP
0.52434 (130)
0.10771 (434)
0.09531 (133)
0.11541 (99)
0.12386 (53)
0.17147 (849)
SL
0.64227 (350)
0.19682 (344)
0.12376 (138)
0.12376 (208)
0.14632 (116)
0.30475 (1,156)
*denotes signifi cance at the .05 level.
SP=St. Paul, SL= St. Louis
Source: SPSS statistical package
Mean percent difference
within markets, (sample size)
(Zest.-sale price)/sale price
48
REAL ESTATE ISSUES Volume 39, Number 1, 2014
FEATURE
Accuracy of Zillow’s Home Value Estimates
7. http://www.housing-information.org/articles/ c_complaint_against_
zillow_online_appraisal_site.
8. Goodman, John L., Jr., and John B. Ittner, “ e Accuracy of Home
Owners’ Estimates of House Value,” Journal of Housing Economics,
Vol. 2, Issue 4, December 1992, pp. 339–357.
9. Kiel, Katherine A. and Je rey E. Zabel, “ e Accuracy of Owner-
Provided House Values: e 1978-1991 American Housing Survey,”
Real Estate Economics, Vol. 27, Issue 2, 1999, pp. 263–298.
10. Hollas, Daniel, Ronald Rutherford and omas omson,
Appraisal Journal, Winter 2010, Vol. 78, Issue 1, pp. 26–32.
11. Doshan, Brett, http://www.HomeVisor.com, Oct. 19, 2012.
12. http://www.zillow.com/zestimate/#update, April 4, 2014.
13. Gelman, Irit and Ningning Wu, Proceedings of the 44th Hawaii
International Conference on System Sciences, p. 9, Jan. 5, 2011.
14. https://www.red n.com/what-is-my-home-worth?estPropertyId=
51230374&src=landing-page, April 5, 2014.
15. http://www.freddiemac.com/hve/distributors.html, April 5, 2014.
16. http://www.lexisnexis.com/legalnewsroom/lexis-hub/b/legaltoolbox/
archive/2011/09/23/automated-valuation-models-from-lexisnexis.aspx.
17. e F ratio is the ratio of the variance between groups to the variance
within groups, i.e., the ratio of the explained variance to the
unexplained variance.
18. Op. cit. at 12.
49
