Have a Problem With Zillow’s Zestimate?

Have a Problem With Zillow’s Zestimate?

(Get Ready There are More AVMs Coming Your Way)

An edited version of this article also ran at Inman News 2.23.15 (inman.com)

If you have concerns about how Zillow’s AVM affects your seller’s perception of their home’s value, get ready as automated valuations will be a major battle ground for portals starting this year.

To stay competitive on the internet all of the major portals and perhaps even a couple of the largest franchises must meet or exceed Zillow’s AVM offering—the market will otherwise quit visiting those portals.

Why does the marketplace demand an automatic valuation?  Put yourself in your clients’ shoes: what fun, you can see your home’s value and some comparable homes; you can see cool aerial pictures of your neighborhood; you can spy on your neighbors’ values (and rooftops and yards); and best of all you can do it all without calling a broker.  This was all impossible for the average home owner before Zillow.  They love it; they now demand it.

With new AVMs coming your way would you like to know why they are inaccurate?

The problem starts with the industry’s appraisal model of valuation and gets worse as you automate the concept with computer programs.  Match three homes to a subject that are comparable in location, style, size, and condition, and then make dollar adjustments to equalize any inconsistencies—all done by trained professionals.  This is the way appraisers, brokers, lenders, investors, the secondary market, economists, and Wall Street values homes.

But how do you decide the dollar amount to adjust between a home 3,300 square feet in size and another at 3,100 square feet?  How much do you adjust for the difference between 4 bedrooms and 3 bedrooms?  Here’s where the appraisal model starts unraveling very quickly, especially when you switch professional judgment for mathematical algorithms.

Sometimes the home with the greatest square feet sells for the most, and sometimes the home with the least square feet sells for the most—even after controlling for location, size, style, and sale date.  Here’s a spreadsheet of sold properties from the same neighborhood, with all homes selling within 6 months of each other, all the same style, and all within 20% of the size of the subject.  Focus on the Price per Square Foot column, then note the items appraisers typically adjust for (bathrooms, bedrooms, square feet, basement size, year built, and lot size).


Note the 36% spread from lowest ($157) to highest ($213) values per foot.  See how the square feet size, and bedroom and bathroom counts, and year built, and lot size are not consistent with value per foot?  Those variances are not easily accounted for by automated systems and honestly how humans adjustment for them is wrong.

If you step away from using “professional judgment” and analyze sale data using multiple regression you find that sometimes bedroom count does have a high correlation to value, but other times almost no correlation to value (correlation essentially means X moves in relation to Y: as #bedrooms goes up, sale price goes up).  Look again at the Excel spreadsheet. Are bath and bedroom counts consistently aligned with price per foot?  Does square feet size consistently align with value per foot?  Does Year built or lot size consistently align with price per foot?  They do not.  Why then does the industry automatically make adjustments for these differences if they do not consistently correlate to value?

You have the same problem with all eight of the physical characteristics used to make adjustments to value: above ground size, bedrooms, bathrooms, basement size, percent of basement finish, lot size, garage spaces, and year built.  At times each has a high correlation to value; other times each has little or no correlation to value (see the study results).  Open up this study and you can scroll through a large series of neighborhoods in different locations and different price points.  See how the eight physical characteristics used by the real estate industry to adjust value vary greatly in their correlation to value?

In neighborhoods where there is very low or no correlation to value you should not make any adjustments to value based on these physical features, but everybody does it anyway.

Why this matters to you is the same reason it matters to Zillow: the physical characteristics of each home only account for a percentage of a home’s value and their correlation to value is inconsistent.

When used appropriately a home’s physical features can give you a base value.  The range of potential values for each home is established by what the market desired the most and what they were willing to pay to attain their desires.  What the market desires has to do with the home’s form, flow, and level of finish.  This is what caused the 36% spread in values per foot mentioned above.

The correlation or impact to value is created by what the marketplace desires the most and what they will pay to attain their desires—specifically for each neighborhood.  There’s no consistent correlation between neighborhoods or price ranges.

All of this means that when an appraiser automatically makes dollar adjustments for differences in square feet and bedroom count without knowing the multiple regression results for that neighborhood, they are probably doing things wrong.  When computer programs try to do the same thing, without any professional judgment, it can get ugly.

You are about to have a half-dozen new “HAL 9000” AVMs (2001 A Space Odyssey) suggesting prices to your clients—and you know how that turned out for everybody in the movie.

Now I know that Zillow’s AVM is not a simpleton; it does not try to derive value based on pulling three comparable homes and making rudimentary dollar adjustments to equalize value.  It uses far more sophisticated Random-Forest-Regression, which is more like running a thousand decision tree simulations nearly simultaneously to derive its value.

The sophistication of their analysis is not its problem; its problem is that it is built on the wrong premise.  Its downfall is that it uses all physical data and does not take into consideration the emotional triggers of each home’s individual form, flow, and finish—the real reasons for the range of values we saw on our Excel spreadsheet above.  This is why the Zestimate offers a value, but to meet a 95% confidence level must then offer a range of values above and below the estimate.  That range of values is caused by the emotional impact of the form, flow and finish of each home, which the Zestimate ignores.

The new AVMs will try to accommodate for these value errors by using text analysis to draw out what the market most desires and what they will pay to attain those desires.  They will then utilize both the physical features of the homes and the analyzed text to better estimate values.

The good news is increased AVM accuracy on the internet.  And if you want your homes to be more accurately valued by these new programs your text descriptions of them must be precise and contain concrete terms such as “slab granite counters” and “42 inch cherry cabinets”, and avoid meaningless fluff like “well maintained” or “wonderful home”.

The bad news (for brokers) is just as with HAL as these new AVMs become more and more accurate the sellers will trust them all the more over the next few years.  As the new AVMs are perfected home sellers will rely on their accuracy and may simply place their homes for sale near an AVM estimated value in portals without brokers.  Brokers will still get paid for bringing a buyer to closing, but your current business model of being paid for listing and being paid for selling a home may be gone within five years.

This is not to scare you or to appear pessimistic; these scenarios are all strategic alternatives you must consider in your longer-range business plan.

Now you know why current AVMs are inaccurate, and you have some idea of how they will improve and how they will eventually affect your business model.

“Dave, this conversation can serve no purpose anymore; goodbye.”


I try to keep you updated on reader feedback and my answers, so once again here they are below (from Inman News readers)

Q: Good article – Over time the AVMs will improve – as the housing stock turns over the prices paid will replace the estimates and the systems will continue to learn from actual activity. The portals will probably evolve to allow the home owners to input more details of their home, improvements, maintenance etc – these estimates still won’t replace the local expertise and knowledge of the real estate agent nor can they accurately take into consideration the real time local market fluctuations – at best they are a very rough guide.

A: Hi Colin, thanks for the comments. We instantly think “portals” when we talk about innovations like AI in AVMs. I saw Brad has an article on the “brand wars” today–their strategies feel like the same old image posturing we see from the major franchises every new year with each brand trying to convince the marketplace their brokers/systems/signs mean something. Can you imagine the impact of a major brand actually innovating and creating the best AVM? Now you have a real impact–not only taking eyes from the portals but actually differentiating themselves based on what the market desires—not to mention the business they’d take from the other franchises.

Q: Technology is inevitable; as with anything else, will people use it? I think perhaps this might be a generational trait?it’s almost ironic; new technology gives you the power to research and find your own real estate investments online, but the same technology wont let you drive your own car? =)

A: Hi Tito, “will people use it”? I’m ancient, having been able to complete my undergraduate degree with a single exposure to something called visi-calc. My graduate degree on the other hand was spent with great time on a computer–perhaps a “286” microprocessor. We now have folks in their 30’s who grew up surrounded by technology–can hardly function without it really. You may be the same way yourself now. When the complexities of technology are intuitively design down to the simplest user interface based on the greatest desires of the marketplace, people will use it; you won’t be able to stop them from using it.

Q: Cars driving themselves is no different than a Tommahawk allowing the crazies to meet Allah…or planes on auto pilot, that technology has been around a while, it’s just being more broadly applied. You can never fully account for the variables in real estate, boots on the ground by a pro is always the way

A: Yea, boots on the ground is what the drone operators always say as they launch the hell-fire missile :)

Q: The fundamental problem with AVMs of any type lie in the insanely simple “BS in, BS out” algorithm. What are these conclusions based upon? While we can rely upon price and date as relatively accurate, something as basic as GLA is being collected from what source? External influences are being considered by whom? Physical condition of the home by? “Illegal” improvements like additions or basement finishing? Motivations of buyer and seller? Closing costs or personal property jammed into a sale? Seller held mortgages? And oh, a hundred other things? Of course the public loves AVMs, the public loves WebMD and the like. However real estate is perhaps the single most unique entity out here with major variables that cannot be accounted for by a program. Straight stats, sure – but stats from Joe the Assessor? Depending on tax records for data? Please. The appraisal industry helped fuel this by relinquishing piles of data over the years, whether or not that has an impact remains to be seen – I doubt it. At the end of the day, use AVMs as very very broad book ends, nothing more. And isn’t it fully how buyers and sellers look to AVMs to support their positions? When they do they’re great, when they don’t….break out the Kleenex.

A: Yes, the old BSBO, occasionally known as GIGO. That is perhaps the most pivotal problem of the AVM industry. The physical data errors can be limited (square feet, bedroom count, etc.). The sadder part of the formula is the quality of the property descriptions by real estate brokers. Just look at the fluff tossed in there with little thought: meaningless rhetoric and clichés. Within any neighborhood the most valuable qualities of the top tier homes can be discovered and used in AVM calculations. Unfortunately brokers at best use “intuition” and offer a description of perfunctory phrases. Yes: GIGO. On the other hand 99% of the population thought the primitive original computers were useless oddities. Oh how technological innovation can occur when you have an open mind and indomitable perseverance.

Q: No AVM provider can guarantee 100% accuracy, but AVMs can function as effective lead generation tools, leading to richer conversations between home shoppers and agents. Because home shoppers expect AVMs in the marketplace, not supplying an AVM could lead to traffic going elsewhere to find one. Onboard Informatics has an industry leading AVM that takes a scientific approach to valuation. We use over 15,000 separate geographic segments to determine property values for each of the 77 million homes we cover. Each property is run through multiple valuation engines to come up with the valuations.

A: Well, aside from the sales pitch, let me touch on the market expecting an AVM. Hey, an AVM helped launch and keeps zillow on the lead of internet real estate. So OI is correct, the market now expects an AVM as a basic experience at a good website. Here’s a quick link explaining the Kano model of meeting user expectations. This will help you think how to best serve your market whether with technology or hands-on service


Q: Bad example, I agree with Jim Pruitt. I would filtr for age. You have late 1970s and early 1980s in one group, with pretty consistent price psf numbers, decreasing as size increases. Then you have mid 1990s, which might be the “same” neighborhood, but not the same sections a decade and a half later. A Realtor familiar with the neighborhood will know the pricing better than a computer. And we can tell from photos and comments in the comps which have been updated and which haven’t. Only an idiot would accept an AVM as true value when it comes to actually selling, so I disagree with your 5 year prediction that AVM will supplant the know Edge able opinion of a local Realtor who knows the neighborhood.

A: Using how you would cluster the data, using year of construction, and looking at the spreadsheet, there’s zero correlation between the YOC and price per foot. So while you will narrow the location down if each cluster was built incrementally, you still have a big spread unexplained by the year built. You have price points at all ranges of the YOC array. The 1990’s houses land at the low, middle, and top value tiers and do your 1980’s houses.

Q: Bad example – this neighborhood is a nightmare. A seller’s agent is going to pick the higher comps. The buyer’s agent will recommend a price based on lower comps. Who knows where to appraiser is going. Realtors aren’t all knowing, we generally don’t anything about the fit and finish inside comps. We are guessing just like everyone else. What are the zestimates for a house in this neighborhood? It is not hard to predict the future here – lenders will develop their own AVM’s – and that will be their appraisal. Then, the major real estate companies will follow to keep up and be competitive. Keller Williams, etc will each have their own AVM’s. Forget Zillow and Realor.com, AVM’s will be the real game changers for real estate,

A: Bad example? Most neighborhoods have worse value spreads, and I’ve looked at thousands. Take a look at the link for the multiple regression analysis provided above. Lenders have been using backend AVMs for years, zillow for a decade. Brokers are way behind.

Q: While AVM’s may get more accurate the word on the street is that buyers & sellers still need good old fashioned local professionals to help them buy & sell a home. I have the opportunity to talk with 100’s of productive agents every month in Pennsylvania and have found that when I talk to the productive folks they aren’t really bothered but the AVM’s as a whole. Could they be more accurate? Sure they could but that would require a larger investment from the AVM itself. Could that affect agents? Possibly but the proof will be in the pudding as they say.

A: Hi John, I agree that providing a professional, skilled service is absolutely needed. And yes many folks have referrals providing that daily. But some do not. So would a high quality AVM help pull potential clients to a broker’s website? I think so. And with nearly 100% of potential clients perusing the web for home information is it a good idea to give the market what they desire? I think so. So everybody wins: the brokers get more leads that are educated on pricing ahead of time, and the sellers and buyers win as they get cool information they desire. About the only loser in this scenario is the broker refusing to accommodate the market with what they desire.





Leave a Reply

Your email address will not be published. Required fields are marked *