Confirmation Bias in Property Valuation

A recent decision of the ARB contained an issue which caught my eye:

“without that information, the Moving Party cannot determine if the analysis is complete, or whether (MPAC) used all appropriate sales information of just chose (sic) those properties (intentional or not) that confirm its position on value while ignoring other relevant sales that contradict its position….If (MPAC’s) list of properties is a self-confirming selection then the value selected would tend to be biased.” (743031 Ontario Ltd. v Municipal Property Assessment Corporation, Region 03, 2019 CanLII 103796 (ON ARB))

This was part of a request for documentation; the taxpayer wanted a list of all of the sales that the assessing authority (MPAC) chose not to include in their analysis. The request was understandably denied, but that isn’t the part that was most interesting to me. Of more interest is the suggestion that in choosing sales the assessor might simply focus on those that fall in line with the expected value.

Can this happen? Of course, and it is by no means limited to the assessor’s side of the field. Valuators must always be on guard against against confirmation bias. Think of a valuator, sifting through sales data. Not all sales are useful indicators of value - some are non-arm’s length, for example. At first the valuator is pure and vigilant, but when there is a lot of data, and when it doesn’t seem to coalesce into something meaningful, a dangerous thing can happen. The data which seems to be out of line can be ignored. Labelled an outlier. “Something is odd with that sale - I will exclude it.” When this happens, what you end up with is a list of sales which appear to support a value with a high degree of certainty, when really it is only an illusion.

So how can this be avoided. There are no shortcuts - the appraiser or assessor needs more time. Those seeming outliers might be highly valuable in terms of the information provided. Perhaps the sale which has a strangely high value is adjacent to a recently approved transportation hub. Maybe the comparable with the shockingly low sale price is actually contaminated land. Or maybe they reveal a flaw in our analysis, a factor we are missing, which if corrected for would lead to a solid result.

This is important for all sides in assessment appeals to recognize - assessors, taxpayers and municipalities. The analysis before you may look convincing, but what does the underlying data really show? In the case above the reason why certain sales were excluded will no doubt come out - probably during cross examination. Before you get to that point, make sure that your expert can answer those questions about their own analysis.