SINCE THEIR COMMERCIAL INTRODUCTION IN THE EARLY 1990S, RESIDENTIAL property-valuation models have significantly altered collateral-valuation methods in mortgage origination and underwriting. However, as early participants in the development and deployment of valuation models, we see profound weaknesses in the approaches taken by lenders, vendors and regulators to understand and apply this new technology. This is particularly true in the areas of testing and implementation. [??] In our view, the conventional process by which automated valuation model (AVM) products are evaluated and compared is fundamentally flawed. Much emphasis is placed on the esoteric aspects of property-valuation modeling and third-decimal-point precision in comparing model performance, yet underlying testing methodologies produce results that are neither representative nor predictive of future performance. [??] Also, the application of property-valuation models to collateral-assessment processes has been disappointingly rudimentary, in our view. In a sense, the industry has struggled to replace a complex, multistage appraisal process with a single mathematical calculation--with predictably disappointing results. [??] At the root of the industry's struggle with property-valuation models is a lack of understanding of probability theory and a concomitant desire to seek validation of the accuracy of every individual estimate. If we applied that same philosophy to rating the likelihood that a given airline flight might not reach its destination, attendance at the upcoming Mortgage Bankers Association (MBA) Annual Conference would be disappointingly sparse.
Although estimates vary, by some accounts as much as 50 percent of originations may involve the use of valuation models as an instrument of loan underwriting. Much of this use is concentrated in the refinance and home-equity lending sector, with purchase-mortgage lending generally relying upon conventional appraisal practices.
Despite property-valuation models' increased adoption and in spite of ongoing opposition from the more traditional elements of the appraiser community, lenders still face major challenges in convincing investors and regulators of the value of valuation models.
While many stakeholders have expressed concern about the performance and validity of property-valuation models, it remains to be seen whether these concerns are valid. We argue in this article that if the technology is to be advanced, the focus should be directed at how property-valuation models are validated and deployed rather than on the arcane aspects of property-valuation modeling and testing.
In our view, two areas warrant further discussion: model testing and validation, and redesign of the collateral-valuation workflow process.
New ground--or not
From the start, many vendors promoted property-valuation models as outright replacements for traditional appraisal reports. Lenders were eager to adopt these models to close loans faster and cheaper against a backdrop of high demand and market competition. Yet increased usage has given rise to concerns by the investors and the regulatory agencies about the potential impact on the quality of loans underwritten using valuation models.
Arguably, this is not new ground. Validation of statistical models was addressed in the previous decade, when mortgage origination transitioned from heads-down, industrial underwriting to the use of credit-scoring models when examining consumer credit. However, testing and validation of credit scores fell under the domain of risk management, while responsibility for evaluating and implementing property-valuation models has come to rest with the appraisal departments. It's fair to say they may not be ideally positioned for the task, given the traditional emphasis of these departments on the accuracy of every single property appraisal.
This is not to say that appraisal departments should not influence validation and deployment of property-valuation models. …