Statistical Analysis Pinpoints Files to Review for Bias
Preiss, Arthur R., Jr., American Banker
The usefulness of statistical analysis as a fair-lending testing tool has been much debated in recent months. Out of this debate a consensus is emerging:
* Denial rates are not very useful for discrimination testing and monitoring.
* Advanced statistical techniques such as regression analysis, even using application file data, are not a testing/monitoring panacea.
* File reviews focused on individual applications may not be well suited to detecting patterns or practices of discrimination.
* The ideal process for discrimination testing and monitoring at this point in time is a combination of statistical analysis and file review.
The reason for this emerging consensus is that statistical analysis and file review techniques are complementary, with offsetting strengths and weaknesses.
Thus, a good fair-lending monitoring strategy for a financial institution is to use statistical analysis to focus its fair- lending review on the possibly troublesome files, and then to review those files to determine why they might be troublesome.
Experience indicates that this approach focuses on a reduced number of select application files. The net benefit to the institution is more effective monitoring at a lower cost.
Despite being easy to compute and readily available, almost everyone agrees that denial rates by themselves provide little insight into whether a pattern of discrimination exists. We wish the world we live in were so simple that single indicators would give us the answer. Clearly, that is not our world.
File comparisons look at accepted and denied applicants who have similar characteristics along several dimensions. The objective of the examiners in comparing the two groups of files is to see if the denied applicants are being given the same access to credit as the similarly situated approved applicants.
This process must be based on similarly situated applicants. There are two disadvantages with this approach. The first problem is that exact or close matches may not exist, and even if they do, it's not clear that the examiner will find them. The second difficulty is that once files are identified, the question is whether the files represent a pattern or practice of discrimination.
One of the advantages of the file review process is that it requires less data entry than statistical testing. However, as automated application tracking systems become the norm, this problem will evaporate.
It is also true that file reviews can be done on small numbers of files, whereas statistical testing requires more files to satisfy the assumptions of the underlying models. Perhaps the most important advantage of file reviews is that they allow the individual circumstances of the applicant to be considered.
The other broad class of tests is a specialized form of regression analysis. Like the file comparison technique, statistical tests have their own set of advantages and disadvantages.
The first advantages of statistical analysis are those associated with determining the existence of exact or close matches, finding those file matches, and deciding whether or not they constitute a pattern or practice of discrimination.
Finding clones is done by statistically weighing an applicant's characteristics. The statistical process will not only identify the clone files that may have received differential treatment, but it will also identify files which may look very different on an individual characteristic basis, but are on average very similar.
These types of offsets are very difficult for individuals doing file reviews to make; however, computers analyze compensating factors very well.
The shortcomings of the file review and statistical analysis techniques individually and the recognition of their complementary virtues has led to the emerging consensus that the statistical analysis and file review processes should be integrated. …