Key concepts: Logistic Regression, Cox model
Propublica has been kind enough to publish markdown notebooks and a detailed description of their analysis of the COMPAS recidivism tool using data from Broward County, FL. This was no doubt an arduous task, due to the fact that they had to find the actual recidivism outcomes for individual people, and use that to test the "decile" scores. (Scores of risk of reoffending.) They used a Cox model to find predictive accuracy and the results were, well, dismal. We're talking AUC's of 0.60 to 0.68, or about enough to place 1,000,000,000th in the world's most depressing Kaggle competition. They also found that, "Black defendants who do not recidivate were nearly twice as likely to be classified by COMPAS as higher risk compared to their white counterparts (45 percent vs. 23 percent)". In the push for public accountability, it's going to take a Herculean effort to make sure that companies like Northpointe, (makers of COMPAS) are held accountable for their predictive accuracy. Coming soon on my Github page, I will use a random forest to see what kinds of variable importance plots come out of the Broward county dataset.
