Evaluating classifier performance

An article in the excellent blog by Will Dwinnel, discusses evaluating classifier performance using Confusion Matrices, ROC, Lift charts and AUROC. Links to external resources are provided:

• AUC: a Statistically Consistent and more Discriminating Measure than Accuracy, by Charles X. Ling, Jin Huang and Harry Zhang
• Evaluating Performance, from “ROC Graphs: Notes and Practical Considerations for Researchers, by T. Fawcett
• The Use of the Area Under the ROC Curve in the Evaluation of Machine Learning Algorithms, by Andrew P. Bradley

In the blog entry "Model Performance Measurement", Matlab code is provided for various performance measurement routines:

• 'L-1' (mean absolute error)
• 'L-2' (mean squared error)
• 'L-4'
• 'L-16'
• 'L-Infinity'
• 'RMS' (root mean squared error)
• 'AUC' (requires tiedrank() from Statistics Toolbox)
• 'Bias'
• 'Conditional Entropy'
• 'Cross-Entropy'
• 'F-Measure'
• 'Informational Loss'
• 'MAPE'
• 'Median Squared Error'
• 'Worst 10%'
• 'Worst 20%'