Confidence interval estimation for quality factors of binary classifiers - ROC curves, AUC for small samples
Authors: Gus’kov S.Yu., Lyovin V.V.
Published in issue: #3(39)/2015
DOI: 10.18698/2308-6033-2015-3-1376
Category: Mechanical Engineering and Machine Science | Chapter: Inspection and Diagnostics in Mechanical Engineering
Polynomial distribution being presented as conditional joint distribution of independent Poisson random variables we build confidence intervals for sum polygons based on grouped data. We then use these estimates to build confidence intervals for ROC curves. These estimations then could be used in automatic defect detection and quality control procedures to find and to identify inhomogeneities and anomalies in structure of constructional materials and their elements for the end to improve robustness and efficiency of these procedures for small samples.
References
[1] Engelmann B., Hayden E., Tasche D. Testing rating accuracy. RISK, 2003, vol. l6, pp. 82-86.
[2] Stein R.M. Benchmarking default prediction models pitfalls and remedies in model validation. J. of Risk Model Validation, 2007, vol.l, no. l, pp. 77-ll3.
[3] Bol’shev L.N. Teoriya veroyatnostey i ee primenenie - Theory of Probability and its Applications, l962, vol. 7, pp. 353-355.
[4] Bol’shev L.N. Teoriya veroyatnostey i ee primenenie - Theory of Probability and its Applications, l965, vol. 7, pp. 356-358.
[5] Garwood F. Fiducial limits for Poisson distribution. Biometrica, l936, vol. 28, pp. 437-442.
[6] Stevens W.L. Fiducial limits of the parameter of discontinuous distribution. Biometrica, l950, vol. 37, pp. ll7-l29.
[7] Sofus A. Macskassy and Foster Provost, Confidence Bands for ROC Curves: Methods and an Empirical Study. CeDER Working Paper 02-04. Stern School of Business, New York University, Jan. 2004, l5 p.
[8] Jokiel-Rokita A., Pulit M. Nonparametric estimation of the ROC curve based on smoothed empirical distribution functions. Statistical Computing, 20l3, vol. 23, pp. 703-7l2.
[9] Baklizi A. A Simple Method for Finding Emperical Liklihood Type Intervals for the ROC Curve. J. of Modern Applied Statistical Methods, 2007, vol. 6, no. 2, pp. 589-595.
[10] Le Meur Y., Vignolle J.-M., Chanussot J. Practical use of receiver operating characteristic analysis to assess the performances of defect detection algorithms. J. of Electronic Imaging, Society of Photo-optical Instrumentation Engineers (SPIE), 2008, vol. l7, no. 3, pp.l0.lll7.
[11] Dobrzanski L.A., Krupinski M., Sokolowski J.H. Methodology of automatic quality control of aluminium castings. J. of Achievements in Materials and Manufacturing Engineering, 2007, vol. 20, no. l-2, pp. 69-78.
[12] Shchegolev S.S., Motkov A.V. The complex automated method of detection of defects of rings of bearings. On-line journal "Naukovedenie", July-August 2014, issue 4 (23). Available at: http://naukovedenie.ru/PDF/110TVN414.pdf (accessed 25 November 2014).
[13] Ivanov V.I., Konovalov N.N., Dergachev A.N. Tekhnologii tekhnosfernoi bezopasnosti - Technospheric safety technology, 2014, issue 6 (58). Available at: http://agps-2006.narod.ru/ttb/2014-6/15-06-14.ttb.pdf (accessed 25 November 2014).