The Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) has been suggested by researchers as a diagnostic tool for assessment and evaluation. While model-based CD-CAT is relatively well-researched in the context of large-scale assessments, this type of system has not received the same degree of development in small-scale settings, where it would be most useful. The main challenge is that the statistical estimation techniques successfully applied to the parametric CD-CAT require large samples to guarantee the reliable calibration of item parameters and accurate assignments of examinees. In response to the challenge, a nonparametric approach that does not require any parameter calibration, and thus can be used in small educational programs, is proposed. Unlike other CD-CAT algorithms, the proposed nonparametric CD-CAT uses the nonparametric classification (NPC) method to assess and update the student's ability state while the test proceeds. Based on a student's responses, possible proficiency classes are identified, and items which can discriminate them are chosen next. The simulation results show that the proposed nonparametric item selection (NPS) method outperformed the compared parametric CD-CAT algorithms and the differences were more significant when the item parameter calibration was not optimal.

The Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) has been suggested by researchers as a diagnostic tool for assessment and evaluation. While model-based CD-CAT is relatively well-researched in the context of large-scale assessments, this type of system has not received the same degree of development in small-scale settings, where it would be most useful. The main challenge is that the statistical estimation techniques successfully applied to the parametric CD-CAT require large samples to guarantee the reliable calibration of item parameters and accurate assignments of examinees. In response to the challenge, a nonparametric approach that does not require any parameter calibration, and thus can be used in small educational programs, is proposed. Unlike other CD-CAT algorithms, the proposed nonparametric CD-CAT uses the nonparametric classification (NPC) method to assess and update the student's ability state while the test proceeds. Based on a student's responses, possible proficiency classes are identified, and items which can discriminate them are chosen next. The simulation results show that the proposed nonparametric item selection (NPS) method outperformed the compared parametric CD-CAT algorithms and the differences were more significant when the item parameter calibration was not optimal.

[1] Chang, H.-H. (2015). Psychometrics behind computerized adaptive testing. Psychometrika, 80, 1-20.
[2] Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110, 850-866.
[3] Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive test- ing: CD-CAT. Psychometrika, 74, 619- 632.
[4] Chiu, C.-Y., & Douglas, J. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30, 225-250.
[5] Chiu, C.-Y. (2013). Statistical refinement of the Q-matrix in cognitive diagnosis. Applied Psychological Measurement, 37, 598-618.
[6] de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179-199.
[7] Hartz, S. M., Roussos, L. A., Henson, R. A., & Templin, J. L. (2005). The fusion model for skill diagnosis: Blending theory with practicality. Unpublished manuscript.
[8] Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191-210.
[9] Junker, B. W. & Sijtsma, K. (2001). Cognitive assessment models with few as- sumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.
[10] Kaplan, M., de la Torre, J., & Barrada, J. R. (2015). New item selection methods for cognitive diagnosis computerized adaptive testing. Applied Psychological Measurement, 39, 167-188.
[11] Kohn, H.-F., & Chiu, C.-Y. (2016). A proof of the duality of the DINA model and the DINO model. Journal of Classification, 33, 171-184.
[12] Liu, J., Xu, G., & Ying, Z. (2013). Theory of self-learning Q-matrix. Bernoulli, 19, 1790-1817.
[13] Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187-212.
[14] Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379-423.
[15] Templin, J. L. & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287-305.
[16] Wang, S., & Douglas, J. (2015). Consistency of nonparametric classification in cognitive diagnosis. Psychometrika, 80, 85-100.
[17] Xu, G., Wang, C., & Shang, Z. (2016). On initial item selection in cognitive diagnostic computerized adaptive testing. British Journal of Mathematical and Statistical Psychology, 69, 291- 315.