Simulated Evidence of Computer Adaptive Test Length: Implications for High Stakes Assessment in Nigeria

Abstract

Like other African countries, high-stake testing in Nigeria has suffered significant setbacks due to the Covid-19 pandemic. Computerised Adaptive Tests (CAT) is a paradigm shift in the educational assessment that ensures accuracy in ability placements. A survey design was employed to describe the psychometric characteristics of a simulated 3-parameter logistic IRT model designs to support off-site assessments. This simulation protocol involved generating examinee and item pool data, specifying the item selection algorithm and specifying CAT administration rules for execution with SimulCAT. Findings revealed that the fixed-length test guarantees a higher testing precision with an observed systematic error less than zero, a CMAE ranging from 0.2 to 0.3 and RMSE being consistent around 0.2. Findings also revealed that the fixed-length test had a higher item exposure rate which can be handled by falling back on the item selection methods that rely less on the a-parameter. Also, item redundancy was lesser for the fixed-length test compared to the variable-length test. Conclusions are for the fixed-length test option for high-stakes assessment in Nigeria.