An Evidence-Based Approach to Distractor Generation in Multiple-Choice Language Tests

Authors

  • David M. March Cambridge Assessment English
  • Darren Perrett Cambridge Assessment English
  • Christopher Hubbard Cambridge Assessment English

DOI:

https://doi.org/10.33423/jhetp.v21i10.4638

Keywords:

higher education, English second-language, language education, language assessment, multiple-choice question, item generation, automated content production

Abstract

The purpose of this project is to explore the feasibility of a new approach for producing evidence-based distractor sets. We use Common Wrong Answers (CWAs), and the associated performance data, generated by candidate responses to open gap-fill tasks, to produce distractor sets for multiple-choice gap-fill tasks based on the same texts. We then investigate whether these distractor sets are effective for use in language tests, in terms of empirical and qualitative review, and consider potential impacts on the production process for test material.

This project explores a new and innovative method of content development, and raises the possibility of a new approach to item production that can semi-autogenerate test items in shorter periods of time without affecting quality or reliability. Although the approach is specific to one task type, it is hoped that further research will expand on the applications of the approach to deliver a version that may be operationalised for use across different task types in the development of language assessments.

Downloads

Published

2021-09-30

How to Cite

March, D. M., Perrett, D., & Hubbard, C. (2021). An Evidence-Based Approach to Distractor Generation in Multiple-Choice Language Tests. Journal of Higher Education Theory and Practice, 21(10). https://doi.org/10.33423/jhetp.v21i10.4638

Issue

Section

Articles