International Journal for the Scholarship of Teaching and Learning
Volume 1, Number 2, 2007
Return to current issues pageResearch Article
Abstract
The Impact of the “all-of-the-above” Option and Student Ability on Multiple Choice Tests
Despite the prevalence of multiple choice items in educational testing, there is a dearth of empirical evidence for multiple choice item writing rules. The purpose of this study was to expand the base of empirical evidence by examining the use of the “all-of-the-above” option in a multiple choice examination in order to assess how different student ability groups would respond to this particular alternative. Ten experimentally manipulated items were generated with “all-of-the-above” as one of the options and were incorporated into three different test formats. Test formats were randomly distributed to university students in the study. The test scores in these test formats were compared as well as the experimentally manipulated items. Results showed that when “all-of-the-above” is used as the correct answer, the item is more difficult for all students, despite the literature assumption that it provides a cueing effect to students. Research findings corroborate literature assumptions that high ability students score significantly higher than other ability students in this type of option.
View Full ArticleBios
Yi-Min Huang
University of Washington
Seattle, Washington, USA
chym@engr.washington.edu
I am a Research Scientist in the Center for Engineering Learning and Teaching at the University of Washington as well as at the National Science Foundation funded Center for the Advancement of Engineering Education. I am also a research affiliate with the Laboratory for User-Centered Engineering Education in the Technical Communication department. My research interests included program evaluation, educational assessment, and research methodology, particularly understanding teaching and learning issues. I have taught Science, Technology, Engineering and Mathematics (STEM) students about teaching portfolio constructions within the Engineering Teaching Portfolio Program (ETPP) and run numerous research groups using qualitative research methodology. In addition to teaching, I have recently conducted research on Studies of Engineering Educator Decisions (SEED) to understand how engineering educators make decisions about their teaching practices.
Michael Trevisan
Washington State University
Pullman, Washington, USA
Trevisan@mail.wsu.edu
I am a Professor in Educational Psychology and the Director of the Assessment and Evaluation Center at Washington State University. I also coordinate the graduate program in assessment and evaluation. My research work is focused on educational measurement, particularly the improvement of multiple-choice items used in achievement tests. This work has helped to build empirical evidence for item writing rules. In addition, I’ve spent a good deal of effort helping professionals in K-12 and higher education build evaluation capacity to productively incorporate program evaluation into their professional practice. In support of this work I currently have approximately $1 million in external funding from the National Science Foundation and the State of Washington.
Andrew Storfer
Washington State University
Pullman, Washington, USA
astorfer@wsu.edu
I am an Associate Professor in the School of Biological Sciences at Washington State University. I teach introductory, non-majors biology to a class of over 1000 students and recently won the 2007 Tom Lutz memorial teaching award for my efforts in this course. Teaching this course is challenging and has increasingly gotten me interested in the science of pedagogy. In addition to teaching, I am conducting research at the interface of two “grand challenges in environmental biology for the 21st Century:” understanding the causes and consequences of the Earth’s diminishing biodiversity and understanding ecology and evolution of infectious diseases. I currently hold grants from the National Science Foundation, the US Geological Survey and the State of Washington to support this work.
Storfer Lab Webpage