A within-subject experiment of item format effects on early primary students' language, reading, and numeracy assessment results

Stuart Woodcock, Steven J. Howard, John Ehrich

    Research output: Contribution to journalArticlepeer-review

    1 Citation (Scopus)

    Abstract

    Standardized testing is ubiquitous in educational assessment, but questions have been raised about the extent to which these test scores accurately reflect students' genuine knowledge and skills. To more rigorously investigate this issue, the current study employed a within-subject experimental design to examine item format effects on primary school students' standardized assessment results in literacy, reading comprehension, and numeracy. Eighty-nine Grade 3 students (ages 8-9 years) completed tests that varied only in item format: multiple choice; open-ended; error detection and correction; explain; and, for numeracy questions, low literacy. Analyses contrasted students' performance across these conditions, as well as item response theory-derived item difficulty and ability discrimination estimates. Findings revealed that difficulty increased and accuracy decreased from multiple-choice to open-response to error-correction and explain questions. However, the most difficult item formats tended to yield the greatest discrimination across student ability levels. Despite previous findings to the contrary, low-literacy numeracy questions did not improve student performance or reduce item difficulty. Overall, findings indicated the impact of differing methods of assessment on standardized test performance and highlighted the need for careful consideration of not only the content of assessments but also their approaches to assessment.

    Original languageEnglish
    Pages (from-to)80-87
    Number of pages8
    JournalSchool Psychology
    Volume35
    Issue number1
    Early online date24 Oct 2019
    DOIs
    Publication statusPublished - Jan 2020

    Keywords

    • standardized
    • assessment
    • testing
    • literacy
    • numeracy

    Fingerprint

    Dive into the research topics of 'A within-subject experiment of item format effects on early primary students' language, reading, and numeracy assessment results'. Together they form a unique fingerprint.

    Cite this