Validation of a locally created and rated writing test used for placement in a higher education EFL program

Robert C. Johnson*, A. Mehdi Riazi

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review

    11 Citations (Scopus)

    Abstract

    This paper reports a study conducted to validate a locally created and rated writing test. The test was used to inform a higher education institution's decisions regarding placement of entering students into appropriate preparatory English program courses. An amalgam of two influential models – Kane's (1992, 1994) interpretive model and Bachman's (2005) and Bachman and Palmer's (2010) assessment use argument – was used to build a validation framework. A mixed methods approach incorporating a diverse array of quantitative and qualitative data from various stakeholders, including examinees, students, instructors, staff, and administrators, guided the collection and analysis of evidence informing the validation. Results established serious doubts about the writing test, not only in terms of interpreted score meaning, but also the impact of its use on various stakeholders, and on teaching and learning. The study reinforces the importance of comprehensive validation efforts, particularly by test users, for all instruments informing decisions about test-takers, including writing tests and other types of direct performance assessments. Results informed a number of suggested changes regarding the rubric and rater training, among others, thus demonstrating the potential of validation studies as ‘road maps’ for immediate opportunities to improve both testing and decisions made based on testing.

    Original languageEnglish
    Pages (from-to)85-104
    Number of pages20
    JournalAssessing Writing
    Volume32
    DOIs
    Publication statusPublished - Apr 2017

    Keywords

    • language testing
    • placement testing
    • test validity
    • argument-based validity

    Fingerprint

    Dive into the research topics of 'Validation of a locally created and rated writing test used for placement in a higher education EFL program'. Together they form a unique fingerprint.

    Cite this