Accuplacer Companion in a foreign language context: an argument-based validation of both test score meaning and impact

Robert C. Johnson, A. Mehdi Riazi

Research output: Contribution to journal › Article › Research › peer-review

Abstract

Use of a single, standardised instrument to make high-stakes decisions about test takers is pervasive in higher education around the world, including English as a foreign language (EFL) contexts. Contrary to longstanding best practices, however, few test users endeavour to meaningfully validate the instrument(s) they use for their specific context and purposes. This study reports efforts to validate a standardised placement test, used in a US-accredited, higher education institution in the Pacific, to exempt, exclude, or place students within its Developmental English Program. A hybrid of two validation structures – Kane’s (1992, 1994) interpretive model and Bachman’s (2005) and Bachman and Palmer’s (2010) assessment use argument – and a broad range of types and sources of evidence were used to ensure a balanced focus on both test score interpretation and test utilisation. Outcomes establish serious doubt as to the validity of the instrument for the local context. Moreover, results provide valuable insights regarding the dangers of not evaluating the validity of an assessment for the local context, the relative strengths and weaknesses of standardised tests used for placement, and the value of argument-based validation.
LanguageEnglish
Pages31-58
Number of pages28
JournalPapers in language testing and assessment
Volume4
Issue number1
Publication statusPublished - 2015

Fingerprint

foreign language
best practice
education
utilization
interpretation
evidence
Values
student

Keywords

  • assessment
  • testing
  • language testing
  • validity
  • validation

Cite this

Johnson, Robert C. ; Riazi, A. Mehdi. / Accuplacer Companion in a foreign language context : an argument-based validation of both test score meaning and impact. In: Papers in language testing and assessment. 2015 ; Vol. 4, No. 1. pp. 31-58.
@article{d57426e698dd43fcbb3245569ea3676c,
title = "Accuplacer Companion in a foreign language context: an argument-based validation of both test score meaning and impact",
abstract = "Use of a single, standardised instrument to make high-stakes decisions about test takers is pervasive in higher education around the world, including English as a foreign language (EFL) contexts. Contrary to longstanding best practices, however, few test users endeavour to meaningfully validate the instrument(s) they use for their specific context and purposes. This study reports efforts to validate a standardised placement test, used in a US-accredited, higher education institution in the Pacific, to exempt, exclude, or place students within its Developmental English Program. A hybrid of two validation structures – Kane’s (1992, 1994) interpretive model and Bachman’s (2005) and Bachman and Palmer’s (2010) assessment use argument – and a broad range of types and sources of evidence were used to ensure a balanced focus on both test score interpretation and test utilisation. Outcomes establish serious doubt as to the validity of the instrument for the local context. Moreover, results provide valuable insights regarding the dangers of not evaluating the validity of an assessment for the local context, the relative strengths and weaknesses of standardised tests used for placement, and the value of argument-based validation.",
keywords = "assessment, testing, language testing, validity, validation",
author = "Johnson, {Robert C.} and Riazi, {A. Mehdi}",
year = "2015",
language = "English",
volume = "4",
pages = "31--58",
journal = "Papers in language testing and assessment",
issn = "2201-0009",
publisher = "Association for Language Testing and Assessment of Australia and New Zealand",
number = "1",

}

Johnson, RC & Riazi, AM 2015, 'Accuplacer Companion in a foreign language context: an argument-based validation of both test score meaning and impact', Papers in language testing and assessment, vol. 4, no. 1, pp. 31-58.

Accuplacer Companion in a foreign language context : an argument-based validation of both test score meaning and impact. / Johnson, Robert C.; Riazi, A. Mehdi.

In: Papers in language testing and assessment, Vol. 4, No. 1, 2015, p. 31-58.

Research output: Contribution to journal › Article › Research › peer-review

TY - JOUR

T1 - Accuplacer Companion in a foreign language context

T2 - Papers in language testing and assessment

AU - Johnson, Robert C.

AU - Riazi, A. Mehdi

PY - 2015

Y1 - 2015

N2 - Use of a single, standardised instrument to make high-stakes decisions about test takers is pervasive in higher education around the world, including English as a foreign language (EFL) contexts. Contrary to longstanding best practices, however, few test users endeavour to meaningfully validate the instrument(s) they use for their specific context and purposes. This study reports efforts to validate a standardised placement test, used in a US-accredited, higher education institution in the Pacific, to exempt, exclude, or place students within its Developmental English Program. A hybrid of two validation structures – Kane’s (1992, 1994) interpretive model and Bachman’s (2005) and Bachman and Palmer’s (2010) assessment use argument – and a broad range of types and sources of evidence were used to ensure a balanced focus on both test score interpretation and test utilisation. Outcomes establish serious doubt as to the validity of the instrument for the local context. Moreover, results provide valuable insights regarding the dangers of not evaluating the validity of an assessment for the local context, the relative strengths and weaknesses of standardised tests used for placement, and the value of argument-based validation.

AB - Use of a single, standardised instrument to make high-stakes decisions about test takers is pervasive in higher education around the world, including English as a foreign language (EFL) contexts. Contrary to longstanding best practices, however, few test users endeavour to meaningfully validate the instrument(s) they use for their specific context and purposes. This study reports efforts to validate a standardised placement test, used in a US-accredited, higher education institution in the Pacific, to exempt, exclude, or place students within its Developmental English Program. A hybrid of two validation structures – Kane’s (1992, 1994) interpretive model and Bachman’s (2005) and Bachman and Palmer’s (2010) assessment use argument – and a broad range of types and sources of evidence were used to ensure a balanced focus on both test score interpretation and test utilisation. Outcomes establish serious doubt as to the validity of the instrument for the local context. Moreover, results provide valuable insights regarding the dangers of not evaluating the validity of an assessment for the local context, the relative strengths and weaknesses of standardised tests used for placement, and the value of argument-based validation.

KW - assessment

KW - testing

KW - language testing

KW - validity

KW - validation

M3 - Article

VL - 4

SP - 31

EP - 58

JO - Papers in language testing and assessment

JF - Papers in language testing and assessment

SN - 2201-0009

IS - 1

ER -

Johnson RC, Riazi AM. Accuplacer Companion in a foreign language context: an argument-based validation of both test score meaning and impact. Papers in language testing and assessment. 2015;4(1):31-58.