Assessing the assessments

development of a tool to evaluate assessment items in chemistry according to learning outcomes

Siegbert Schmid, Madeleine Schultz, Samuel J. Priest, Glennys O'Brien, Simon M. Pyke, Adam Bridgeman, Kieran F. Lim, Daniel C. Southam, Simon B. Bedford, Ian M. Jamie

Research output: Chapter in Book/Report/Conference proceedingChapter

1 Citation (Scopus)

Abstract

Higher education in Australia is in a phase of rapid change due to significant regulatory changes, with new standards currently being implemented for registration of institutions and accreditation of degrees. Over the past five years the Australian chemistry community has come to a consensus on common Chemistry Threshold Learning Outcomes (CTLOs) that every Bachelor level chemistry graduate from an Australian university will have attained. The CTLOs will inform the standards used to accredit institutions and degrees. Building upon this, the Royal Australian Chemical Institute (RACI), the professional body for chemists in Australia, has changed its accreditation process for chemistry degree programs and now uses these CTLOs as the basis for accreditation. Therefore, it is paramount to ensure that assessment items used allow students to demonstrate attainment of the CTLOs for a chemistry major. The "Assessing the Assessments" project has used an iterative process to develop an evaluation framework to assist academic staff at tertiary institutions to determine the alignment of their assessment items with the CTLOs. In conjunction with professional development workshops in which colleagues explore the alignment of assessment items with the CTLOs, a sophisticated tool has been developed which can be used to evaluate assessment items. The tool yields ratings for both engagement with and assessment of each CTLO within the assessment task evaluated, highlighting areas of potential improvement in current assessment practices. Comparison of self-evaluations of tasks submitted to the project by academic staff with evaluations conducted by the project team shows that in the majority of cases, faculty over-estimate the ability of their assessment items to confirm achievement of CTLOs. Recommendations to increase the coverage of CTLOs through changes to assessment procedures are presented. Through the development of the framework, difficulties with interpretation and application of some of the CTLOs have been elucidated.

Original languageEnglish
Title of host publicationTechnology and assessment strategies for improving student learning in chemistry
EditorsMadeleine Schultz, Siegbert Schmid, Thomas Holme
Place of PublicationWashington, DC
PublisherAmerican Chemical Society
Pages225-244
Number of pages20
Volume1235
ISBN (Electronic)9780841231801
ISBN (Print)9780841231818
DOIs
Publication statusPublished - 2016

Publication series

NameACS Symposium Series
Volume1235
ISSN (Print)00976156
ISSN (Electronic)19475918

    Fingerprint

Cite this

Schmid, S., Schultz, M., Priest, S. J., O'Brien, G., Pyke, S. M., Bridgeman, A., ... Jamie, I. M. (2016). Assessing the assessments: development of a tool to evaluate assessment items in chemistry according to learning outcomes. In M. Schultz, S. Schmid, & T. Holme (Eds.), Technology and assessment strategies for improving student learning in chemistry (Vol. 1235, pp. 225-244). (ACS Symposium Series; Vol. 1235). Washington, DC: American Chemical Society. https://doi.org/10.1021/bk-2016-1235.ch013