How is the use of technology in education evaluated? A systematic review

Research output: Contribution to journalArticleResearchpeer-review

Abstract

There are a large variety of methodologies, contexts and perspectives that have been used to evaluate the use of technology in education. The vast array of literature involving learning technology evaluation makes it challenging to acquire an accurate sense of the different aspects of learning that are evaluated, and the possible approaches that can be used to evaluate them. This study conducted a systematic review of how technology use in education has been evaluated, based on 365 papers published in Computers and Education between 2015 and 2017. The papers analyzed encompassed a diverse range of education levels, disciplines and technologies, that were broadly commensurate with characterizations of the learning technology field from other studies. The analysis found that the evaluation of learning technology use tended to focus on eight themes: learning outcomes, affective elements, behaviors, design, technology elements, pedagogy, presence, and institutional environment. In the majority of studies (66.6%) at least one established instrument was used to evaluate the effectiveness of learning technology usage, however, a wide variety of instruments were observed within sub-themes of evaluation. The 22 instruments used to evaluate an aspect of learning technology usage in more than one study are identified. Evaluation trends for different disciplines, technologies and educational levels are also established. The findings provide an analytical framework that educators and researchers can utilize when evaluating the use of technology in education, and could potentially inform the development of new, more holistic and more robust evaluation methods.

LanguageEnglish
Pages27-42
Number of pages16
JournalComputers and Education
Volume133
DOIs
Publication statusPublished - May 2019

Fingerprint

Education
education
learning
evaluation
technology assessment
level of education
educator
methodology
trend

Keywords

  • evaluation methodologies
  • interactive learning environments
  • applications in subject areas

Cite this

@article{e39417ddd8f142a8927d0a7ed00d280b,
title = "How is the use of technology in education evaluated? A systematic review",
abstract = "There are a large variety of methodologies, contexts and perspectives that have been used to evaluate the use of technology in education. The vast array of literature involving learning technology evaluation makes it challenging to acquire an accurate sense of the different aspects of learning that are evaluated, and the possible approaches that can be used to evaluate them. This study conducted a systematic review of how technology use in education has been evaluated, based on 365 papers published in Computers and Education between 2015 and 2017. The papers analyzed encompassed a diverse range of education levels, disciplines and technologies, that were broadly commensurate with characterizations of the learning technology field from other studies. The analysis found that the evaluation of learning technology use tended to focus on eight themes: learning outcomes, affective elements, behaviors, design, technology elements, pedagogy, presence, and institutional environment. In the majority of studies (66.6{\%}) at least one established instrument was used to evaluate the effectiveness of learning technology usage, however, a wide variety of instruments were observed within sub-themes of evaluation. The 22 instruments used to evaluate an aspect of learning technology usage in more than one study are identified. Evaluation trends for different disciplines, technologies and educational levels are also established. The findings provide an analytical framework that educators and researchers can utilize when evaluating the use of technology in education, and could potentially inform the development of new, more holistic and more robust evaluation methods.",
keywords = "evaluation methodologies, interactive learning environments, applications in subject areas",
author = "Lai, {Jennifer W. M.} and Matt Bower",
year = "2019",
month = "5",
doi = "10.1016/j.compedu.2019.01.010",
language = "English",
volume = "133",
pages = "27--42",
journal = "Computers and Education",
issn = "0360-1315",
publisher = "Elsevier",

}

How is the use of technology in education evaluated? A systematic review. / Lai, Jennifer W. M.; Bower, Matt.

In: Computers and Education, Vol. 133, 05.2019, p. 27-42.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - How is the use of technology in education evaluated? A systematic review

AU - Lai,Jennifer W. M.

AU - Bower,Matt

PY - 2019/5

Y1 - 2019/5

N2 - There are a large variety of methodologies, contexts and perspectives that have been used to evaluate the use of technology in education. The vast array of literature involving learning technology evaluation makes it challenging to acquire an accurate sense of the different aspects of learning that are evaluated, and the possible approaches that can be used to evaluate them. This study conducted a systematic review of how technology use in education has been evaluated, based on 365 papers published in Computers and Education between 2015 and 2017. The papers analyzed encompassed a diverse range of education levels, disciplines and technologies, that were broadly commensurate with characterizations of the learning technology field from other studies. The analysis found that the evaluation of learning technology use tended to focus on eight themes: learning outcomes, affective elements, behaviors, design, technology elements, pedagogy, presence, and institutional environment. In the majority of studies (66.6%) at least one established instrument was used to evaluate the effectiveness of learning technology usage, however, a wide variety of instruments were observed within sub-themes of evaluation. The 22 instruments used to evaluate an aspect of learning technology usage in more than one study are identified. Evaluation trends for different disciplines, technologies and educational levels are also established. The findings provide an analytical framework that educators and researchers can utilize when evaluating the use of technology in education, and could potentially inform the development of new, more holistic and more robust evaluation methods.

AB - There are a large variety of methodologies, contexts and perspectives that have been used to evaluate the use of technology in education. The vast array of literature involving learning technology evaluation makes it challenging to acquire an accurate sense of the different aspects of learning that are evaluated, and the possible approaches that can be used to evaluate them. This study conducted a systematic review of how technology use in education has been evaluated, based on 365 papers published in Computers and Education between 2015 and 2017. The papers analyzed encompassed a diverse range of education levels, disciplines and technologies, that were broadly commensurate with characterizations of the learning technology field from other studies. The analysis found that the evaluation of learning technology use tended to focus on eight themes: learning outcomes, affective elements, behaviors, design, technology elements, pedagogy, presence, and institutional environment. In the majority of studies (66.6%) at least one established instrument was used to evaluate the effectiveness of learning technology usage, however, a wide variety of instruments were observed within sub-themes of evaluation. The 22 instruments used to evaluate an aspect of learning technology usage in more than one study are identified. Evaluation trends for different disciplines, technologies and educational levels are also established. The findings provide an analytical framework that educators and researchers can utilize when evaluating the use of technology in education, and could potentially inform the development of new, more holistic and more robust evaluation methods.

KW - evaluation methodologies

KW - interactive learning environments

KW - applications in subject areas

UR - http://www.scopus.com/inward/record.url?scp=85060589011&partnerID=8YFLogxK

U2 - 10.1016/j.compedu.2019.01.010

DO - 10.1016/j.compedu.2019.01.010

M3 - Article

VL - 133

SP - 27

EP - 42

JO - Computers and Education

T2 - Computers and Education

JF - Computers and Education

SN - 0360-1315

ER -