TY - JOUR
T1 - Evaluation of a task-specific checklist and global rating scale for ultrasound-guided regional anesthesia
AU - Wong, Daniel M.
AU - Watson, Mathew J.
AU - Kluger, Roman
AU - Chuan, Alwin
AU - Herrick, Michael D.
AU - Ng, Irene
AU - Castanelli, Damian J.
AU - Lin, Lisa C.
AU - Lansdown, Andrew
AU - Barrington, Michael J.
PY - 2014/8/21
Y1 - 2014/8/21
N2 - Checklists and global rating scales (GRSs) are used for assessment of anesthesia procedural skills. The purpose of this study was to evaluate the reliability and validity of a recently proposed assessment tool comprising a checklist and GRS specific for ultrasound-guided regional anesthesia. METHODS: In this prospective, fully crossed study, we videotaped 30 single-target nerve block procedures performed by anesthesia trainees. Following pilot assessment and observer training, videos were assessed in random order by 6 blinded, expert observers. Interrater reliability was evaluated with intraclass correlation coefficients (ICCs) based on a 2-way random-effects model that took into account both agreement and correlation between observer results. Construct validity and feasibility were also evaluated. RESULTS: The ICC between assessors' total scores was 0.44 (95% confidence interval, 0.27-0.62). All 6 observers scored "experienced trainees" higher than "inexperienced trainees" (median total score 76.7 vs 54.2, P = 0.01), supporting the test's construct validity. The median time to assess the videos was 4 minutes 29 seconds. CONCLUSIONS: This is the first study to evaluate the reliability and validity of a combined checklist and GRS for ultrasound-guided regional anesthesia using multiple observers and taking into account both absolute agreement and correlation in determining the ICC of 0.44 for interrater reliability. There was evidence to support construct validity.
AB - Checklists and global rating scales (GRSs) are used for assessment of anesthesia procedural skills. The purpose of this study was to evaluate the reliability and validity of a recently proposed assessment tool comprising a checklist and GRS specific for ultrasound-guided regional anesthesia. METHODS: In this prospective, fully crossed study, we videotaped 30 single-target nerve block procedures performed by anesthesia trainees. Following pilot assessment and observer training, videos were assessed in random order by 6 blinded, expert observers. Interrater reliability was evaluated with intraclass correlation coefficients (ICCs) based on a 2-way random-effects model that took into account both agreement and correlation between observer results. Construct validity and feasibility were also evaluated. RESULTS: The ICC between assessors' total scores was 0.44 (95% confidence interval, 0.27-0.62). All 6 observers scored "experienced trainees" higher than "inexperienced trainees" (median total score 76.7 vs 54.2, P = 0.01), supporting the test's construct validity. The median time to assess the videos was 4 minutes 29 seconds. CONCLUSIONS: This is the first study to evaluate the reliability and validity of a combined checklist and GRS for ultrasound-guided regional anesthesia using multiple observers and taking into account both absolute agreement and correlation in determining the ICC of 0.44 for interrater reliability. There was evidence to support construct validity.
UR - http://www.scopus.com/inward/record.url?scp=84906806302&partnerID=8YFLogxK
U2 - 10.1097/AAP.0000000000000126
DO - 10.1097/AAP.0000000000000126
M3 - Article
VL - 39
SP - 399
EP - 408
JO - Regional Anesthesia and Pain Medicine
JF - Regional Anesthesia and Pain Medicine
SN - 1098-7339
IS - 5
ER -