Improving the validity of self-ratings of abilities: the impact of rating format and order

Previous research has found self-ratings of abilities to have low validity. Aiming to identify ways to improve self-ratings, this study investigated the impact of rating format (single-item global ratings versus multiple-item specific ratings) and order (global ratings before specific ratings versus global ratings after specific ratings) on the validity of self-rated abilities. A self-rating measure of communication skills was developed for this study for data collection from 178 medical students. The measure consisted of a global item assessing communication skills in general and eleven specific items, with each evaluating a particular dimension of communication skills. Students' self-ratings were correlated with the ratings from two groups of raters, namely communication tutors and clinical tutors, to establish validity. Results showed that students tended to provide more valid self-ratings on the specific items measuring different dimensions of communication skills than on the global item. We also found that the validity of students' self-ratings on the global item was improved when students rated themselves on the specific items prior to the global item. Theoretical and practical implications of the findings are discussed, and future research directions are suggested.