Effectiveness of peer review in teaching and learning user centered conceptual design among large cohorts of information technology students

Conceptual design often sets the direction of the development of a product. Educating Software Engineering and Information Technology (IT) students to consider the needs of users during the conceptual design stage is important. The assessment of conceptual design is a time consuming process. When budgets are limited, financial resources are inadequately allocated to allow staff to spend quality time assessing the conceptual design artifacts of large classes. We present a methodology to teach, capture and evaluate conceptual design artifacts for large classes. In two separate studies, two groups of over 185 undergraduate IT students reviewed their peers' design artifacts using a comprehensive rubric. Their reviews were later checked by two researchers using the same rubric. It was found that not only were the student-peers' reviews close to the researchers' reviews, it was possible to give students valuable and timely feedback and scaffold them to reflect, an essential characteristic for professional competence. In addition, we found that assessment by staff was not feasible due to inadequate resources. We conclude that for large classes, conceptual design artifacts can be evaluated and valuable feedback provided in a timely manner by peers with the guidance of a comprehensive rubric.