Effects of graphing conventions and response options on interpretation of small n graphs

The present study examined the effects of manipulation of two graphing conventions on judgements of time-series data by novice raters. These conventions involved the presence of phase change lines between baseline and intervention data and whether data points across phase changes were connected. The 1990 study of Matyas and Greenwood was also partially replicated and participant error rates were examined when responses were on a non-binary scale (no effect, uncertain, clear effect) in contrast to the binary scale used in the original study. Thirty postgraduate special education students rated intervention effects on 36 graphs. There was no substantive evidence that graphing conventions affected judgements. Type I errors, defined as a response of 'clear effect' when no intervention effect was present, were very low (0-7%) and Type II errors were correspondingly high (0-100%), particularly with low intervention effects and high random error. Thus, judges were very conservative when using a non-binary response scale, in contrast with the results of Matyas and Greenwood. Several directions for further research are proposed.