Evaluating neural networks as a method for identifying students in need of assistance

Karo Castro-Wunsch, Alireza Ahadi, Andrew Petersen

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

26 Citations (Scopus)


Course instructors need to be able to identify students in need of assistance as early in the course as possible. Recent work has suggested that machine learning approaches applied to snapshots of small programming exercises may be an effective solution to this problem. However, these results have been obtained using data from a single institution, and prior work using features extracted from student code has been highly sensitive to differences in context. This work provides two contributions: first, a partial reproduction of previously published results, but in a different context, and second, an exploration of the efficacy of neural networks in solving this problem. Our findings confirm the importance of two features (the number of steps required to solve a problem and the correctness of key problems), indicate that machine learning techniques are relatively stable across contexts (both across terms in a single course and across courses), and suggest that neural network based approaches are as effective as the best Bayesian and decision tree methods. Furthermore, neural networks can be tuned to be reliably pessimistic, so they may serve a complementary role in solving the problem of identifying students who need assistance.

Original languageEnglish
Title of host publicationSIGCSE 2017
Subtitle of host publicationProceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education
Place of PublicationNew York, NY
PublisherAssociation for Computing Machinery
Number of pages6
ISBN (Electronic)9781450346986
Publication statusPublished - 2017
Externally publishedYes
EventACM SIGCSE Technical Symposium on Computer Science Education (48th : 2017) - Seattle, United States
Duration: 8 Mar 201711 Mar 2017


ConferenceACM SIGCSE Technical Symposium on Computer Science Education (48th : 2017)
Abbreviated titleSIGCSE'17
CountryUnited States


  • introductory programming
  • CS1
  • at-risk students
  • source code snapshot analysis
  • educational data mining
  • learning analytics
  • replication
  • reproduction


Dive into the research topics of 'Evaluating neural networks as a method for identifying students in need of assistance'. Together they form a unique fingerprint.

Cite this