Using edit distance to analyse errors in a natural language to logic translation corpus

Dave Barker-Plummer, Robert Dale, Richard Cox

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

8 Citations (Scopus)


We have assembled a large corpus of student submissions to an automatic grading system, where the subject matter involves the translation of natural language sentences into propositional logic. Of the 2.3 million translation instances in the corpus, 286,000 (approximately 12%) are categorized as being in error. We want to understand the nature of the errors that students make, so that we can develop tools and supporting infrastructure that help students with the problems that these errors represent. With this aim in mind, this paper describes an analysis of a significant proportion of the data, using edit distance between incorrect answers and their corresponding correct solutions, and the associated edit sequences, as a means of organising the data and detecting categories of errors. We demonstrate that a large proportion of errors can be accounted for by means of a small number of relatively simple error types, and that the method draws attention to interesting phenomena in the data set.
Original languageEnglish
Title of host publicationEDM 2012
Subtitle of host publicationproceedings of the 5th International Conference on Educational Data Mining : Chania, Greece, June 19-21 2012
EditorsKalina Yacef, Osmar Zaïane, Arnon Hershkovitz, Michael Yudelson, John Stamper
PublisherInternational Educational Data Mining Society
Number of pages8
ISBN (Print)9781742102764
Publication statusPublished - 2012
EventInternational Conference on Educational Data Mining (5th : 2012) - Chania, Greece
Duration: 19 Jun 201221 Jun 2012


ConferenceInternational Conference on Educational Data Mining (5th : 2012)
CityChania, Greece


Dive into the research topics of 'Using edit distance to analyse errors in a natural language to logic translation corpus'. Together they form a unique fingerprint.

Cite this