An Empirical study of errors in translating natural language into logic

David Barker-Plummer, Richard Cox, Robert Dale, John Etchemendy

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review


Every teacher of logic knows that the ease with which a student can translate a natural language sentence into formal logic depends, amongst other things, on just how that natural language sentence is phrased. This paper reports findings from a pilot study of a large scale corpus in the area of formal logic education, where we used a very large dataset to provide empirical evidence for specific characteristics of natural language problem statements that frequently lead to students making mistakes. We developed a rich taxonomy of the types of errors that students make, and implemented tools for automatically classifying student errors into these categories. In this paper, we focus on three specific phenomena that were prevalent in our data: Students were found (a) to have particular difficulties with distinguishing the conditional from the biconditional, (b) to be sensitive to word-order effects during translation, and (c) to be sensitive to factors associated with the naming of constants. We conclude by considering the implications of this kind of large-scale empirical study for improving an automated assessment system specifically, and logic teaching more generally.
Original languageEnglish
Title of host publicationCogsci 2008
Subtitle of host publicationproceedings of the 30th annual meeting of the cognitive science society
Place of PublicationAustin
PublisherCognitive Science Society
Number of pages6
ISBN (Print)9780976831846
Publication statusPublished - 2008
EventAnnual Conference of the Cognitive Science Society (30th : 2008) - Washington, DC
Duration: 23 Jul 200826 Jul 2008


ConferenceAnnual Conference of the Cognitive Science Society (30th : 2008)
CityWashington, DC


  • errors
  • slips
  • Proof & Logic
  • misconceptions
  • natural language
  • e-learning
  • human reasoning
  • automated assessment
  • educational data mining
  • first-order logic
  • propositional logic


Dive into the research topics of 'An Empirical study of errors in translating natural language into logic'. Together they form a unique fingerprint.

Cite this