Trading spaces

Computation, representation, and the limits of uninformed learning

Andy Clark*, Chris Thornton

*Corresponding author for this work

Research output: Contribution to journalShort survey

106 Citations (Scopus)

Abstract

Some regularities enjoy only an attenuated existence in a body of training data. These are regularities whose statistical visibility depends on some systematic recoding of the data. The spate of possible recodings is, however, infinitely large - it is tile space of applicable Turing machines. As a result, mappings that pivot on such attenuated regularities cannot, in general, be found by brute-force search. The class of problems that present such mappings we call the class of 'type-2 problems'. 'Type-1 problems, by contrast, present tractable problems of search insofar as the relevant regularities can be found by sampling the input data as originally coded. Type-2 problems, we suggest, present neither rare nor pathological cases. They are rife in biologically realistic settings and in domains ranging from simple animat (simulated animal or autonomous robot) behaviors to language acquisition. Not only are such. problems rife - they are standardly solved! This presents a puzzle. How, given the statistical intractability of these type-2 cases, does nature turn the trick? One answer, which we do not pursue, is to suppose that evolution gifts us with exactly the right set of recoding biases so as to reduce specific type-2 problems to (tractable) type-1 mappings. Such a heavy-duty nativism is no doubt sometimes plausible. But we believe there are other, more general mechanisms also at work. Such mechanisms provide general (not task-specific) strategies for managing problems of type-2 complexity. Several such mechanisms are investigated. At the heart of each is a fundamental ploy - namely, the maximal exploitation of states of representation already achieved by prior, simpler (type-1) learning so as to reduce the amount of subsequent computational search. Such exploitation both characterizes and helps make unitary sense of a diverse range of mechanisms. These include simple incremental learning (Elman 1993), modular connectionism (Jacobs et al. 1991), and the developmental hypothesis of 'representational redescription' (Karmiloff-Smith 1979; 1992. In addition, the most distinctive features of human cognition - language and culture - may themselves be viewed as adaptations enabling this representation/computation trade-off to be pursued on an even grander scale.

Original languageEnglish
Pages (from-to)57-66
Number of pages10
JournalBehavioral and Brain Sciences
Volume20
Issue number1
DOIs
Publication statusPublished - Mar 1997
Externally publishedYes

Keywords

  • Connectionism
  • Learning
  • Representation
  • Search
  • Statistics

Fingerprint Dive into the research topics of 'Trading spaces: Computation, representation, and the limits of uninformed learning'. Together they form a unique fingerprint.

  • Cite this