XOR has no local minima: A case study in neural network error surface analysis

Research output: Contribution to journalArticleResearchpeer-review

Abstract

This paper presents a case study of the analysis of local minima in feedforward neural networks. Firstly, a new methodology for analysis is presented, based upon consideration of trajectories through weight space by which a training algorithm might escape a hypothesized local minimum. This analysis method is then applied to the well known XOR (exclusive-or) problem, which has previously been considered to exhibit local minima. The analysis proves the absence of local minima, eliciting significant aspects of the structure of the error surface. The present work is important for the study of the existence of local minima in feedforward neural networks, and also for the development of training algorithms which avoid or escape entrapment in local minima.

LanguageEnglish
Pages669-681
Number of pages13
JournalNeural Networks
Volume11
Issue number4
DOIs
Publication statusPublished - Jun 1998

Fingerprint

Feedforward neural networks
Surface analysis
Neural networks
Trajectories
Weights and Measures

Keywords

  • Error surface
  • Exclusive-or
  • Feedforward nets
  • Local minimum
  • XOR

Cite this

@article{7819d9952beb4d06bba74fec1470ddd2,
title = "XOR has no local minima: A case study in neural network error surface analysis",
abstract = "This paper presents a case study of the analysis of local minima in feedforward neural networks. Firstly, a new methodology for analysis is presented, based upon consideration of trajectories through weight space by which a training algorithm might escape a hypothesized local minimum. This analysis method is then applied to the well known XOR (exclusive-or) problem, which has previously been considered to exhibit local minima. The analysis proves the absence of local minima, eliciting significant aspects of the structure of the error surface. The present work is important for the study of the existence of local minima in feedforward neural networks, and also for the development of training algorithms which avoid or escape entrapment in local minima.",
keywords = "Error surface, Exclusive-or, Feedforward nets, Local minimum, XOR",
author = "Hamey, {Leonard G C}",
year = "1998",
month = "6",
doi = "10.1016/S0893-6080(97)00134-2",
language = "English",
volume = "11",
pages = "669--681",
journal = "Neural Networks",
issn = "0893-6080",
publisher = "Elsevier",
number = "4",

}

XOR has no local minima : A case study in neural network error surface analysis. / Hamey, Leonard G C.

In: Neural Networks, Vol. 11, No. 4, 06.1998, p. 669-681.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - XOR has no local minima

T2 - Neural Networks

AU - Hamey, Leonard G C

PY - 1998/6

Y1 - 1998/6

N2 - This paper presents a case study of the analysis of local minima in feedforward neural networks. Firstly, a new methodology for analysis is presented, based upon consideration of trajectories through weight space by which a training algorithm might escape a hypothesized local minimum. This analysis method is then applied to the well known XOR (exclusive-or) problem, which has previously been considered to exhibit local minima. The analysis proves the absence of local minima, eliciting significant aspects of the structure of the error surface. The present work is important for the study of the existence of local minima in feedforward neural networks, and also for the development of training algorithms which avoid or escape entrapment in local minima.

AB - This paper presents a case study of the analysis of local minima in feedforward neural networks. Firstly, a new methodology for analysis is presented, based upon consideration of trajectories through weight space by which a training algorithm might escape a hypothesized local minimum. This analysis method is then applied to the well known XOR (exclusive-or) problem, which has previously been considered to exhibit local minima. The analysis proves the absence of local minima, eliciting significant aspects of the structure of the error surface. The present work is important for the study of the existence of local minima in feedforward neural networks, and also for the development of training algorithms which avoid or escape entrapment in local minima.

KW - Error surface

KW - Exclusive-or

KW - Feedforward nets

KW - Local minimum

KW - XOR

UR - http://www.scopus.com/inward/record.url?scp=0032100147&partnerID=8YFLogxK

U2 - 10.1016/S0893-6080(97)00134-2

DO - 10.1016/S0893-6080(97)00134-2

M3 - Article

VL - 11

SP - 669

EP - 681

JO - Neural Networks

JF - Neural Networks

SN - 0893-6080

IS - 4

ER -