Local minima in the error surfaces of feed-forward neural networks are significant because they may entrap gradient based training algorithms. Recent results have identified conditions under which local minima do not occur. The present paper considers three distinct definitions of local minimum, concluding that a new definition, called regional minimum, corresponds most closely to intuition. Using this definition, we analyse weight configurations in which a hidden node is ignored or redundant and show that these are not local minima. The practical implications of this result for gradient based learning are discussed.
|Title of host publication||Proceedings of the Seventh Australian Conference on Neural Networks : ACNN'96, Canberra, 10-12 April 1996|
|Editors||P Bartlett, A Burkitt, R Williamson|
|Place of Publication||Canberra|
|Number of pages||6|
|Publication status||Published - 1996|
|Event||Australian Conference on Neural Networks (7th : 1996) - Canberra, Australia|
Duration: 10 Apr 1996 → 12 Apr 1996
|Conference||Australian Conference on Neural Networks (7th : 1996)|
|Period||10/04/96 → 12/04/96|
Hamey, L. (1996). Results on weight configurations that are not local minima in feed-forward neural networks. In P. Bartlett, A. Burkitt, & R. Williamson (Eds.), Proceedings of the Seventh Australian Conference on Neural Networks : ACNN'96, Canberra, 10-12 April 1996 (pp. 173-178). Canberra: ANU.