Results on weight configurations that are not local minima in feed-forward neural networks

Leonard Hamey

Results on weight configurations that are not local minima in feed-forward neural networks

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

Abstract

Local minima in the error surfaces of feed-forward neural networks are significant because they may entrap gradient based training algorithms. Recent results have identified conditions under which local minima do not occur. The present paper considers three distinct definitions of local minimum, concluding that a new definition, called regional minimum, corresponds most closely to intuition. Using this definition, we analyse weight configurations in which a hidden node is ignored or redundant and show that these are not local minima. The practical implications of this result for gradient based learning are discussed.

Original language	English
Title of host publication	Proceedings of the Seventh Australian Conference on Neural Networks : ACNN'96, Canberra, 10-12 April 1996
Editors	P Bartlett, A Burkitt, R Williamson
Place of Publication	Canberra
Publisher	ANU
Pages	173-178
Number of pages	6
ISBN (Print)	0731524292
Publication status	Published - 1996
Event	Australian Conference on Neural Networks (7th : 1996) - Canberra, Australia Duration: 10 Apr 1996 → 12 Apr 1996

Conference

Conference	Australian Conference on Neural Networks (7th : 1996)
Country/Territory	Australia
City	Canberra
Period	10/04/96 → 12/04/96

Cite this

@inproceedings{84e1792d4c0743119d261a9630891476,

title = "Results on weight configurations that are not local minima in feed-forward neural networks",

abstract = "Local minima in the error surfaces of feed-forward neural networks are significant because they may entrap gradient based training algorithms. Recent results have identified conditions under which local minima do not occur. The present paper considers three distinct definitions of local minimum, concluding that a new definition, called regional minimum, corresponds most closely to intuition. Using this definition, we analyse weight configurations in which a hidden node is ignored or redundant and show that these are not local minima. The practical implications of this result for gradient based learning are discussed.",

author = "Leonard Hamey",

year = "1996",

language = "English",

isbn = "0731524292",

pages = "173--178",

editor = "P Bartlett and A Burkitt and R Williamson",

booktitle = "Proceedings of the Seventh Australian Conference on Neural Networks : ACNN'96, Canberra, 10-12 April 1996",

publisher = "ANU",

note = "Australian Conference on Neural Networks (7th : 1996) ; Conference date: 10-04-1996 Through 12-04-1996",

}

Hamey, L 1996, Results on weight configurations that are not local minima in feed-forward neural networks. in P Bartlett, A Burkitt & R Williamson (eds), Proceedings of the Seventh Australian Conference on Neural Networks : ACNN'96, Canberra, 10-12 April 1996. ANU, Canberra, pp. 173-178, Australian Conference on Neural Networks (7th : 1996), Canberra, Australia, 10/04/96.

Results on weight configurations that are not local minima in feed-forward neural networks. / Hamey, Leonard.
Proceedings of the Seventh Australian Conference on Neural Networks : ACNN'96, Canberra, 10-12 April 1996. ed. / P Bartlett; A Burkitt; R Williamson. Canberra: ANU, 1996. p. 173-178.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Results on weight configurations that are not local minima in feed-forward neural networks

AU - Hamey, Leonard

PY - 1996

Y1 - 1996

N2 - Local minima in the error surfaces of feed-forward neural networks are significant because they may entrap gradient based training algorithms. Recent results have identified conditions under which local minima do not occur. The present paper considers three distinct definitions of local minimum, concluding that a new definition, called regional minimum, corresponds most closely to intuition. Using this definition, we analyse weight configurations in which a hidden node is ignored or redundant and show that these are not local minima. The practical implications of this result for gradient based learning are discussed.

AB - Local minima in the error surfaces of feed-forward neural networks are significant because they may entrap gradient based training algorithms. Recent results have identified conditions under which local minima do not occur. The present paper considers three distinct definitions of local minimum, concluding that a new definition, called regional minimum, corresponds most closely to intuition. Using this definition, we analyse weight configurations in which a hidden node is ignored or redundant and show that these are not local minima. The practical implications of this result for gradient based learning are discussed.

M3 - Conference proceeding contribution

SN - 0731524292

SP - 173

EP - 178

BT - Proceedings of the Seventh Australian Conference on Neural Networks : ACNN'96, Canberra, 10-12 April 1996

A2 - Bartlett, P

A2 - Burkitt, A

A2 - Williamson, R

PB - ANU

CY - Canberra

T2 - Australian Conference on Neural Networks (7th : 1996)

Y2 - 10 April 1996 through 12 April 1996

ER -

Results on weight configurations that are not local minima in feed-forward neural networks

Abstract

Conference

Fingerprint

Cite this