A novel deep learning approach: stacked evolutionary auto-encoder

Yaoming Cai; Zhihua Cai; Meng Zeng; Xiaobo Liu; Jia Wu; Guangjun Wang

doi:10.1109/IJCNN.2018.8489138

A novel deep learning approach

stacked evolutionary auto-encoder

Yaoming Cai, Zhihua Cai, Meng Zeng, Xiaobo Liu, Jia Wu, Guangjun Wang

Department of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution

1 Citation (Scopus)

Abstract

Deep neural networks have been successfully applied to many data mining problems in recent works. The training of deep neural networks relies heavily upon gradient descent methods, however, which may lead to the failure of training due to the vanishing gradient (or exploding gradient) and local optima problems. In this paper, we present SEvoAE method based on using Evolutionary Multiobjective optimization (EMO) algorithm to train single layer auto-encoder, and sequentially learning deeper representation in a stacking way. SEvoAE is able to achieve accurate feature representation with good sparseness by globally simultaneously optimizing two conflicting objective functions and allows users to flexibly design objective functions and evolutionary optimizers. We compare results of the proposed method with existing architectures for seven classification problems, showing that the proposed method is able to outperform existing methods with a reduced risk of overfitting the training data.

Original language	English
Title of host publication	2018 International Joint Conference on Neural Networks (IJCNN)
Place of Publication	Piscataway, NJ
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	1-8
Number of pages	8
ISBN (Electronic)	9781509060146
ISBN (Print)	9781509060153
DOIs	https://doi.org/10.1109/IJCNN.2018.8489138
Publication status	Published - 1 Jul 2018
Event	2018 International Joint Conference on Neural Networks, IJCNN 2018 - Rio de Janeiro, Brazil Duration: 8 Jul 2018 → 13 Jul 2018

Conference

Conference	2018 International Joint Conference on Neural Networks, IJCNN 2018
Country	Brazil
City	Rio de Janeiro
Period	8/07/18 → 13/07/18

Keywords

data mining
evolutionary computation
gradient methods
learning (artificial intelligence)
neural nets
stacked Evolutionary auto-encoder
deep neural networks
data mining problems
gradient descent methods
vanishing gradient
local optima problems
SEvoAE method
single layer auto-encoder
conflicting objective functions
evolutionary optimizers
training data
evolutionary multiobjective optimization algorithm
feature representation
classification problems
deep learning approach
Training
Optimization
Decoding
Neural networks
Linear programming
Feature extraction
Stacking
Deep learning
Auto-encoder
Evolutionary Multiobjective Optimization

Access to Document

10.1109/IJCNN.2018.8489138

Link to publication in Scopus

Fingerprint Dive into the research topics of 'A novel deep learning approach: stacked evolutionary auto-encoder'. Together they form a unique fingerprint.

Cite this

@inproceedings{d623ed4558e143e69e908408e62b240f,

title = "A novel deep learning approach: stacked evolutionary auto-encoder",

abstract = "Deep neural networks have been successfully applied to many data mining problems in recent works. The training of deep neural networks relies heavily upon gradient descent methods, however, which may lead to the failure of training due to the vanishing gradient (or exploding gradient) and local optima problems. In this paper, we present SEvoAE method based on using Evolutionary Multiobjective optimization (EMO) algorithm to train single layer auto-encoder, and sequentially learning deeper representation in a stacking way. SEvoAE is able to achieve accurate feature representation with good sparseness by globally simultaneously optimizing two conflicting objective functions and allows users to flexibly design objective functions and evolutionary optimizers. We compare results of the proposed method with existing architectures for seven classification problems, showing that the proposed method is able to outperform existing methods with a reduced risk of overfitting the training data.",

keywords = "data mining, evolutionary computation, gradient methods, learning (artificial intelligence), neural nets, stacked Evolutionary auto-encoder, deep neural networks, data mining problems, gradient descent methods, vanishing gradient, local optima problems, SEvoAE method, single layer auto-encoder, conflicting objective functions, evolutionary optimizers, training data, evolutionary multiobjective optimization algorithm, feature representation, classification problems, deep learning approach, Training, Optimization, Decoding, Neural networks, Linear programming, Feature extraction, Stacking, Deep learning, Auto-encoder, Evolutionary Multiobjective Optimization",

author = "Yaoming Cai and Zhihua Cai and Meng Zeng and Xiaobo Liu and Jia Wu and Guangjun Wang",

year = "2018",

month = "7",

day = "1",

doi = "10.1109/IJCNN.2018.8489138",

language = "English",

isbn = "9781509060153",

pages = "1--8",

booktitle = "2018 International Joint Conference on Neural Networks (IJCNN)",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

address = "United States",

}

Cai, Y, Cai, Z, Zeng, M, Liu, X, Wu, J & Wang, G 2018, A novel deep learning approach: stacked evolutionary auto-encoder. in 2018 International Joint Conference on Neural Networks (IJCNN). Institute of Electrical and Electronics Engineers (IEEE), Piscataway, NJ, pp. 1-8, 2018 International Joint Conference on Neural Networks, IJCNN 2018, Rio de Janeiro, Brazil, 8/07/18. https://doi.org/10.1109/IJCNN.2018.8489138

A novel deep learning approach : stacked evolutionary auto-encoder. / Cai, Yaoming; Cai, Zhihua; Zeng, Meng; Liu, Xiaobo; Wu, Jia; Wang, Guangjun.

2018 International Joint Conference on Neural Networks (IJCNN). Piscataway, NJ : Institute of Electrical and Electronics Engineers (IEEE), 2018. p. 1-8.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution

TY - GEN

T1 - A novel deep learning approach

T2 - stacked evolutionary auto-encoder

AU - Cai, Yaoming

AU - Cai, Zhihua

AU - Zeng, Meng

AU - Liu, Xiaobo

AU - Wu, Jia

AU - Wang, Guangjun

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Deep neural networks have been successfully applied to many data mining problems in recent works. The training of deep neural networks relies heavily upon gradient descent methods, however, which may lead to the failure of training due to the vanishing gradient (or exploding gradient) and local optima problems. In this paper, we present SEvoAE method based on using Evolutionary Multiobjective optimization (EMO) algorithm to train single layer auto-encoder, and sequentially learning deeper representation in a stacking way. SEvoAE is able to achieve accurate feature representation with good sparseness by globally simultaneously optimizing two conflicting objective functions and allows users to flexibly design objective functions and evolutionary optimizers. We compare results of the proposed method with existing architectures for seven classification problems, showing that the proposed method is able to outperform existing methods with a reduced risk of overfitting the training data.

AB - Deep neural networks have been successfully applied to many data mining problems in recent works. The training of deep neural networks relies heavily upon gradient descent methods, however, which may lead to the failure of training due to the vanishing gradient (or exploding gradient) and local optima problems. In this paper, we present SEvoAE method based on using Evolutionary Multiobjective optimization (EMO) algorithm to train single layer auto-encoder, and sequentially learning deeper representation in a stacking way. SEvoAE is able to achieve accurate feature representation with good sparseness by globally simultaneously optimizing two conflicting objective functions and allows users to flexibly design objective functions and evolutionary optimizers. We compare results of the proposed method with existing architectures for seven classification problems, showing that the proposed method is able to outperform existing methods with a reduced risk of overfitting the training data.

KW - data mining

KW - evolutionary computation

KW - gradient methods

KW - learning (artificial intelligence)

KW - neural nets

KW - stacked Evolutionary auto-encoder

KW - deep neural networks

KW - data mining problems

KW - gradient descent methods

KW - vanishing gradient

KW - local optima problems

KW - SEvoAE method

KW - single layer auto-encoder

KW - conflicting objective functions

KW - evolutionary optimizers

KW - training data

KW - evolutionary multiobjective optimization algorithm

KW - feature representation

KW - classification problems

KW - deep learning approach

KW - Training

KW - Optimization

KW - Decoding

KW - Neural networks

KW - Linear programming

KW - Feature extraction

KW - Stacking

KW - Deep learning

KW - Auto-encoder

KW - Evolutionary Multiobjective Optimization

UR - http://www.scopus.com/inward/record.url?scp=85056560811&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2018.8489138

DO - 10.1109/IJCNN.2018.8489138

M3 - Conference proceeding contribution

SN - 9781509060153

SP - 1

EP - 8

BT - 2018 International Joint Conference on Neural Networks (IJCNN)

PB - Institute of Electrical and Electronics Engineers (IEEE)

CY - Piscataway, NJ

ER -