Towards effective learning for face super-resolution with shape and pose perturbations

Xiyuan Hu, Zhenfeng Fan, Xu Jia, Zhihui Li, Xuyun Zhang, Lianyong Qi, Zuxing Xuan*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Recent development of convolutional neural networks (CNNs) has activated a lot of studies and boosted the performance greatly on image super-resolution. This paper addresses the issue of face super-resolution, which attracts a lot of interests in the photographic industry. We propose to make use of the face-specific priors to enhance the performance of face super-resolution with the convolutional neural networks. Classical facial prior models represent the 2D facial shape in a compact low-dimensional space expressed by principal components. Here, we impose perturbations on the low dimensional space and generate face samples with novel appearance. First, we conduct 2D facial image perturbations through 2D facial landmarks. Then, we carry on the study with perturbations on 3D facial landmarks. Facial pose and shape are perturbated to generate novel appearances of a single 2D facial image. These novel facial samples are then fed into the training process of the convolutional neural networks for face super-resolution. The experimental results demonstrate that the proposed method is adaptable to various networks and achieves superior performance for the face super-resolution task.

Original languageEnglish
Article number106938
Pages (from-to)1-10
Number of pages10
JournalKnowledge-Based Systems
Volume220
DOIs
Publication statusPublished - 23 May 2021

Keywords

  • Convolutional neural networks
  • Face super-resolution
  • Facial landmarks
  • 3D face model

Cite this