Guided image-to-image translation by discriminator-generator communication

Yuanjiang Cao*, Lina Yao, Le Pan, Quan Z. Sheng, Xiaojun Chang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

The goal of Image-to-image (I2I) translation is to transfer an image from a source domain to a target domain, which has recently drawn increasing attention. One major branch of this research is to formulate I2I translation based on Generative Adversarial Network (GAN). As a zero-sum game, GAN can be reformulated as a Partially-observed Markov Decision Process (POMDP) for generators, where generators cannot access full state information of their environments. This formulation illustrates the information insufficiency in the GAN training. To mitigate this problem, we propose to add a communication channel between discriminators and generators. We explore multiple architecture designs to integrate the communication mechanism into the I2I translation framework. To validate the performance of the proposed approach, we have conducted extensive experiments on various benchmark datasets. The experimental results confirm the superiority of our proposed method.

Original languageEnglish
Pages (from-to)1528-1538
Number of pages11
JournalIEEE Transactions on Multimedia
Volume26
Early online date3 Jul 2023
DOIs
Publication statusPublished - 2024

Fingerprint

Dive into the research topics of 'Guided image-to-image translation by discriminator-generator communication'. Together they form a unique fingerprint.

Cite this