DIFAI: DIVERSE FACIAL INPAINTING USING STYLEGAN INVERSION

Dongsik Yoon; Jeong Gi Kwak; Yuanming Li; David Han; Hanseok Ko

doi:10.1109/ICIP46576.2022.9898012

DIFAI: DIVERSE FACIAL INPAINTING USING STYLEGAN INVERSION

Dongsik Yoon, Jeong Gi Kwak, Yuanming Li, David Han, Hanseok Ko

School of Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

Image inpainting is an old problem in computer vision that restores occluded regions and completes damaged images. In the case of facial image inpainting, most of the methods generate only one result for each masked image, even though there are other reasonable possibilities. To prevent any potential biases and unnatural constraints stemming from generating only one image, we propose a novel framework for diverse facial inpainting exploiting the embedding space of StyleGAN. Our framework employs pSp encoder and SeFa algorithm to identify semantic components of the StyleGAN embeddings and feed them into our proposed SPARN decoder that adopts region normalization for plausible inpainting. We demonstrate that our proposed method outperforms several state-of-the-art methods.

Original language	English
Title of host publication	2022 IEEE International Conference on Image Processing, ICIP 2022 - Proceedings
Publisher	IEEE Computer Society
Pages	1141-1145
Number of pages	5
ISBN (Electronic)	9781665496209
DOIs	https://doi.org/10.1109/ICIP46576.2022.9898012
Publication status	Published - 2022
Event	29th IEEE International Conference on Image Processing, ICIP 2022 - Bordeaux, France Duration: 2022 Oct 16 → 2022 Oct 19

Publication series

Name	Proceedings - International Conference on Image Processing, ICIP
ISSN (Print)	1522-4880

Conference

Conference	29th IEEE International Conference on Image Processing, ICIP 2022
Country/Territory	France
City	Bordeaux
Period	22/10/16 → 22/10/19

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

Facial Image Inpainting
Pluralistic Image Inpainting
StyleGAN Inversion

ASJC Scopus subject areas

Software
Computer Vision and Pattern Recognition
Signal Processing

Access to Document

10.1109/ICIP46576.2022.9898012

Cite this

DIFAI: DIVERSE FACIAL INPAINTING USING STYLEGAN INVERSION. / Yoon, Dongsik; Kwak, Jeong Gi; Li, Yuanming et al.
2022 IEEE International Conference on Image Processing, ICIP 2022 - Proceedings. IEEE Computer Society, 2022. p. 1141-1145 (Proceedings - International Conference on Image Processing, ICIP).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Yoon, D, Kwak, JG, Li, Y, Han, D & Ko, H 2022, DIFAI: DIVERSE FACIAL INPAINTING USING STYLEGAN INVERSION. in 2022 IEEE International Conference on Image Processing, ICIP 2022 - Proceedings. Proceedings - International Conference on Image Processing, ICIP, IEEE Computer Society, pp. 1141-1145, 29th IEEE International Conference on Image Processing, ICIP 2022, Bordeaux, France, 22/10/16. https://doi.org/10.1109/ICIP46576.2022.9898012

@inproceedings{ebf0cebd6c7c4394b1a8ef622e558cdb,

title = "DIFAI: DIVERSE FACIAL INPAINTING USING STYLEGAN INVERSION",

abstract = "Image inpainting is an old problem in computer vision that restores occluded regions and completes damaged images. In the case of facial image inpainting, most of the methods generate only one result for each masked image, even though there are other reasonable possibilities. To prevent any potential biases and unnatural constraints stemming from generating only one image, we propose a novel framework for diverse facial inpainting exploiting the embedding space of StyleGAN. Our framework employs pSp encoder and SeFa algorithm to identify semantic components of the StyleGAN embeddings and feed them into our proposed SPARN decoder that adopts region normalization for plausible inpainting. We demonstrate that our proposed method outperforms several state-of-the-art methods.",

keywords = "Facial Image Inpainting, Pluralistic Image Inpainting, StyleGAN Inversion",

author = "Dongsik Yoon and Kwak, {Jeong Gi} and Yuanming Li and David Han and Hanseok Ko",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 29th IEEE International Conference on Image Processing, ICIP 2022 ; Conference date: 16-10-2022 Through 19-10-2022",

year = "2022",

doi = "10.1109/ICIP46576.2022.9898012",

language = "English",

series = "Proceedings - International Conference on Image Processing, ICIP",

publisher = "IEEE Computer Society",

pages = "1141--1145",

booktitle = "2022 IEEE International Conference on Image Processing, ICIP 2022 - Proceedings",

}

TY - GEN

T1 - DIFAI

T2 - 29th IEEE International Conference on Image Processing, ICIP 2022

AU - Yoon, Dongsik

AU - Kwak, Jeong Gi

AU - Li, Yuanming

AU - Han, David

AU - Ko, Hanseok

PY - 2022

Y1 - 2022

N2 - Image inpainting is an old problem in computer vision that restores occluded regions and completes damaged images. In the case of facial image inpainting, most of the methods generate only one result for each masked image, even though there are other reasonable possibilities. To prevent any potential biases and unnatural constraints stemming from generating only one image, we propose a novel framework for diverse facial inpainting exploiting the embedding space of StyleGAN. Our framework employs pSp encoder and SeFa algorithm to identify semantic components of the StyleGAN embeddings and feed them into our proposed SPARN decoder that adopts region normalization for plausible inpainting. We demonstrate that our proposed method outperforms several state-of-the-art methods.

AB - Image inpainting is an old problem in computer vision that restores occluded regions and completes damaged images. In the case of facial image inpainting, most of the methods generate only one result for each masked image, even though there are other reasonable possibilities. To prevent any potential biases and unnatural constraints stemming from generating only one image, we propose a novel framework for diverse facial inpainting exploiting the embedding space of StyleGAN. Our framework employs pSp encoder and SeFa algorithm to identify semantic components of the StyleGAN embeddings and feed them into our proposed SPARN decoder that adopts region normalization for plausible inpainting. We demonstrate that our proposed method outperforms several state-of-the-art methods.

KW - Facial Image Inpainting

KW - Pluralistic Image Inpainting

KW - StyleGAN Inversion

UR - http://www.scopus.com/inward/record.url?scp=85146706809&partnerID=8YFLogxK

U2 - 10.1109/ICIP46576.2022.9898012

DO - 10.1109/ICIP46576.2022.9898012

M3 - Conference contribution

AN - SCOPUS:85146706809

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 1141

EP - 1145

BT - 2022 IEEE International Conference on Image Processing, ICIP 2022 - Proceedings

PB - IEEE Computer Society

Y2 - 16 October 2022 through 19 October 2022

ER -

DIFAI: DIVERSE FACIAL INPAINTING USING STYLEGAN INVERSION

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this