Coloring with words: Guiding image colorization through text-based palette generation

Hyojin Bahng; Seungjoo Yoo; Wonwoong Cho; David Keetae Park; Ziming Wu; Xiaojuan Ma; Jaegul Choo

doi:10.1007/978-3-030-01258-8_27

Coloring with words: Guiding image colorization through text-based palette generation

Hyojin Bahng, Seungjoo Yoo, Wonwoong Cho, David Keetae Park, Ziming Wu, Xiaojuan Ma, Jaegul Choo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

19 Citations (Scopus)

Abstract

This paper proposes a novel approach to generate multiple color palettes that reflect the semantics of input text and then colorize a given grayscale image according to the generated color palette. In contrast to existing approaches, our model can understand rich text, whether it is a single word, a phrase, or a sentence, and generate multiple possible palettes from it. For this task, we introduce our manually curated dataset called Palette-and-Text (PAT). Our proposed model called Text2Colors consists of two conditional generative adversarial networks: the text-to-palette generation networks and the palette-based colorization networks. The former captures the semantics of the text input and produce relevant color palettes. The latter colorizes a grayscale image using the generated color palette. Our evaluation results show that people preferred our generated palettes over ground truth palettes and that our model can effectively reflect the given palette when colorizing an image.

Original language	English
Title of host publication	Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
Editors	Martial Hebert, Vittorio Ferrari, Cristian Sminchisescu, Yair Weiss
Publisher	Springer Verlag
Pages	443-459
Number of pages	17
ISBN (Print)	9783030012571
DOIs	https://doi.org/10.1007/978-3-030-01258-8_27
Publication status	Published - 2018
Event	15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: 2018 Sept 8 → 2018 Sept 14

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11216 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	15th European Conference on Computer Vision, ECCV 2018
Country/Territory	Germany
City	Munich
Period	18/9/8 → 18/9/14

Bibliographical note

Publisher Copyright:
© Springer Nature Switzerland AG 2018.

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-01258-8_27

Cite this

Bahng, H., Yoo, S., Cho, W., Park, D. K., Wu, Z., Ma, X., & Choo, J. (2018). Coloring with words: Guiding image colorization through text-based palette generation. In M. Hebert, V. Ferrari, C. Sminchisescu, & Y. Weiss (Eds.), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings (pp. 443-459). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11216 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01258-8_27

Coloring with words: Guiding image colorization through text-based palette generation. / Bahng, Hyojin; Yoo, Seungjoo; Cho, Wonwoong et al.
Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Martial Hebert; Vittorio Ferrari; Cristian Sminchisescu; Yair Weiss. Springer Verlag, 2018. p. 443-459 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11216 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Bahng, H, Yoo, S, Cho, W, Park, DK, Wu, Z, Ma, X & Choo, J 2018, Coloring with words: Guiding image colorization through text-based palette generation. in M Hebert, V Ferrari, C Sminchisescu & Y Weiss (eds), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11216 LNCS, Springer Verlag, pp. 443-459, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 18/9/8. https://doi.org/10.1007/978-3-030-01258-8_27

Bahng H, Yoo S, Cho W, Park DK, Wu Z, Ma X et al. Coloring with words: Guiding image colorization through text-based palette generation. In Hebert M, Ferrari V, Sminchisescu C, Weiss Y, editors, Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Springer Verlag. 2018. p. 443-459. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-01258-8_27

Bahng, Hyojin ; Yoo, Seungjoo ; Cho, Wonwoong et al. / Coloring with words : Guiding image colorization through text-based palette generation. Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. editor / Martial Hebert ; Vittorio Ferrari ; Cristian Sminchisescu ; Yair Weiss. Springer Verlag, 2018. pp. 443-459 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{9992cad36872452fa3280adce16c2b72,

title = "Coloring with words: Guiding image colorization through text-based palette generation",

abstract = "This paper proposes a novel approach to generate multiple color palettes that reflect the semantics of input text and then colorize a given grayscale image according to the generated color palette. In contrast to existing approaches, our model can understand rich text, whether it is a single word, a phrase, or a sentence, and generate multiple possible palettes from it. For this task, we introduce our manually curated dataset called Palette-and-Text (PAT). Our proposed model called Text2Colors consists of two conditional generative adversarial networks: the text-to-palette generation networks and the palette-based colorization networks. The former captures the semantics of the text input and produce relevant color palettes. The latter colorizes a grayscale image using the generated color palette. Our evaluation results show that people preferred our generated palettes over ground truth palettes and that our model can effectively reflect the given palette when colorizing an image.",

author = "Hyojin Bahng and Seungjoo Yoo and Wonwoong Cho and Park, {David Keetae} and Ziming Wu and Xiaojuan Ma and Jaegul Choo",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2018.; 15th European Conference on Computer Vision, ECCV 2018 ; Conference date: 08-09-2018 Through 14-09-2018",

year = "2018",

doi = "10.1007/978-3-030-01258-8_27",

language = "English",

isbn = "9783030012571",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "443--459",

editor = "Martial Hebert and Vittorio Ferrari and Cristian Sminchisescu and Yair Weiss",

booktitle = "Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings",

}

TY - GEN

T1 - Coloring with words

T2 - 15th European Conference on Computer Vision, ECCV 2018

AU - Bahng, Hyojin

AU - Yoo, Seungjoo

AU - Cho, Wonwoong

AU - Park, David Keetae

AU - Wu, Ziming

AU - Ma, Xiaojuan

AU - Choo, Jaegul

PY - 2018

Y1 - 2018

N2 - This paper proposes a novel approach to generate multiple color palettes that reflect the semantics of input text and then colorize a given grayscale image according to the generated color palette. In contrast to existing approaches, our model can understand rich text, whether it is a single word, a phrase, or a sentence, and generate multiple possible palettes from it. For this task, we introduce our manually curated dataset called Palette-and-Text (PAT). Our proposed model called Text2Colors consists of two conditional generative adversarial networks: the text-to-palette generation networks and the palette-based colorization networks. The former captures the semantics of the text input and produce relevant color palettes. The latter colorizes a grayscale image using the generated color palette. Our evaluation results show that people preferred our generated palettes over ground truth palettes and that our model can effectively reflect the given palette when colorizing an image.

AB - This paper proposes a novel approach to generate multiple color palettes that reflect the semantics of input text and then colorize a given grayscale image according to the generated color palette. In contrast to existing approaches, our model can understand rich text, whether it is a single word, a phrase, or a sentence, and generate multiple possible palettes from it. For this task, we introduce our manually curated dataset called Palette-and-Text (PAT). Our proposed model called Text2Colors consists of two conditional generative adversarial networks: the text-to-palette generation networks and the palette-based colorization networks. The former captures the semantics of the text input and produce relevant color palettes. The latter colorizes a grayscale image using the generated color palette. Our evaluation results show that people preferred our generated palettes over ground truth palettes and that our model can effectively reflect the given palette when colorizing an image.

UR - http://www.scopus.com/inward/record.url?scp=85055100908&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01258-8_27

DO - 10.1007/978-3-030-01258-8_27

M3 - Conference contribution

AN - SCOPUS:85055100908

SN - 9783030012571

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 443

EP - 459

BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings

A2 - Hebert, Martial

A2 - Ferrari, Vittorio

A2 - Sminchisescu, Cristian

A2 - Weiss, Yair

PB - Springer Verlag

Y2 - 8 September 2018 through 14 September 2018

ER -

Coloring with words: Guiding image colorization through text-based palette generation

Abstract

Publication series

Other

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this