Coloring with words: Guiding image colorization through text-based palette generation

Hyojin Bahng, Seungjoo Yoo, Wonwoong Cho, David Keetae Park, Ziming Wu, Xiaojuan Ma, Jaegul Choo

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    19 Citations (Scopus)

    Abstract

    This paper proposes a novel approach to generate multiple color palettes that reflect the semantics of input text and then colorize a given grayscale image according to the generated color palette. In contrast to existing approaches, our model can understand rich text, whether it is a single word, a phrase, or a sentence, and generate multiple possible palettes from it. For this task, we introduce our manually curated dataset called Palette-and-Text (PAT). Our proposed model called Text2Colors consists of two conditional generative adversarial networks: the text-to-palette generation networks and the palette-based colorization networks. The former captures the semantics of the text input and produce relevant color palettes. The latter colorizes a grayscale image using the generated color palette. Our evaluation results show that people preferred our generated palettes over ground truth palettes and that our model can effectively reflect the given palette when colorizing an image.

    Original languageEnglish
    Title of host publicationComputer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
    EditorsMartial Hebert, Vittorio Ferrari, Cristian Sminchisescu, Yair Weiss
    PublisherSpringer Verlag
    Pages443-459
    Number of pages17
    ISBN (Print)9783030012571
    DOIs
    Publication statusPublished - 2018
    Event15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany
    Duration: 2018 Sept 82018 Sept 14

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume11216 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Other

    Other15th European Conference on Computer Vision, ECCV 2018
    Country/TerritoryGermany
    CityMunich
    Period18/9/818/9/14

    Bibliographical note

    Publisher Copyright:
    © Springer Nature Switzerland AG 2018.

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • General Computer Science

    Fingerprint

    Dive into the research topics of 'Coloring with words: Guiding image colorization through text-based palette generation'. Together they form a unique fingerprint.

    Cite this