标题: Semantic Colorization with Internet Images [打印本页] 作者: 晃晃 时间: 2011-12-28 09:46 标题: Semantic Colorization with Internet Images Semantic Colorization with Internet Images
Alex Yong-Sang Chia1 Shaojie Zhuo2 Raj Kumar Gupta3 Yu-Wing Tai
Siu-Yeung Cho3 Ping Tan2 Stephen Lin5
1Institute for Infocomm Research 2National University of Singapore 3Nanyang Technological University
4Korea Advanced Institute of Science and Technology 5Microsoft Research Asia
Abstract
Colorization of a grayscale photograph often requires considerable
effort from the user, either by placing numerous color scribbles over
the image to initialize a color propagation algorithm, or by looking
for a suitable reference image from which color information can
be transferred. Even with this user supplied data, colorized images
may appear unnatural as a result of limited user skill or inaccurate
transfer of colors. To address these problems, we propose a col-
orization system that leverages the rich image content on the inter-
net. As input, the user needs only to provide a semantic text label
and segmentation cues for major foreground objects in the scene.
With this information, images are downloaded from photo sharing
websites and filtered to obtain suitable reference images that are re-
liable for color transfer to the given grayscale photo. Different im-
age colorizations are generated from the various reference images,
and a graphical user interface is provided to easily select the desired
result. Our experiments and user study demonstrate the greater ef-
fectiveness of this system in comparison to previous techniques.
1 Introduction
Image colorization can bring a grayscale photo to life, but often
demands extensive user interaction. In techniques such as [Levin
et al. 2004; Huang et al. 2005], a user typically needs to specify
many color scribbles on the image to achieve a desirable result.
Moreover, it can be difficult for a novice user to provide these color
scribbles in a consistent and perceptually coherent manner. Other
methods take a different approach by using a color image of a sim-
ilar scene as a reference, and transferring its colors to the grayscale
input image [Reinhard et al. 2001;Welsh et al. 2002]. This requires
less skill from the user, but a suitable reference image may take
much effort to find. In addition, inaccuracies in color transfer can
lead to results that appear unnatural.
To colorize grayscale photos with less manual labor, we present
a system that takes advantage of the tremendous amount of im-
age data available on the internet. The internet is almost certain
to contain images suitable for colorizing a given grayscale input,
but finding those images in a sea of photos is a challenging task,
especially since search engines often return images with incompat-
ible content. We address this problem with a novel image filter-
ing method that analyzes spatial distributions of local and regional
image features to identify candidate reference regions most com-
patible with the grayscale target. The user needs only to provide
semantic labels and segmentation cues for major foreground ob-
jects in the grayscale image, which is more intuitive than previous
scribble based user interaction. For each foreground object, a mul-
titude of images is downloaded from the internet using the semantic
label as a search term, and our system filters them down to a small
number of best matches. To minimize the amount of user input, our
system does not require the user to label and segment background
regions. Rather, it exploits correlations between the foregrounds
and backgrounds of scenes by re-using the images downloaded for
the foreground objects, which likely contain some backgrounds that
can serve as a reference for background colorization.
From the filtered reference images, the system transfers colors to
the corresponding foreground objects and background with a graph-
based optimization based on local properties at a super-pixel resolu-
tion. Since the filtering method seeks reference objects with spatial
distributions of features most consistent with the target object, color
transfer becomes more reliable, as corresponding locations between
the reference and target can be identified more accurately. Various
colorization results are generated from the set of reference images,
and the user is provided an intuitive interface to rapidly explore the
results and select the most preferred colorization.
