Semantic Colorization with Internet Images

晃晃

Semantic Colorization with Internet Images

Alex Yong-Sang Chia1 Shaojie Zhuo2 Raj Kumar Gupta3 Yu-Wing Tai

Siu-Yeung Cho3 Ping Tan2 Stephen Lin5

1Institute for Infocomm Research 2National University of Singapore 3Nanyang Technological University

4Korea Advanced Institute of Science and Technology  5Microsoft Research Asia









Abstract

Colorization of a grayscale photograph often requires considerable

effort from the user, either by placing numerous color scribbles over

the image to initialize a color propagation algorithm, or by looking

for a suitable reference image from which color information can

be transferred. Even with this user supplied data, colorized images

may appear unnatural as a result of limited user skill or inaccurate

transfer of colors. To address these problems, we propose a col-

orization system that leverages the rich image content on the inter-

net. As input, the user needs only to provide a semantic text label

and segmentation cues for major foreground objects in the scene.

With this information, images are downloaded from photo sharing

websites and filtered to obtain suitable reference images that are re-

liable for color transfer to the given grayscale photo. Different im-

age colorizations are generated from the various reference images,

and a graphical user interface is provided to easily select the desired

result. Our experiments and user study demonstrate the greater ef-

fectiveness of this system in comparison to previous techniques.

1 Introduction

Image colorization can bring a grayscale photo to life, but often

demands extensive user interaction. In techniques such as [Levin

et al. 2004; Huang et al. 2005], a user typically needs to specify

many color scribbles on the image to achieve a desirable result.

Moreover, it can be difficult for a novice user to provide these color

scribbles in a consistent and perceptually coherent manner. Other

methods take a different approach by using a color image of a sim-

ilar scene as a reference, and transferring its colors to the grayscale

input image [Reinhard et al. 2001;Welsh et al. 2002]. This requires

less skill from the user, but a suitable reference image may take

much effort to find. In addition, inaccuracies in color transfer can

lead to results that appear unnatural.

To colorize grayscale photos with less manual labor, we present

a system that takes advantage of the tremendous amount of im-

age data available on the internet. The internet is almost certain

to contain images suitable for colorizing a given grayscale input,

but finding those images in a sea of photos is a challenging task,

especially since search engines often return images with incompat-

ible content. We address this problem with a novel image filter-

ing method that analyzes spatial distributions of local and regional

image features to identify candidate reference regions most com-

patible with the grayscale target. The user needs only to provide

semantic labels and segmentation cues for major foreground ob-

jects in the grayscale image, which is more intuitive than previous

scribble based user interaction. For each foreground object, a mul-

titude of images is downloaded from the internet using the semantic

label as a search term, and our system filters them down to a small

number of best matches. To minimize the amount of user input, our

system does not require the user to label and segment background

regions. Rather, it exploits correlations between the foregrounds

and backgrounds of scenes by re-using the images downloaded for

the foreground objects, which likely contain some backgrounds that

can serve as a reference for background colorization.

From the filtered reference images, the system transfers colors to

the corresponding foreground objects and background with a graph-

based optimization based on local properties at a super-pixel resolu-

tion. Since the filtering method seeks reference objects with spatial

distributions of features most consistent with the target object, color

transfer becomes more reliable, as corresponding locations between

the reference and target can be identified more accurately. Various

colorization results are generated from the set of reference images,

and the user is provided an intuitive interface to rapidly explore the

results and select the most preferred colorization.









全文请下载附件：

C.R.CAN

对你的思念象袅袅的轻烟不绝如缕，对你的祝福是潺潺的小溪叮咚作响。或许岁月将往事褪色，或许空间将彼此隔离。但值得珍惜的依然是你给我的情谊。再次对你说声：新年快乐！

奇

长了不少见识

C.R.CAN

心中有爱，爱咋咋地

奇

再看一看，再顶楼主

C.R.CAN

我也来支持下

奇

响应天帅号召，顶

奇

加精、加亮滴铁子，尤其要多丁页丁页

奇

不错哦,顶一下......

C.R.CAN

呵呵,真得不错哦!!