[其它] Interactive Generation of Dancing Animation with Music Synchronization

彬彬

797 主题	1 听众	1万积分

资深设计师

Rank: 7 Rank: 7 Rank: 7

纳金币: 5568
精华: 0

电梯直达

楼主

发表于 2011-12-30 18:53:03 |只看该作者 |倒序浏览

1 Introduction

With the explosive growth of user-generated content (UGC) such

as YouTube(R), computer animation that is synchronized with mu-

sic has been in great demand as a new type of UGC. For example,

it is popular to manually create such animation by a free software

called MikuMikuDance in Japan. However, it usually takes much

time to create a short piece of animation with requirement for nec-

essary knowledge, which is a matter of considerable concern for

UGC. On the other hand, several automatic systems are reported

such as [Xu et al. 2011], where dancing animation is synchronized

with the input music by rhythm and intensity features. However,

the automatically generated animation cannot reflect a user inten-

tion to see a specific dance motion (called performance motion) for

a specific part of music, thus user’s unique aesthetic appreciation is

not satisfied, which is essential for UGC.

In this paper, we present a novel approach to generate computer an-

imation by re-using motion data in an intuitive and simple way with

the use of interactive, user-in-the-loop techniques. Different from

the manual tools or fully automatic systems, our system allows user

to create his/her own animation interactively as choreographers do.

Basically, at specific time instants (e.g., the chorus), user composes

the desired performance motions by his/her unique sense. Then,

our system deals with the unspecified part and searches the best

motions in the database considering the requirements for music syn-

chronization, user’s composition, and motion smoothness.

2 Our Approach

Our system consists of off-line procedure and on-line procedure. A

graph structure, which is called meta motion graph, is constructed

in off-line. In the on-line procedure, given a set of performance

motions, the user arranges the performance motions in the timeline

as he/she likes by selecting a performance motion at proper time

instants when listening to the music, leaving the unspecified part in

the timeline being handled by the system. This procedure is called

performance composition in this paper. According to the perfor-

mance composition and beat information in the music, the motion

is concatenated from the meta motion graph by dynamic program-

ming, which is called motion synthesis.

Meta motion graph with performance motions: The original mo-

tion graph technique[Kovar et al. 2002] only focuses on the kine-

matics of motion. However, kinematics on its own is not enough

to solve our task due to the absence of such necessary features as

rhythm information and performance motion information. The ba-

e-mail: fji-xu, ko-takagi, sakazawag@kddilabs.jp

sic idea in the proposed meta motion graph is to concatenate motion

clips with a unit of beat interval. Namely, we segment the motions

into short clips by beat frames and organize these motion clips into

a graph structure by connecting the beat frames with similar poses

as shown in Fig. 1. In such a graph, any path has clear beat infor-

mation by counting the nodes. Therefore, the motion can easily be

synchronized with music. Another characteristics in our graph is to

embed the information of performance motions in the database.

In details, a meta motion graph consists of the set of nodes, edges,

and edge weights, as shown in Fig. 1. The node set includes all of

the beat frames in the motion database. Two successive beat frames

are connected by a uni-directional edge, where an edge label is at-

tached to embed the information of performance motion. For beat

frames with similar poses, they are connected by a bi-directional

edge and its edge weight is assigned as the pose similarity.

Motion synthesis: Here it is essential to properly define a cost

function. Basically, three requirements should be met in the gen-

erated motion. Firstly, beat instants in the generated motion should

be synchronized with those in the music. We can control the cost

of beat synchronization to zero by the method of [Xu et al. 2011].

Secondly, the performance composition should be well satisfied.

We define the distance between the desired performance motions

by the user and the generated ones by the system as: 0 if they are

the same performance motion or 1 for others. Thirdly, the gener-

ated animation should be as smooth as possible, where the source of

motion artifacts is the motion inconsistence at bi-directional edges.

Therefore, a penalty should be paid for selecting a bi-directional

edge, resulting in the cost function as: