Human Movements - The Easiest Motion Capture File Format

nikola — Fri, 29 May 2009 21:02:00 GMT

This is the second post of the "Reallistic Human Movements with Silverlight" series. I'll spend a little time talking about the .bvh file format structure. I’m still working on the code and that’s why I’ll make this post short in order to get back to the mocap code. There is a link to more relevant info at the bottom.

I updated to the motion capture source code (download attachment to this blog) that contains a major fix related to joint rotation.

The latest sample is here: http://nokola.com/mocap.html

Some people thought the sample is a saved video – it is not. The sample shows rendering of 3D motion capture data at real time on Silverlight. Try out the new “body fat” slider! If you click “Add another person” then “Show Info” you’ll see the x, y, z angles for joint rotations in realtime.

Before going on, a quick overview of one way animation studios do motion capture today:

1. A human (e.g. actor or martial artist) is placed in a special costume for motion capture

2. There are numerous cameras surrounding that person (e.g. 24 cameras) that take videos of the person

3. Software processes the videos to "understand" where the person's hands, feet, etc are and saves this information in a motion capture file

One such file format is .BVH (Biovision motion capture).

The bvh contains 2 sections:

1. Skeleton data describing the hierarchy and initial pose of the human skeleton whose motion is being captured

2. Frame data describing how the human's joints move over time

The skeleton data looks something like this:

HIERARCHY

ROOT Hips

{

OFFSET 0.00 0.00 0.00

CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation

JOINT Chest

{

OFFSET 0.00 5.21 0.00

CHANNELS 3 Zrotation Xrotation Yrotation

JOINT Neck

{

OFFSET 0.00 18.65 0.00

CHANNELS 3 Zrotation Xrotation Yrotation

JOINT Head

{

OFFSET 0.00 5.45 0.00

CHANNELS 3 Zrotation Xrotation Yrotation

End Site

{

OFFSET 0.00 3.87 0.00

}

…

For each joint, we have its 3D offset relative to its parent joint (Hips is the parent of Chest) and its channel information.

You can read more about this here: http://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html

One thing missing from the above page is that it is very important to note the order of the channels –this is the order in which you have to rotate the joints. For example order of “Zrotation Xrotation Yrotation” means that you have to rotate by z,x,y in that order.

The bug that I fixed in my code was related to that. If you don’t follow the order in the file, you’ll end up with weirdly moving figures or jitter in the playback.

The second part of the .bvh file is the frame information – I encourage you to read about it here: http://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html

This resource explains it pretty well!

More Info

One person also asked me “how do I capture my own motion data?” Usually this is pretty costly (e.g. $10,000-$20,000), but could be made with lower cost too!

I’ll talk a little bit about this in the next post. In the meantime you can search for “motion capture single camera” or "markerless motion capture" on the internet. This seems to be the cheaper way to do it (cheaper as in $50-$500). Recent scientific developments (last few months) in markerless motion capture really open the possibility for the public to use this technology!

Reallistic Human Movements in Silverlight - My Own Motion Capture Viewer!

nikola — Tue, 26 May 2009 22:36:00 GMT

Hoho! yes! How to make reallistic human movements in Silverlight?

Check out this stickman animation demo here: http://www.nokola.com/mocap.html

Download source code for the above sample in the attachments to this post below.

You're looking at 3D realtime animation of stick man figures using motion data found on the internet.

The motion capture viewer took me the whole day yesterday - still nothing compared to doing these animations manually frame by frame.

As a start, you need movement data (motion capture or mocap data). You can find a lot of that online. This site http://sites.google.com/a/cgspeed.com/cgspeed/motion-capture/3dsmax-friendly-release-of-cmu-motion-database contains thousands of human motions - e.g. jump, punch, kick, wash windows, mop floor, and other.

I used the .BVH files - they have a short description of the skeleton being animated (joints, head etc) as well as rotational information (e.g. "wrist should be rotated at 50 degrees at time X")

Since this is a work in progress, I'm just publishing this quick blog post - more info will come later, because I want to make the animations transition into each other.

In the future I'll talk about the motion capture file formats and some challenges I faced (and are still facing!) implementing the above sample.

Here's a quick overview of motion capture from Wikipedia:

"Motion capture, motion tracking, or mocap are terms used to describe the process of recording movement and translating that movement onto a digital model. Initially invented in Scotland, it is used in military, entertainment, sports, and medical applications. In filmmaking it refers to recording actions of human actors, and using that information to animate digital character models in 3D animation. When it includes face, fingers and captures subtle expressions, it is often referred to as performance capture."

http://en.wikipedia.org/wiki/Motion_capture

Motion capture is used in games, movie special effects, and now in demos like the one above :)

You had me at "Hello World!" : Motion Capture

Human Movements - The Easiest Motion Capture File Format

Reallistic Human Movements in Silverlight - My Own Motion Capture Viewer!