Cheap Facial Motion Capture - Part 3

In order for us to import the exported data to Maya we will need to rely on a third party application to handle the conversion.
The tool we are going to use is called Survey Solver Tracking Converter.
It's a free tool designed by Mincho Alekov Marinov.
When the tool is installed, just click the icon to bring up its interface.
Choose Moviemento as "Import Type".
Choose Maya Locators/2D as "Export Type".
Set the "Focal Length" to 50 mm.
Load up the image sequence used in part 2.
Click Convert Data! in order to start the process.
The locators are now imported and if you scrub through the timeline you will see them moving around in sync with to the video.
Before we proceed any further we will need to fix the scale.
Otherwise the bone moments will be enormous.
Open the Outliner and select the group called SS_Tracks.
Scale it down to roughly the same size as the head.
It doesn’t need to be a hundred percent accurate.
Start by selecting a locator and then shift click on a corresponding bone.
Select Parent Constrain with its default setting.
Click the Apply button in order to constrain the bone to the locator.
Repeat the process for all the bones.
When done, click play and verify that all bones are moving in a correct fashion.
Make sure to set the playback to real time at 30 frames per second when reviewing.
This can be changed after import by opening the Animation preferences.
Remember to select Keep keys at current frames if you decide to update the frame rate.
You might encounter a problem with constraints when reopening the scene.
This can however be fixed and more important, prevented.
To fix the problem you need to select the faulty bone.
Move it to a correct position.
Update the offset of the parent constrain via the Attribute Editor.
In order to continue refining and adding to the motion we need to bake the animation.
Select all bones that currently use a constrained motion.
Choose Bake simulation with default settings.
Click on the option "From Channel Box" and select the translate attributes via the channel box.
Initiate the bake process by clicking on Apply.
You can now delete the constraints with the command "Delete by type –> Constraints".
Verify that the bones are still moving before continuing.
Next step is to have the character’s lower gum move in a more natural way.
Select the jaw bone and then the lower gum.
Create a parent constrain.
Continue by selecting the head bone and then the lower gum.
Create another parent constrain.
With the gum constrained to two bones we can adjust the weight in order to balance the amount of movement.
Do this by selecting the gum and tweak the constraint weights found in the channel box.
We are now going to fix the initial face pose for the character.
There are two options to choose from in this case.
One will require you to directly alter the animation curves from the motion capture data.
It’s fast and intuitive way of dealing with the problem.
Select a bone in need of adjustments.
Open the Graph Editor and select translate Y.
Select all keyframes and move them with the move tool.
This will change the entire curve for the bone's vertical movement.
The problem with this solution is that we will destroy the original data making it hard to revert the process later on.
Option number two involves using the feature animation layers.
This will add animation on top of existing data allowing us to disable the layer and its changes at any time.
We can also weight how much a layer will influence the original data.
Begin by selecting all the bones that you intend to edit.
Select the animation layer tab and choose to "Create a Layer from Selected".
Give the layer a name for easier management.
Now click on "Set Weight to 1.0 and Key Layer".
This will activate the layer and allow us to view any changes that we make.
Now it’s only a matter of changing the position of the bones and set regular keyframes.
The new bone position will be added to the original animation.
We will continue by faking some depth movement to the mid lip bones.
We will exercise the squash and stretch animation principle in order to get the mid lips to move forward as the outer ones contract.
And move back as the outer ones expand.
We will use a tool called distance tool to measure the change in width for the outer lips bones.
Select the Distance tool and select the lower outer lips.
Use snap to point for an easier workflow.
When the locators have been placed you need to parent them to bones that they will measure.
By doing this Maya will constantly update the distance between the bones as the animation progresses.
Now we need to create some nodes that will be process the distance information and then move the mid lip bones along the Z axes.
Open the Hypershade Editor.
Select the distance node and click on "Input and Output connections".
Click on the utilities tab and create the following nodes;
Set Range and PlusMinusAverage.
We begin by converting the distance value by grabbing the distance tool on to the Set Range node.
Select other and then connect the Distance in the left column to the value x through z in the right column.
One value is all that we need but I’ve selected all for ease.
Now scrub through the animation and write down the maximum and minimum value displayed by the distance tool.
Put these values into the old max and old min of the Set Range node.
To create actual conversion we need to enter some values to the min and max column.
Iinput a relative small value for now because we will need to tweak it later on.
Drag the Set Range node on to the PlusMinusAverage and select input0.
Open up the Attribute Editor for PlusMinusAverage and make sure that the operation is set to "Sum".
Click on "Add new Item" in the Input 3D section.
The value we will add here is the current translate Z for the midLowerLip.
Select the bone and copy the value, then paste it to the PlusMinusAverage node.
And finally we will connect the output of PlusMinusAverage to translate Z of midLowerLip.
With the PlusMinusAverage selected, open the connection editor.
Select the midLowerLip and click on reload right.
Now connect the Output3Dz in the left column to translateZ in the right column.
Play the animation and review the result before you repeat the process for the midUpperLip.
Now that both lips are moving along Z axis we will tweak their movement to better suit our character.
Select both Set Range nodes and select the min values via the channelbox.
Tweak the values to your liking as you play through the animation.
Repeat the tweak process for the max values.
With the majority of the work behind us we can now focus on some final polish on our animation.
Start by importing the sound clip to the scene.
The file format should be uncompressed wave.
If no sound curve is visible in the timeline, right click on the timeline and select the imported clip from the sound section.
Select the midLowerLip, make sure that the recently created animation layer is active.
Now create keyframes for the bone to help emphasis some of the sounds.
Pay particular details to sounds containing F, P, M and B.
And that concludes this tutorial.
I hope this has been inspiring and I would love to see the stuff that you come up with using this information.
Please post any comments, feedback or animations in my youtube channel.
This has been a presentation by Eric Thelander.
Good luck with your projects!