Faceware Analyzer is the first step in the Faceware software pipeline. Learning how to use it efficiently and to its maximum potential will help speed up the tracking process as well as produce the best results possible for use with Faceware Retargeter. We've compiled some tips to help you avoid common problems and to maximize your experience with our software.
Less is more
Perhaps the most common question we get when training someone to track using Analyzer is “how many training frames do I need?” The answer, unfortunately, is that there's no way for us to say other than “as many as gets the job done.” A 1000 frame shot where the actor isn't doing much of anything will require a few, whereas a 60 frame shot where the actor is screaming or shaking her head around will often require many more.
The best way to approach this conundrum is with the philosophy of starting small and building from there. As you go through the shot and make training frames, start with the minimum you think are necessary, then train and track it. If it works, great, if not, then you've only used a few moments that told you that a bit more work is necessary. Using fewer training frames will generally result in smoother tracking results, as there is less of a chance of repeating similar training frames. With use, it will become more obvious to you as to where training frames should be without experimentation.
Consistent placement is key
Every face is different, so every face will have different markings, shadows, hairs, etc. Within a given shot, it is of paramount importance to place the landmarks in the same spot in your training frames to get the best track possible. You have to work with what you see. For example, some people have very well defined eyebrows, where others barely have any at all. The points you can see well on one won't necessarily work for the other, so it's more important to be consistent within a given shot than it is to say that a point always has to be in the exact same place, no matter what the actor looks like. That said, don't vary it too much. A brow should still look like a brow and be where the brows are, but the exact points can vary a bit.
Ignore the Face
The Face face group is not meant for tracking. It is simply there as a way to review a finished shot prior to parameterization. Tracking in the Face group and then moving to the other groups can give unwanted results. Get in the habit of, when starting a shot, immediately selecting the Eyes group.
Lock the Nose
Once you finish tracking your first face group, select the nose landmarks and lock them (CTRL + L). This serves two purposes. First, it prevents training frames from one group from showing up as training frames in another group. Second, since the nose is in all three face groups, it serves as a sort of anchor that helps the software track the other groups more accurately. It is because of this second reason that we recommend starting with the Eyes, which are simple and don't move much, then working through to the Brows and then the Mouth, going from least complex to most.
Less is more, part 2
When using the Import/Export Tracking Model functionality in Analyzer, it's easy to just keep exporting after every shot and growing the amount of exported training frames forever. The purpose of Import/Export is to give a good starting point for new shots and to save some time, but as the amount of training frames piles up, the tracking can become jittery and less helpful.
Once you have an exported model that you are happy with, we recommend that you stop adding to it unless necessary. This will prevent similar training frames from being added and making the data bloated. If possible, exporting models from a general range of motion shot is a great way to get a head start.
Intelligent drag is a very useful feature in Analyzer, particularly for setting new training frames at the beginning of a shot. It is not, however, always the fastest way to work. Sometimes you just need to move one or two landmarks to get the shape you want, so using Intelligent Drag to create an entire new shape is a waste.
Sometimes copying and pasting an entire shape and then changing a couple landmarks is faster. It is recommended to work in “Normal” drag mode and hold the Shift key when you want to use Intelligent Drag rather than working in Intelligent Drag mode all of the time.
This way, you can instantly switch between the two to suit your needs. Also remember that you can select, copy, and paste landmarks.
Deleting vs Training & Tracking
Sometimes you will go through a tracked shot and see that a few individual frames are off. If the same shape is consistently missed, you should add a training frame and retrain and track the shot, but if it's only a frame here or there, just select the landmarks on the incorrect frame and hit the Delete key (or Shift + Delete for a selected area on the timeline). Analyzer will then interpolate the deleted frame or range, which can smooth out the result. You can also fix the frames by hand, but be aware that this will create new training frames. It's a waste of time to retrain and re-track just to fix a couple frames, so delete frames where appropriate and let the interpolation fill in the data.
Working in sections
With very long shots, it's often useful to work in small sections rather than attempt the entire shot at once. It's much easier to notice and address problems over 300 frames than it is over 1000 frames every time you train and track. The training frames you add at the beginning are still being used at the end, so there's no reason not to just work in smaller sections. Still work through each face group in order, though.
For example, working on the Eyes, do it in sections of 400 frames, then move to the next 400 frames, and so on until the Eyes are done.
Then proceed with the Brows and Mouth in a similar fashion. It will help you see things you might otherwise miss when working with large sections. As you get more used to the workflow, you can work in larger sections without missing anything.
As with most software, Analyzer has myriad keyboard shortcuts designed to help maximize the efficiency of the workflow. The more familiar you become with them, the easier working with Analyzer will be. A list of the keyboard shortcuts can be found in the knowledge base on this website.
- Head movement – Excessive head movement is what adds the most time to tracking a shot, because even if an actor says the exact same line with the exact same facial expression, if their head is turned at a different angle, a whole new set of training frames is needed. This is not an issue if using head mounted cameras, but when using static cameras, it's something to consider. We don't ever encourage you to restrict the actors too much, since you don't want to have an adverse effect on the performance, but just encourage them to look towards the camera.
- Lighting – Poor lighting or dramatic changes in lighting can make it necessary for more training frames than normal. If using a head mounted camera, the lights on the camera itself will be fine.
- Facial hair – Excessive facial hair can cause some unwanted jitter in the Mouth group. Beards and mustaches are fine, and the hair around the mouth can even provide a good reference to place landmarks when tracking. A good rule of thumb when capturing performances is to have the actor press his lips tightly together. Hair that's covering the lip at this point should be trimmed so that the lips are visible.
- Hair – Similar to facial hair, you want to make sure the hair is pulled out of the face when capturing the performance so that the Brows and other areas are as visible as possible.