How MotionScan Technology Works

We play video games on flat, two-dimensional TV screens or computer monitors. But our real world is three-dimensional, as are the faces of the people who inhabit it. So, one of the major challenges for game makers is creating faces that have 3-D appeal. That's why MotionScan requires a complex light- and sound-proof studio setup filled with a lot of beefy hardware and software.

Game designers start by recording real flesh-and-blood actors performing while seated in the studio, which is illuminated with smooth, even light that leaves no dark areas or shadows that could throw off the system's accuracy. Markers are applied to the chests and necks of the actors; later in the process, these basically give the software a reference point for substituting the actor's head for that of his or her animated character.

MotionScan is different from regular motion-capture tech because it records so many angles simultaneously, creating a 3-D image on every take. Producers use 32 high-definition (2-megapixel) cameras that cost around $6,000 each; in fact, they're the same cameras used by NASA to record space shuttle launches [source: Gizmodo]. The cameras are meticulously positioned in pairs at specific spots all around, above and below the actor in order to capture all of the nuances of human head and facial movements at up to 30 frames per second. That adds up to about 1 gigabyte of data per second.

The cameras send their data to nine powerful computer servers, which can each process about 300 megabytes per second. These servers power animation software that interprets the video feeds, immediately creating animated textures and shapes that correspond to the actor's movements, all the while keeping images and sounds perfectly synchronized.

This processed data can then be applied to a digital puppet that animators manipulate with great precision. All of this adds up to a major technological accomplishment, and, as you'll see, it's one that takes a lot of work to set up.

There are more than 1,000 pieces of gear used in the MotionScan system. It can take technicians almost three working days to set up all of this equipment, which must be aligned to exacting specifications in order to work just right.

In addition to the 16 pairs of cameras aimed at recording facial details, there's also a 33rd camera that provides an overview scene of the studio. This last camera feeds video to a nearby room, from which the director can monitor the recording process and offer feedback to the actor.

The director may occasionally remind performers to remain relatively still. Too much movement makes it harder for MotionScan to detect minute facial details.

After the actor is done performing, game producers can select views generated by many cameras at many different angles. With all of that visual data, animators can duplicate lifelike facial movements and emotions. In the context of an intense game scene, those facial features might disclose critical information to the player.

And because the actor is being shot from so many perspectives, animators can pick and choose the angle they want to use for a specific scene. In addition, the animator can add or subtract different lighting effects, such as intensity and angle, to match the person's face to a dark and gritty bar or to a sunny summer day in a field.

A character who is lying might not be able to make eye contact with the player, and his or her eyebrows and lips probably show some noticeable discomfort. Likewise, if a character is happy to be speaking to the player, a friendly, open smile and wide eyes would communicate that message.

Next, you'll see how one company is using MotionScan to generate some of the most humanlike characters ever to grace an animated video game.

The massive computing horsepower driving MotionScan automatically tweaks fine details in each sequence, meaning animators don't have to labor for hours to make each facial expression look more human.

That means shorter filming and programming sessions, lower shooting budgets and quicker progress from concept to market, all saving the developer money and potentially making or breaking the game's profit margin.

Now, all that's left is to see how much gamers notice and appreciate MotionScan's breakthroughs.The game L.A. Noire, which will officially be released in May 2011, is the first type of media to use MotionScan, which was integrated into the game to add lifelike facial animation. Noire is an ominous, mature crime game for adults set in 1940s-era Los Angeles that's designed to play like an interactive movie. In keeping with its film-like aspirations, the game casts Aaron Staton (from television series "Mad Men") in the lead character's role.

The game relies heavily on character development and dialogue, and as such, accurate facial expressions are paramount -- without those nuances, players would have a harder time reading characters and moving through the plot. For example, as this is a crime drama, some characters' faces offer clear clues that they are lying. Interrogation and witness questioning are critical elements to help move the plot forward, so body language and facial expressions are key.

In the future, Depth Analysis, the company behind the trailblazing technology, will probably license it to other video game developers and also film producers. So, one day, you might see the technology pop up in animated feature films.

For now, the company is committed to perfecting and expanding the capabilities of MotionScan. Ultimately, MotionScan's makers would like to capture full-body images, which would be particularly useful for feature filmmakers. However, this is a technological hurdle that MotionScan hasn't yet been able to clear, in large part due to the immense computer power and even more complicated studio construction that would be required.

MotionScan's full-body debut will have to wait. In the meantime, companies are already challenging Depth Analysis' motion capture techniques. Keep reading to see how the competition might find a way to upstage MotionScan with better graphics -- or maybe even by using less sophisticated animation.

Now you know how important body language and facial expressions are to making game characters more captivating. But turning analog facial tics into digital emotions is no easy task. There are about 19 muscles in the human face, and duplicating all of their emotional acrobatics takes innovative programming and a complicated, cutting-edge group of technologies like MotionScan.

Other developers are eager to embrace MotionScan's goals -- and to achieve them better than Depth Analysis. A company called Remedy is working on facial expression technology that records about 64 facial poses; from this base set, animators can create expressions in real time and without additional actor performances. The technology is even said to account for facial blood flow and minute changes in skin color that happen as skin creases during muscle movement.

If they're successful, Remedy and Depth Analysis will both produce the kind of hyper-realistic animated characters that can make for very engrossing games. These humanlike characters are one of many efforts to bridge the so-called "uncanny valley," a theory originally associated with robotics.

When game designers mention the uncanny valley, they're referring to the gap between our perceptions of real people and their animated representations. The more realistic our digital doppelgangers are, the more probable it is that we'll find them disturbing. People are willing to accept cartoony caricatures of human beings -- and even pretty realistic ones -- but overly realistic renderings tend to be perceived as eerie and unsettling.

So, as human characters become more and more lifelike, the uncanny valley effect takes hold and begins to ruin the fun. It seems counterintuitive, but for this reason, those intricately detailed animations might actually detract from a game's ability to grab and hypnotize its players.

Animators and game makers are very much aware of the uncanny valley phenomenon, so they're striving to make their animations ever better. But some critics say game developers should reverse direction, focusing less on producing exact duplications of human movement and putting more effort into maintaining mesmerizing game play. Only then, they insist, do games really become more addictive, more fun and more profitable for the companies that make them.

No matter how MotionScan fares, it seems that animators are bent on making the most humanlike characters ever digitally created. These new developments in animation technology will likely make games more arresting visually, but they could also take gaming to a whole new level, resulting in a higher art form that's more exciting than ever before.