It's important to note that not all video and photo editing techniques based in artificial intelligence and machine learning are deepfakes. Those in academics who work in the field see deepfakes as amateurish, relegated to mere face-swapping.
A group at the University of California Berkeley is working on a technique that takes an entire body in motion — a professional dancer — and swaps it onto an amateur's body on video. With a little AI wizardry, then, even someone with two left feet can at least appear to move like Baryshnikov. The Berkeley group detail its work in the paper, Everybody Dance Now.
The technique is not perfect, of course. But this is tricky stuff. Even pulling off a computer-generated moving face is difficult. As of now, most AI-generated faces, even on deepfakes — especially on deepfakes — are obvious forgeries. Something, almost invariably, seems a little off.
"I think that one thing is the shadow details of the faces," says Tinghui Zhou, a grad student in computer science at Berkeley and one of the authors of Everybody Dance Now. "We [humans] are very good at identifying whether a face is real or not — the shadow details, how the wrinkles move, how the eyes move — all those kind of details need to be exactly right. I think the machine-learning system these days is still not able to capture all those details."
Another new AI video-manipulation system — or, as its architects call it, a "photo-realistic re-animation of portrait videos" — actually uses one "source" actor that can alter the face on a "target" actor.
You, the "source" (for example), move your mouth a certain way, computers map the movement, feed it into the learning program and the program translates it to a video in which Obama mouths your words. You laugh, or raise your eyebrow, and Obama does, too.
A paper on that process, known as Deep Video Portraits, was presented at a computer graphics and interactive techniques conference in Vancouver in mid-August 2018, and reveals a place for the program: Hollywood.
"[C]omputer-generated videos have been an integral part of feature-film movies for over 30 years. Virtually every high-end movie production contains a significant percentage of computer-generated imagery, or CGI, from Lord of the Rings to Benjamin Button," the authors write. "These results are hard to distinguish from reality and it often goes unnoticed that this content is not real ... but the process was time-consuming and required domain experts. The production of even a short synthetic video clip costs millions in budget and multiple months of work, even for professionally trained artists, since they have to manually create and animate vast amounts of 3D content."
Thanks to AI, we can now produce the same imagery in a lot less time. And cheaper. And — if not now, soon — just as convincingly.