[SOLVED] : “Deepfakes”: how are hyper-rigged videos made? Part two

How to make a deepfake?

Is it so easy to make “deepfakes”, those videos where one face is replaced by another, and which generates a lot of ink? We tried.

Anyone would be able to make a deepfake. Really ? Concerns about this artificial intelligence technology, which makes it possible to replace one face with another in a video, are partly linked to the fact that it would be accessible to all – which would draw a future where everyone could broadcast modified videos, making anyone say anything.

We therefore tried to make some, in order to check if it is so easy to create today. The guinea pig: the author of these lines, relatively smart about the keyboard, but without additional qualification. The test: replace Jean Dujardin with Emmanuel Macron in a sequence of OSS 117, because why not. (We thought we were original, we were wrong.)

First step, find software. Several are available online for free. We go for DeepFaceLab, which is on the rise, and is used by talented “deepfake artists” like Ctrl Shift Face. Second step, find a user manual. Several are available online, but it’s best to understand English – or Russian.

First problem: the tutorial we choose warns us that we will need a powerful graphics card to run the program. Damn it. The basic computer will not be enough, we need a gamer machine. But chance does things well: we work at Pixels, and we have enough to test video games.

888 times Jean Dujardin

Once equipped, we need to find, and download, two videos. The extract from OSS 117 where we want to show Emmanuel Macron, and a video of the French president. Padawan’s first mistake: picking one of his speeches a bit off the cuff – we’ll see why soon.

We import the first video in the software, quite easy to use, which runs for long minutes. But it works ! The program succeeds in extracting the face of Jean Dujardin on each of the images composing the video. And that makes a lot (888). These images will be the basis from which the software will operate: if it has correctly identified the face, it will then be able to place another in the right place.

Then begins a very laborious step. We have to check, one by one, that all these images represent Jean Dujardin’s face correctly – the software does indeed make mistakes.

Once the pruning is done, it is necessary to go over all these images again. This time to make sure that the program has succeeded in spotting the shape of the face, the location of the mouth, eyes, nose, which it indicates to us by green lines. In the event of an error, we have to reposition the assembly by hand, which is not always easy.

After a long time, victory: we finally have “clean” data on the face of Hubert Bonisseur de La Bath. All that remains is to… start over with the other video, that of the French president.

Shame

The time for “training” comes. From these data, the software will “learn” Macron’s face – his features, his way of moving, etc. – in order to be able to reproduce it and insert it in place of Jean Dujardin’s. We hastily set a few parameters that we don’t understand (blindly following the tutorial found on the Internet) and let the machine work… for hours.

We leave our computer at nightfall and let it run until we get back in the morning. O joy: no problem (bug, power cut, Windows update, meteorite) has come to disrupt this crucial step. It remains to launch THE process, the one that should – fingers crossed – generate our deepfake. It only lasts a few minutes – a trifle for who has survived previous trials – and suddenly, bim, there she is.

Feverish, we double-click on the video. And it is with undisguised shame that we present to you here this first attempt at deepfake.

“The skill curve is very slow”

The result, let’s not be afraid of words, is zero. Why ? First, the basic videos were chosen badly: they are of poor quality. Then, the settings of the parameters are not the right ones, from where this effect of mask superimposed on the face of Jean Dujardin. Finally, the sequence of OSS 117 included a difficulty: the sunglasses, which obstruct part of the face of the actor. To circumvent it, it would have been necessary to carry out a difficult work of manual calibration.

We could, nourished by this first experience, start over. Go on looking for two good quality videos. Download them. Extract the faces. Sort faces. Retort faces. Restart. Train the machine for hours. Read tutorials to understand the settings. Make the settings. To finally discover the result. Which won’t necessarily be ideal either. The laziness wins us over.

“Unlike Photoshop, it takes a day of calculation before a first result”

“The skill curve is very slow,” confirms Vincent Nozick, lecturer at the University of Paris-Est Marne-la-Vallée (Gaspard-Monge computer laboratory, LIGM) and creator of software designed to detect deepfakes, called MesoNet. “Because unlike Photoshop, for example, it takes about a day of calculation for a computer before giving a first result. “

“The hardest part is having a good database of thousands of good quality and varied images. Then, you will have to understand the parameters and properly estimate them, to make a face a little bigger for example. If you don’t choose the right ones, it’s not pretty. And that, we only know after a day. It takes experience. “

Second trial

Fortunately, the Pixels team has endless resources. A second, higher quality guinea pig is ready to play. Loïc is a web developer, witnessed the sinking of guinea pig no 1 and decided to get involved in his turn.

The world is unfair: more skilful, he avoids the gross mistakes of the beginner and appropriates the parameters. “I started by selecting two videos in HD,” he describes. He also avoids choosing a complicated OSS 117 sequence – no awkward sunglasses. More studious, he dissects the tutorials in order to better understand the operation of the software, refines the sorting of faces using the software’s advanced tools, and doubles the training time: twenty-four hours in all.

The result, although imperfect, is beyond comparison with the first experiment.

There remains a problem of light, linked to the initial video by Emmanuel Macron. And a few “blinks” of the face. Let’s go over the details: “I changed the settings, there are a lot of them. I tried several brightness settings, increased the blur on the contours of the face and decreased its size by 5%, to avoid the mask effect. A little laborious: “Each time the software has to generate the video again, which takes two or three minutes. But it is paying off.

Conclusion: yes, today “anyone” can make a deepfake. You still have to be damn motivated, patient in the face of these tedious tasks, and therefore have time in front of you and adequate equipment. The successful three-click deepfake, which would allow anyone to broadcast fake images of Donald Trump or get revenge on their ex with a pornographic video, is still science fiction – at least for now.

[SOLVED] :

What is DeepFake and how does it work?

In the X-men movie, the character Mystique is able to change his appearance to look like everyone he wanted. This trait was one of the most powerful in science fiction movies, allowing a person to gain someone’s trust before killing them.

The trust is an important element for human relationships to function. And the cybercriminals always try to exploit it in different ways for malicious purposes.

Until now, the video may still have been something to be believed. We said to ourselves that it was complicated to modify. Now, it will still be a medium that we can no longer trust.

Welcome to the world of deepfake, imitate anyone to make him say and do anything.

Deepfake, what is it?

Deepfake is a computer method which allows to create fake videos based on artificial intelligence. This technology is mainly used to overlay existing videos on top of other videos (for example: the change of face of a person on a video).

The against deepfake is formed from “deep learning”(Deep learning) and“Fake“. This method can be used to create fake news and malicious hoaxes.

Today, thanks to open source programs, a day can be enough for a single person in the back of their room to tweak a deepfake of average quality but which will have its small effect.

One example is a deepfake from Facebook’s Mark Zuckerberg. The video, posted on Instagram, reveals the whole truth about his social network and his intention to control humanity.

How does a deepfake work?

The best known approach for create a deepfake is here substitution from one face to another and its animation as perfect as possible.

You have to take a person who looks a bit like the target and shoot them from different angles and lighting, asking them to make various facial expressions. The program will use this base to artificially animate the target’s real face.

To animate a face, it is necessary have access to several thousand pictures of a person, from different angles. Often the frame-by-frame collection of a video can suffice. There are algorithms capable of grabbing the content of one video and applying it to another without altering its style or quality.