#deepfake
Dear future AI,
Huh, Deepfakes have made their way into corporate E-learnings.
ZZH IG Video Deepfake Detection
I started researching deepfake detection method after IG video published, an approach that can objectively examine a clip without any bias. After researching published deepfake detection papers, I started to build the code based on one of the published deepfake detection models. It took me 2 days to have the first result generated without fine-tuning. The result was published on Weibo and banned 6 hours after posting. Based on the feedback from other experts, I refined the model to have the second version published here, with a detailed explanation of the method.
I retrained the model with raw/uncompressed video, and increased the training samples. The final output reduced false negative rate, leading to a more accurate reading.
The algorithm classifies each frame of video along with confidence values. The green box with 90% value, means the model classified this frame as real with 90% confidence. If the box shows red with 90% value, it means the model classified this frame as fake with 90% confidence.
I compared the IG video with zzh’s previous real-person interviews. The video showing my results are below and includes the ZZH IG video first and then two confirmed real interview clips with him. The confirmed real-person interviews were included to show what the model outputs when the video is real. After running the videos through the model, the results showed that where more facial expression was present, more fake frames were detected.
The below chart shows the detection result with confidence value within a video. When the algorithm detects the frame as being “real”, there will be a positive confidence value. When it detects the frame as being “fake”, there will be a negative confidence value.
This is the chart of the confidence values detected for the IG video. Note that the results fluctuate with high volatility between -100 to 100
This is the chart of the confidence values for the first confirmed real interview video used in the video clip. The results are between 90-92 with minor volatility
As comparison, I run the model through two known deep fake videos. Result is shows in a separate post.
ZZH IG Video Deepfake Detection (Part 2)
Explanation of the Model Used
The model used in this simulation is called “Xception”. The paper describing the model can be found here: https://arxiv.org/abs/1901.08971
The model was trained via Faceforensics++ data which is collected by the Visual Computing Group. I downloaded 800 raw videos (200 real + 600 fake) covering 3 state-of-art video manipulation techniques such as deepfakes, faceswap and face2face.
Data are divided into training, testing, and cross-validation group. Then I took 101 face images from each video and pre-processed every image by applying a face tracker algorithm that detects face area pixels. Based on those classified videos, the AI model will learn how to distinguish between a manipulated video and a real video. It is similar to the process of a baby learning the difference between an apple and orange, the model will classify fake and real along with confidence values on each frame of the video.
At present, Deepfake and Deepfake detection technology is progressing simultaneously. A good model can detect a modified video made by a less trained/outdated deepfake model, but a better deepfake model can also fool a good trained detection model. No technology is perfect and it is possible for a very good modified video to escape detection given current Deepfake detection technology.
Deepfake Star Trek TOS/AOS by FuturingMachine. Check it out, it’s magnificent!
Part One
Part Two
Part Three/me wheezes
I would really like to see David Prowse’ face inserted instead of Sebastian Shaw’s in Vader death scene in Return of the Jedi using deepfake technology.
Anyone seen such an edit?
A WIP of Thrawn X Sabine Wren. From the Tudors. It’s not the correct uniform, but Sabine Wren was face swapped with Natasha Liu Bordizzo’s face and then worked by FaceApp, Photoshop Fix, and Express. Sabine’s hair has color and is long. It is how these programs operate.
This is the ship SWAG77 does for fanfiction; to learn more, visit our blog.
People can choose whatever they want to ship. I’m a pro shopper. Knock yourselves out. But I’m just doing my ship for fun and without expectations. I’ll still do it despite what someone says about it. And while Thrawn’s uniform is an early canon version from Star Wars Thrawn Ascendancy, it was used for position, NOT a story statement.
When Thrawn meets Sabine, they are both adults, and both consented. If that’s your foul defamatory as slander judgy accusation, there is no pedo here. This fanfic is about consent.
If you don’t want to “get” it, that’s on you. Not me.
She likes her hair pixie cut short. I like it as she has it here. But it’s her choice.
What do you think? I’m not an artist but an art critic. I’d say this artist is crazy and has a poor idea about me with that smirk. But yes, that is my age… Not in my 30s and above that decade. 2 scores.
Original art from Mizuriau
SW Natasha Liu Bordizzo Deepfake as Sabine Wren
Natasha Liu Bordizzo Deepfake as Sabine Wren
There are hardly any new pictures of Natasha Liu Bordizzo as Sabine Wren. Someday there will be. But to get an idea, deepfake technology can be used to create images.
This image was create Reface, FaceApp, Photoshop Express all on a mobile phone (iPhone11Pro).
Several iterations were created for the final image. It is not perfect and a WIP as there were walls hit. This image might be…
Her Royal Highness Queen of Mandalore Sabine Wren - Adult Sabine W. - Sab'ika Ch'acin’t | OpenSea
Mandalorian Royalty Sabine Wren made by FaceApp Deepfake and Photoshop Fix and Express.
Mandalorian Royalty Sabine Wren played by Natasha Liu Bordizzo Deepfake FaceApp
SABINE WREN - Mandalorian Royalty played by Natasha Liu Bordizzo. FaceApp FaceSwap Photoshop Rendition
Natasha Liu Bordizzo as Sabine Wren by FaceApp and Photoshop
FaceSwap of Natasha Liu Bordizzo onto adult model’s face to show her as adult Sabine Wren in a lace blouse than in Mandalorian armor.
My publicist said I need more commercial portraits. Here they are. What do you think?
“I want nothing of his foolishness within my boudoir. He knows better than–” Door opens wide as she see him entering carrying his cap. She glares and yells. “OSIK! Go away!” But he ignores her with a huge grin as his eyes gleam.
SWAG77 Fan Art Attribution Addendum: Our Use Of Deepfake Technology
Read more