A bearded Rihanna gyrates and sings about shining bright like a diamond. A female Jack Sparrow looks like she’d be a right laugh over a pint. The cartoon contours of The Incredible Hulk lend envious tint to Donald Trump’s awfully familiar cheek bumps.
Selfie culture has a fancy new digital looking glass: Reface (previously Doublicat) is an app that uses AI-powered deepfake technology to let users try on another face/form for size. Aka “face swap videos”, in its marketing parlance.
Deepfake technology — or synthesized media, to give it its less pejorative label — is just getting into its creative stride, according to Roman Mogylnyi, CEO and co-founder of RefaceAI, which makes the eponymous app whose creepily lifelike output you may have noticed bubbling up in your social streams in recent months.
The startup has Ukrainian founders — as well as Mogylnyi, there’s Oles Petriv, Yaroslav Boiko, Dima Shvets, Denis Dmitrenko, Ivan Altsybieiev and Kyle Sygyda — but the business is incorporated in the US. Doubtless it helps to be nearer to Hollywood studios whose video clips power many of the available face swaps. (Want to see Titanic‘s Rose Hall recast with Trump’s visage staring out of Kate Winslet’s body? No we didn’t either — but once you’ve hit the button it’s horribly hard to unsee… 😷)
TechCrunch noticed a bunch of male friends WhatsApp-group-sharing video clips of themselves as scantily clad female singers and figured the developers must be onto something — a la Face App, or the earlier selfie trend of style transfer (a craze that was sparked by Prisma and cloned mercilessly by tech giants).
Reface’s deepfake effects are powered by a class of machine learning frameworks known as GANs (generative adversarial network) which is how it’s able to get such relatively slick results, per Mogylnyi. In a nutshell it’s generating a new animated face using the twin inputs (the selfie and the target video), rather than trying to mask one on top of the other.
Deepface technology has of course been around for a number of years, at this point, but the Reface team’s focus is on making the tech accessible and easy to use — serving it up as a push-button smartphone app with no need for more powerful hardware and near instant transformation from a single selfie snap. (It says it turns selfies into face vectors representing distinguishing user’s facial features — and pledges that uploaded photos are removed from its Google Cloud platform “within an hour”.)
No need for tech expertise nor lots of effort to achieve a lifelike effect. The inexorable social shares flowing from such a user friendly tech application then work to chalk off product marketing.
It was a similar story with the AI tech underpinning Prisma — which left that app open to merciless cloning, though it was initially only transforming photos. But Mogylnyi believes the team behind the video face swaps has enough of a head (ha!) start to avoid a similar fate.
He says usage of Reface has been growing “really fast” since it added high res videos this June — having initially launched with only far grainier GIF face swaps on offer. In terms of metrics the startup us not disclosing active monthly users but says it’s had around 20 million downloads at this point across 100 countries. (On Google Play the app has almost a full five star rating, off of approaching 150k reviews.)
“I understand that an interest from huge companies might come. And it’s obvious. They see that it’s a great thing — personalization is the next trend, and they are all moving in the same direction, with Bitmoji, Memoji, all that stuff — but we see personalized, hyperrealistic face swapping as the next big thing,” Mogylnyi tells TechCrunch.
“Even for [tech giants] it takes time to create such a technology. Even speaking about our team we have a brilliant team, brilliant minds, and it took us a long time to get here. Even if you spawn many teams to work on the same problems surely you will get somewhere… but currently we’re ahead and we’re doing our best to work on new technologies to keep in pace,” he adds.
Reface’s app is certainly having a moment right now, bagging top download slots on the iOS App Store and Google Play in 100 countries — helped, along the way, by its reflective effects catching the eye of the likes of Elon Musk and Britney Spears (who Mogylnyi says have retweeted examples of its content).
But he sees this bump as just the beginning — predicting much bigger things coming down the sythensized pipe as more powerful features are switched on. The influx of bitesized celebrity face swaps signals an incoming era of personalized media, which could have a profoundly transformative effect on culture.
Mogylnyi’s hope is that wide access to synthensized media tools will increase humanity’s empathy and creativity — providing those who engage with the tech limitless chances to (auto)vicariously experience things they maybe otherwise couldn’t ever (or haven’t yet) — and so imagine themselves into new possibilities and lifestyles.
He reckons the tech will also open up opportunities for richly personalized content communities to grow up around stars and influencers — extending how their fans can interact with them.
“Right now the way influencers exist is only one way; they’re just giving their audience the content. In my understanding in our case we’ll let influencers have the possibility to give their audience access to the content and to feel themselves in it. It’s one of the really cool things we’re working on — so it will be a part of the platform,” he says.
“What’s interesting about new-gen social networks [like TikTok] is that people can both be like consumers and providers at the same time… So in our case people will also be able to be providers and consumers but on the next level because they will have the technology to allow themselves to feel themselves in the content.”
“I used to play basketball in school years but I had an injury and I was dreaming about a pro career but I had to stop playing really hard. I’ll never know how my life would have gone if I was a pro basketball player so I have to be a startup entrepreneur right now instead… With our platform I actually will have a chance to see how my pro basketball career would look like. Feel myself in the content and live this life,” he adds.
This vision is really the mirror opposite of the concerns that are typically attached to deepfakes, around the risk of people being taken in, tricked, shamed or otherwise manipulated by intentionally false imagery.
So it’s noteworthy that Reface is not letting users loose on their technology in a way that could risk an outpouring of problem content. For example, you can’t yet upload your own video to make into a deepfake — although the ability to do so is coming. For now, you have to pick from a selection of preloaded celebrity clips and GIFs which no one would mistake for the real-deal.
That’s a very deliberate decision, with Mogylnyi emphasizing they want to be responsible in how they bring the tech to market.
User generated video and a lot more — full body swaps are touted, next year — are coming, though. But before they turn on more powerful content generation functionality they’re working on building a counter tech to reliably detect such generated content. Mogylnyi says it will only open up usage once they’re confident of being able to spot their own fakes.
“It will be this autumn, actually,” he says of launching UGC video (plus the deepfake detection capability). “We’ll launch it with our Reface Studio… which will be a tool for content creators, for small studios, for small post production studios, maybe some music video makers.”
“We also have five different technologies in our pipeline which we’ll show in the upcoming half a year,” he adds. “There are also other technologies and features based on current tech [stack] that we’ll be launching… We’ll allow users to swap faces in pictures with the new stack and also a couple of mechanics based on face swapping as well, and also separate technologies as well we’re aiming to put into the app.”
He says higher quality video swapping is another focus, alongside building out more technologies for post production studios. “Reface Studio will be like an overall tool for people who want full access to our technologies,” he notes, saying the pro tool will launch later this year.
The Ukrainian team behind the app has been honing their deep tech chops for years — starting working together back in 2011 straight out of university and going on to set up a machine learning dev shop in 2013.
Work with post production studios followed, as they were asked to build face swapping technology to help budget-strapped film production studios do more while having to move their actors move around less.
By 2018, with plenty of expertise under their belt, they saw the potential for making deepface technology more accessible and user friendly — launching the GIF version of the app late last year, and going on to add video this summer when they also rebranded the app to Reface. The rest looks like it could be viral face swapping tech history…
So where does all this digital shapeshifting end up? “In our dreams and in our vision we see the app as a personalization platform where people will be able to live different lives during their one lifetime. So everyone can be anyone,” says Mogylnyi. “What’s the overall problem right now? People are scrolling content, not looking deep into it. And when I see people just using our app they always try to look inside — to look deeply into the picture. And that’s what really inspires us. So we understand that we can take the way people are browsing and the way they are consuming content to the next level.”
Comment