Why so serious, Donald Trump? Maybe you’d find less need to reach for ALL CAPS as you compose your tweets if you smiled a little more? Go on, have a go… There, not so hard was it!
Okay, okay, if you’re finding the vision of Trump’s visage with a smile on it generating ‘uncanny valley’ levels of unease and creepiness you’d be right.
The smile is FAKE NEWS folks! Created, after the fact, by a photo-realistic face-morphing iOS app, called FaceApp, which uses neural networks to edit selfies — letting users bend reality in a few choice ways, such as by adding a smile, or shaving off a few years or applying a beautifying ‘hotness’ filter. Who said augmented reality requires donning a pair of goggles?
FaceApp is being developed by a small team out of Saint-Petersburg, Russia.
“We developed a new technology that uses neural networks to modify a face on any photo while keeping it photorealistic. For example, it can add a smile, change gender and age, or just make you more attractive,” explains founder and CEO Yaroslav Goncharov, a former head of department at Yandex who is funding the project out of his own pocket.
Selfie beautification apps aren’t new of course. Recently a veteran of the category, Meitu, that’s been popular in China for years picked up some steam in the US — offering a ‘vaseline screen‘ style effect to soften and smooth skin tones and add kawaii glints to eyes.
Another example, Lollycam, also offers beautification filters plus the ability to augment selfies in all sorts of creative ways with additional cinematic effects.
And that’s just two of the indie options kicking around the category — which also includes plenty of sophisticated in-house selfie-styling options at social sharing giants like Snapchat. But the enduring popularity of selfies suggests humanity’s inner Narcissus remains prone to being tickled by tricks and tweaks that can put a new spin on the presentation of self. So the ideas keep coming.
Goncharov also says FaceApp is taking a different tack by not applying a filter to selfies. Rather it’s using deep learning technology to alter the photo itself.
“Our main differentiator is photorealism,” he tells TechCrunch. “After applying a filter, it is still your photo. Other apps intentionally change a picture in a way it is entertaining, but not a real photo anymore (Meitu can make you an anime character, Prisma turns it to art, MSQRD adds cartoon masks etc.).
“We believe that such entertaining effects are subject to trends, but photorealism is timeless.”
“In addition to a sound product concept, we think that we are quite ahead in terms of technology. As far as I know, there are no products or research papers that can claim similar quality on this task,” he adds.
Whether photorealism really is timeless in an era of increasing reality augmentation remains to be seen. But Goncharov reckons he has a viral hit on his hands, saying FaceApp has racked up more than a million downloads just two weeks since launch (the app is iOS only for now).
Countries where it’s been making the biggest splash so far include Japan, Netherlands, Hong Kong, Norway, Russia, and Argentina, with Goncharov noting the app has topped the App Store charts in four countries, and made top iOS app in the Photo & Video category in seven so far.
Another Russian-made photo-altering app, Prisma, which uses AI to apply style transfer art effects to images, also picked up steam quickly last summer, grabbing 1.6M downloads for its photo-tweaking tool in a week — going on to pass more than 70M downloads (and bag ~2M active daily users) within six months, with copycats aplenty. (Prisma has now launched a social-sharing platform of its own to try to turn a photo feature tool into a fully fledged community.)
Safe to say, the network effects of mainstream photo-sharing platforms such as Instagram, where both Prisma pics and FaceApp selfies have gained eyeballs, are able to lift newcomers into the limelight relatively quickly — at least if they have something people find novel enough to share. But how sustainable such fast-rising virality is, is the key question for apps that piggyback on platform giants.
In terms of the specific technology it’s using to alter selfies in a photorealistic way, Goncharov says FaceApp makes use of “deep generative convolutional neural networks”.
“This technology is quite mature for some tasks, such as artistic style transfer or super-resolution, but has extreme challenges for photo-realistic tasks especially with high resolution images,” he argues. “I don’t think similar effects can be achieved with conventional (not deep learning) algorithms.
“Adding a smile may look like a simple modification on the surface but, in fact, is extremely difficult. Smiling is not just lips moving — the entire face changes in a subtle, but complicated, way. Also, for this effect to work on most real life photos, you need to account for too many factors: posture, lighting, skin color, shape of lips and eyes, photo quality, etc.”
Testing the app I found the quality of the results do vary a fair bit — depending on the selfie you’re applying them to and the particular effect you’re trying to achieve.
As well as adding a smile, there are options for aging a selfie, making it look younger, making it more attractive, and a gender-bending feature (changing male selfies to females and vice versa). Photos have to be uploaded to FaceApp’s servers for processing, but at the time we tested it it was only taking a few seconds from tap to finished effect.
Some results (such as the added smile) look generally more plausible than others (the aging effect is pretty obviously unreal). But at a quick glance most of the effects are at least in the photo-realistic ballpark, even if they’re also a bit surreal/creepy…
Goncharov says the team of four full-time engineers developed FaceApp’s core tech in-house, though he confirms they are using some AI open source libraries — such as Google’s Tensorflow.
“They are relatively low level/general use libraries that can be used to build almost anything,” he says, adding: “It took us eight months to release the first version of FaceApp, thanks to our prior background in deep learning and computer vision.”
He also says they have spent nothing on marketing for FaceApp at this point, with all early growth being organic, thanks to social shares. (Morphed photos being automatically badged with ‘FaceApp’ and the app pushing social share options at users immediately after they process an image will be helping there.)
How might the team monetize the app, assuming they can keep growing usage? One option might be sponsored filters, says Goncharov. “Our upcoming effects would support it in a nice way. Before we start to implement this, we need to hit quite high numbers in terms of daily active users to use this model efficiently. However, given the app performance we see, we believe that it is a viable option.”
A lot might depend on the variety and quality of the upcoming effects — i.e. if FaceApp is to be anything more than a flash in the selfie styling pan. Adding a smile to a famously grumpy selfie is fun once, but doesn’t seem to offer lasting utility.
Still, some smiles are a lot more winning than others…
Comment