Facebook will rely on an army of outside developers to contribute augmented reality image filters and interactive experiences to its new Camera Effects platform. After today’s Facebook F8 conference, the first effects will become available inside Facebook’s Camera feature on smartphones, but the Camera Effects platform is designed to eventually be compatible with future augmented reality hardware, such as eyeglasses.
While critics thought Facebook was just mindlessly copying Snapchat with its recent Stories and Camera features in Facebook, Messenger, Instagram and WhatsApp, Mark Zuckerberg tells TechCrunch his company was just laying the groundwork for today’s Camera Effects platform launch.
Because stylish, affordable AR glasses might be years away but everyone already has a camera phone, Zuckerberg tells me “the first augmented reality platform that becomes mainstream isn’t going to be glasses, it’s going to be cameras.” He says other apps (like Snapchat) have been keeping their augmented reality experience creation tools to themselves, but “building an open platform I think is going to be one of the big advances that pushes this forward . . . all developers can work on [the open platform] so that way you don’t just have 10 or 20 effects but you have thousands.”
That’s the same logic behind my August thought piece on why the camera should be Facebook’s next platform, which it now is. Developers let Facebook offer a much wider diversity of experiences.
Facebook is now allowing developers to apply for access to the closed beta of the platform’s AR Studio tool. It will allow developers to use precise location, object recognition and depth detection to create their effects. Facebook’s camera will be able to recognize specific objects like a coffee cup, and bubble up related effects to users like steam coming off the cup or sharks swimming inside the coffee.
Meanwhile, anyone can now use the Frame Studio to upload simple, static overlaid image filters that will appear in Facebook Camera to their friends or a Page’s fans. There’s no need for coding knowledge to use Frame Studio, you just upload an image and your name will appear on the Frame’s preview and News Feed posts to give you credit.
For now Facebook won’t allow any branding or advertising unless pre-approved, though it will eventually monetize the platform, with a spokesperson saying “We are focused on making this product great for people before introducing paid ways for businesses to participate.”
The three categories of augmented reality content
Zuckerberg imagines AR experiences breaking down into three major categories: augmenting objects in the physical world with additional information like putting directions atop the street or a restaurant’s reviews on their storefront, layering new virtual objects onto reality like a chess board or working television and enhancing objects that are already there with extra effects, like giving you a glitter beard or adding a castle turret to your apartment building.
Zuckerberg tells me people underestimate AR as just, “Oh, a lot of people are using these primitive tools because, I don’t know, like this is what kids do now.” But he believes that as the necessary tech finally arrives and developers get creative, we’ll be doing much more than just adding fake mustaches to our faces.
Developers could make informational tools like overlaid suggestions from experts on businesses, interactive games and mind-bending art that Zuckerberg says would be impossible to create in the real world. Meanwhile, users will be able to add augmented reality notes to different specific locations, so they could tell friends their favorite dish at a restaurant.
Developers can user three inputs to trigger their augmented reality effects: Face Tracker (but not identity or facial recognition), Sensor data, such as the gyroscope and location, and Scripting APIs to pull in data from other apps and respond to user inputs in real time. Launch partners and experiences include Manchester United pulling in soccer match data and “GOOAALL!” image overlays with crowd noises, while game developer EA has built a Mass Effect-themed helmet mask that pulls in stats from your progress in the game. Nike will overlay a sweatband on your head and pull in your recent running route as an overlaid map.
AR Studio developers can also make effects for Facebook Live, with effects launching today including an overlaid viewer polling This or That feature and comment hashtags that surface a Giphy Live animated GIF the broadcaster can show on-screen.
Zuckerberg’s vision of an AR world
For now, Zuckerberg says that at Facebook, “we definitely see ourselves as more on the software and services side.” Luckily, the next phase of AR on camera phones won’t require special hardware. “We don’t need to build the phones because the phones are already there.” Yet he admits, “In the beginning of any platform, the hardware and software are not separable,” so when AR glasses technology comes of age, Facebook might end up making its own.
At that point, Zuckerberg’s grand vision sees augmented reality as an economic equalizer. “The ability to just have glasses or eventually contact lenses where you can overlay whatever information you want . . .[means] we could put a TV on the wall and have it be a $1 app instead of a $500 piece of hardware that a lot of people can’t afford. So if the glasses cost $500 or whatever it is, you’re saving a ton of money compared to all the other hardware that you would have to buy.”
And that’s where augmented reality blends with Zuckerberg’s 6,000-word manifesto on the future of community. One aspect of his letter discussed how automation could force some people out of traditional jobs and into roles more directly serving their community and creating art. AR could make the latter easier and more affordable.
“I think more people are going to be doing work and spending more of their time . . . connecting with people and expressing themselves and doing art,” Zuckerberg tells me. “And that I think dovetails very well with an AR world where a lot of what we’re going to be able to do is just about creating ideas and crafting them and sharing them. In AR, paint is free.”
The previous computing platforms were entirely digital and accessible to anyone connected to the internet. But the real world is huge — bigger than even Facebook can fill with augmented reality experiences. By opening this developer platform, Facebook doesn’t have to build everything by itself. While Snapchat might have had a headstart in bringing AR to consumers, Facebook has just mapped out how it plans to take augmented reality several steps further.
Comment