Why image recognition is about to transform business

At Facebook’s recent annual developer conference, Marc Zuckerberg outlined the social network’s artificial intelligence (AI) plans to “build systems that are better than people in perception.” He then demonstrated an impressive image recognition technology for the blind that can “see” what’s going on in a picture and explain it out loud.

From programs that help the visually impaired and safety features in cars that detect large animals to auto-organizing untagged photo collections and extracting business insights from socially shared pictures, the benefits of image recognition, or computer vision, are only just beginning to make their way into the world — but they’re doing so with increasing frequency and depth.

It’s busy enough that the upcoming LDV Vision Summit, an annual conference dedicated to all things visual tech, from VR and cameras to medical imaging and content analysis, is already in its third year. “The advancements in computer vision these days are creating tremendous new opportunities in analyzing images that are exponentially impacting every business vertical, from automotive to advertising to augmented reality,” says Evan Nisselson of LDV Capital, which organizes the summit.

As with other forms of AI — natural language procession, bioinformatics, gaming — the field of computer vision has benefited greatly from the expansion of open-source, deep learning technology, user-friendly programming tools and faster and more affordable computing.

Many a headline references deep learning and artificial intelligence as the next big thing, but how exactly do these different tools work, and in what ways are businesses using them to offer image tech to the world? Is Google’s TensorFlow the same thing as Facebook’s DeepFace or Microsoft’s Project Oxford? Not exactly. To help clarify things, here’s a quick breakdown of current image technology tools and how businesses are using them.

Training material: Open data

Thanks to deep learning techniques, a machine learning technique loosely modeled after the human brain, computers can be taught to accurately identify what’s in pictures faster than ever — but they need massive amounts of data to do it.

Enter ImageNet and Pascal VOC. Years in the making, these massive and free-to-anyone databases contain millions of images tagged with keywords about what’s inside the pictures — everything from cats and mountains to pizza and sports activities. These open datasets are the basis for machine learning around images (the only way computers can accurately identify cats in photos is because they have already learned what cats look like by analyzing millions of pictures tagged with the word “cat”).

Best known for its annual visual recognition challenge, ImageNet was launched by computer scientists at Stanford and Princeton in 2009 with 80,000 tagged images. It has since grown to include more than 14 million tagged images, any of which are up for grabs at any time for machine training purposes.

Powered by various universities in the U.K., Pascal VOC has fewer pictures, but each one has richer annotations. This improves the accuracy and breadth of the machine learning and, for some applications, speeds up the overall process, because it allows for the omission of cumbersome computer subtasks.

Now, everyone from Google and Facebook to startups and universities use these open source picture sets to feed their machine learning beasts, but the big technology companies have the advantage of access to millions of user-labeled images from apps such as Google Photos and Facebook. Have you ever wondered why Google and Facebook let you upload so many pictures for free? It’s because those pictures are used to train their deep learning networks to become more accurate.

Building blocks: Open-source software libraries and frameworks

Once you have the data, it’s time to build a machine that can learn from it. Enter open-source software libraries. Freely available, these frameworks serve as starting points for building machine learning systems to service different kinds of computer vision functions, from facial and emotion recognition to medical screening and large obstacle (read: deer) detection in cars. These machine learning systems are then fed pictures from ImageNet and its ilk, proprietary images (aka Google Photos) or other sources (like anonymized, indexed clinical records).

Google TensorFlow is one of the better-known libraries, if only because it was covered widely when selected parts were open sourced late last year. TensorFlow, some of which is still proprietary to Google, is used to develop many of the company’s AI initiatives, from autonomous cars and translation to Google Now and Google Photos.

But TensorFlow is hardly the first — or only — open-source framework. UC Berkeley’s Caffe has been around since 2009, and remains popular because of its ease of customizability and large community of innovators, not to mention heavy use by Pinterest and Yahoo!/Flickr. Even Google turns to Caffe for certain projects such as DeepDream.

Created in 2002, Torch is also popular, owing to its use by Facebook AI Research (FAIR), which open sourced some of its modules in early 2015. Some of these tools are optimized to run on more than one graphics processor or computer to amplify capacity and speed up the deep learning process. Similarly, NVIDIA’s cuDNN is an open-source software library that optimizes a computer’s graphics processing unit (GPU) performance, making machine learning even faster.

These tools, while flexible and robust, require teams of computer vision engineers and hardware, so only companies that want to make computer vision a major part of their product strategy, where they’d want to own the software, need apply.

Ready-to-wear: Hosted APIs

Not every company has the resources, or wants to invest in the resources, to build out a computer vision engineering team. Even if you’ve found the right team, it can be a lot of work to get it just right, which is where hosted API services come in. Carried out in the cloud, these solutions offer menus of out-of-the-box image recognition services that can be easily integrated with an existing app or used to build out a specific feature or an entire business.

Say the Travel Channel needs “landmark detection” to show relevant photos on landing pages for specific landmarks, or eHarmony wants to filter out “unsafe” profile images uploaded by their users. Neither of these companies needs or wants to get into the deep learning image recognition development business, but can still benefit from its capabilities.

Google Cloud Vision, for example, offers a series of image detection services from facial and optical character recognition (text) to landmark and explicit content detection, and charges on a per-photo basis. Microsoft Cognitive Services (née Project Oxford) offers a collection of visual image recognition APIs, including emotion, celebrity and face detection, and charges a specific rate per 1,000 transactions. Meanwhile, startups like Clarifai offer computer vision APIs that help companies organize their content, filter out unsafe user-generated images and videos and make purchasing recommendations based on viewed or taken photos.

Custom computer vision technology

Of course, it doesn’t have to be apples or oranges. Computer vision engineering teams don’t need to be Google-sized, and companies big and small that don’t want to build their own AI systems may still want robust, custom image recognition solutions. If a beauty or cosmetics company wants to find, say, pictures of people with high-volume hair to serve ads about body-minimizing shampoo, it’ll need someone to create a custom algorithm to search for high-volume hair, since that isn’t the first thing that the more commoditized solutions offer out of the box.

Same with logos or car make and model, which are still niche commercial applications that currently aren’t available in the open-source arena. And if a closed dataset isn’t readily available, no matter, because a good percentage of the images shared on social media these days are public, anyway, making for a rich source of images with which to feed the machine learning beast.

Some companies use combinations of open data and open-source frameworks, as long as they have a team of engineers, or they might just use hosted APIs if computer vision is not something on which they are staking their entire business.

And for companies with a wide range of very specific needs, there are custom solutions. No matter how it’s approached, though, it’s clear that image recognition rarely exists in isolation; it’s made stronger by access to more and more pictures, real-time big data, unique applications and speed. The businesses that make the most of these connections are the ones that will be best poised for success.