Over the past few years, computer vision applications have become ubiquitous. From phones that recognize the faces of their users, to cars that drive themselves, to satellites that track ship movements, the value of computer vision has never been clear.
But hardware shortages and labor disruptions in the pandemic’s wake are challenging companies’ ability to make good on the promise of computer vision, even as the pandemic itself has accelerated the potential of its use cases.
Following is a look at how companies across a range of industries are deploying computer vision to improve and optimize key business processes, from retail fulfillment to health-care diagnostics.
What is computer vision?
Computer vision is a field of artificial intelligence that is focused on processing images and videos to extract meaningful information. Examples of computer vision in action include optical character recognition, image recognition, pattern recognition, facial recognition, and object detection and classification.
Industries that make heavy use of computer vision include manufacturing, healthcare, automotive, agriculture, and logistics and supply chain. In enterprises, top drivers for deploying computer vision include automation, process improvement and productivity, and regulatory compliance and safety.
“The market is growing so fast that it’s hard to keep tabs on it,” says IDC analyst Matt Arcaro, adding that the pandemic has accelerated computer vision adoption — for example, for monitoring occupancy to help ensure social distancing or to keep track of how many people were using public transit.
“Because there are plenty of CCTV cameras in place, it’s an elegant upgrade” to incorporate computer vision, Arcaro says. “And, in many cases, due to government mandates or organizational choices, the investment dollars have been there.”
According to IDC, the total worldwide market for computer vision technologies will grow to $2.1 billion this year, from $760 million in 2020, with a compound annual growth rate of 57% expected through 2025, to a total market value of $7.2 billion.
Most of this market is currently on-prem, but IDC expects public cloud deployments to account for 48% of computer vision spending by 2025.
Scaling and expediting retail fulfillment and delivery
The retail industry has seen dramatic disruption during the pandemic, with customers moving more of their shopping online and increasingly switching to home delivery.
Walmart, for example, reported that the number of shoppers getting their shopping delivered increased six times compared to before the pandemic. To meet the challenge, the multinational chain of hypermarkets increased pickup and delivery capacity by 20% last year and plans to increase it another 35% this year.
To make this happen, Walmart is investing in several categories of technology equipped with computer vision, including drones and autonomous vehicles. The company last July announced plans to roll out robots from Symbotic in 25 of its 42 regional distribution centers. The robots use computer vision, among other AI technologies, to move freight around warehouses.
Meanwhile, American supermarket chain Kroger has been investing in micro-fulfillment centers — small-scale, heavily-automated distribution warehouses located close to where customers live. The goal is to deliver groceries to customers in as little as 30 minutes, according to the company. Since last summer, Kroger has opened facilities in Florida, Alabama, Texas, California, Ohio, and Georgia, and has plans to open 17 more facilities, including both hubs and spokes, over the next 24 months.
At a hub site, more than 1,000 bots “whizz around giant 3D grids, orchestrated by proprietary air-traffic control systems,” according to the company. Instead of moving around entire pallets of products, as happens in a regional distribution center, here the robots fetch individual items. Computer vision is used to sort and pack items so that, for example, heavy items are at the bottom and bags are evenly weighted.
On-demand retail company Fabric, which specializes in micro-fulfillment centers for use by retailers that can’t build their own, uses automation extensively in its facilities, says co-founder Ori Avraham. “We use computer vision as a key capability of our robotic solution,” he says. “For example, robots’ accurate navigation on the floor is based on vision-based analysis of floor stickers. This process happens in real-time as part of the robot navigation.”
The robotic picking arms use computer vision as well, he says. “For that, we use a segmentation and classification algorithm to allow us to pick and place items. Both of these capabilities are crucial to our ability to operate our micro fulfillment centers successfully.”
Last month, Fabric opened a new micro-fulfillment center in Dallas, adding to its existing operations in New York; Washington, DC; and Tel Aviv. It has partnerships with Walmart, Instacart, and FreshDirect and plans to double its network of micro-fulfillment centers by the end of the year.
Streamlining and improving manufacturing processes
Manufacturing is another industry being revolutionized by computer vision, which is used extensively on production lines to inspect products, automate processes, and optimize productivity.
Mike Griffin, chief data scientist at Insight, a Tempe, Ariz.-based technology consulting firm, has worked with several manufacturing clients on computer vision projects. One partnership involved developing a system in which a handheld device could be used to take a photograph of a bin of products and automatically provide a count of the number of products in the bin.
“[The client] wanted to be able to hire people with disabilities to do counting,” Griffin says. “It sounds like an easy [system to develop], but the challenge there is the vision application has to do more than interpret what it can see, but it also has to interrupt what it can’t see.”
Products might be stacked on top of each other, hiding those at the bottom from view. So the computer vision system had to take a two-dimensional image and translate it into a three-dimensional model. “We needed to be at least 80% accurate on our inventory, including boxes wrapped in clear plastic with a lot of glare on them,” says Griffin.
To train the system, employees walked with cell phones and took videos. Then an intern manually labeled 500 images taken from those videos, containing 30,000 boxes. So few images were required because computer vision is a relatively mature area of artificial intelligence, with many pre-trained models. For example, to create a new model for a custom data set, like boxes, transfer learning is used.
“We’ll take a model that’s been trained on millions of images of cats and dogs and cars and whatnot,” Griffin says. “So a lot of the hard work has already been done. And then we can add our 500 images of boxes or 1,000 images of tires to that model and retrain it with that additional set of images.”
Transfer learning allows for faster model training, with smaller data sets, than otherwise possible. “You can also create synthetic data,” Griffin adds. “For example, a construction company wanted to identify hazards and they only had a couple of hundred training images. We created additional images, putting those orange hazard cones in, say, a field, or a parking lot, to augment their set of images to boost that training.”
Another innovative use of image processing in manufacturing is to translate testing data into images and then use machine learning on the generated images.
“Test failures can be near each other but it’s not obvious that they’re related to each other until you translate that data into images,” Griffin says. “They’re near each other in the test space, as opposed to being near each other in physical space.”
Improving healthcare diagnostics
In healthcare, computer vision is used extensively in diagnostics, such as in AI-powered image and video interpretation. It is also used to monitor patients for safety, and to improve healthcare operations, says Gartner analyst Tuong Nguyen.
“The potential for computer vision is enormous,” he says. “It’s basically helping machines make sense of the world. The applications are infinite — really, anything you need to see. The entire world.”
According to the fourth annual Optum survey on AI in healthcare, released at the end of 2021, 98% of healthcare organizations either already have an AI strategy or are planning to implement one, and 99% of healthcare leaders believe AI can be trusted for use in health care.
Medical image interpretation was one of the top three areas cited by survey respondents where AI can be used to improve patient outcomes. The other two areas, virtual patient care and medical diagnosis, are also ripe for computer vision.
Take, for example, idiopathic pulmonary fibrosis, a deadly lung disease that affects hundreds of thousands of people worldwide. The disease has no known cause or cure and is very difficult to diagnose. In the US alone, about 40,000 people die from the disease every year.
According to PwC, it typically takes more than two years for idiopathic pulmonary fibrosis to be diagnosed; by then, the average life expectancy of those finally diagnosed is just three to five years.
The Open Source Imaging Consortium Data Repository, supported by PwC and Microsoft, is building a platform to share anonymized imaging data to help with diagnosing the disease. By the end of this year, the organization expects to have 15,000 scans in its database.
With AI and machine learning, doctors can diagnose the disease faster and more accurately, giving them more time to treat patients.
And, in the future, the same platform can also be used for other rare diseases.
Other industries being disrupted by computer vision
In the automotive sector, computer vision is used to assist drivers and to monitor drivers to ensure they are paying attention to the road. It’s also key to enabling self-driving cars, a major growth engine for the use of computer vision in the automotive industry, says IDC’s Aracaro.
But there is another key market for autonomous driving, and computer vision in general, says Arcaro: Agriculture. “John Deere is doing something really critical there,” he says, noting that computer vision is also being used in agriculture to sort products, to monitor plant and animal health, and to monitor and manage agricultural assets.
In cybersecurity, image analytics can be used to read signatures or spot phishing websites that are designed to look similar to real websites — but different enough to evade other detection methods.
In the hospitality industry, computer vision helps track where guests go while onboard cruise ships in order to improve their experience.
In the financial services industry, image processing captures data from documents to improve efficiency of business processes.
“[Computer vision] spans almost every industry,” says Dinesh Batra, vice president of data and artificial intelligence at Capgemini Invent. “It has been a hugely successful tool for enterprises in recent years — and its prominence will only continue to accelerate.”
Visibly bright future
And yet despite the abundance of use cases already employed, computer visionhas significant room for growth.
“It’s still early days,” says Gartner’s Nguyen. “I expect to see more vendors show up in this space addressing different elements of the value chain. There is still a lot of opportunity to come as the technology gets better, more affordable, and more accessible. We’ll start seeing it used anywhere and everywhere.”
It’s not all smooth sailing, however. According to Gartner, obstacles to adoption include hardware shortages and lack of processing capabilities. In some applications, there are still issues with accuracy. Computer vision systems also need to be integrated into the production lines as well as back-end systems, both of which can be a challenge.
So while COVID-19 has increased the demand and potential for computer vision in business, the attendant hardware shortages and labor disruptions caused in the pandemic’s wake have made it difficult for many enterprises thus far to capitalize on the promise of the technology.
But as those issues abate in the future, companies will assuredly be primed for giving the technology a close look.
