It sometimes takes a surprisingly long time for technologies to catch on, despite their obvious potential. Augmented Reality (AR) is an example of this. While we’ve had the resources to support visual information overlays (think Google Glass) for nearly a decade, companies are only now beginning to discover how to take full advantage of its capabilities.
Think about how the landscape changes. At the moment, thousands of experiments are underway in what could be called augmented operations at companies around the world. One of the biggest takes place at China Southern Airlines, where the team from the company’s tech arm, China Southern Technic, has weaved together augmented reality, artificial intelligence, cloud computing, the Internet of Things and 5G connectivity into a single application that can expand human capabilities, improve safety and improve performance.
Safety 2.0
One of the first processes to benefit from CSA’s adoption of augmented reality is the safety inspection. (The recent crash of a China Eastern Airlines Boeing 737 only increases the need to use all available safety inspection tools.)
After any passenger aircraft landing anywhere in the world, a maintenance, repair, and operations (MRO) engineer must perform a thorough aircraft inspection. An inspection on a Boeing 737 usually takes more than 100 steps, and an Airbus 320 more than 200 steps. This is a fundamental yet critical part of airline management, an urgent task that frontline MRO engineers must perform every day, often several times. CSA is no exception. As the world’s third largest airline, the airline repeats this procedure more than 2,500 times a day, a task that takes CSA’s MRO engineers approximately 1,000 hours of work.
At most airports in the world, technicians check each inspection item against a voluminous paper-based job card, a block of more than 20 sheets of paper that they must hold during the entire job. Until recently, CSA’s MRO engineers also worked this way, performing this work while juggling paper, pens, walkie-talkies and the job card. But now, at 22 airports, the CSA flies out, most of the information, registration and communication tools are integrated into a single AR display. This display puts a whole range of resources at the disposal of the technicians – not just text, but also images, videos, graphs and speech, in any combination that is useful to the technicians.
While the AR glasses are expected to shave 6 percent of those 1,000 daily hours, in our research into the integration of this technology at CSA, we found that the benefits of the AR glasses go far beyond the labor dividend. They’re not just a new way of getting information, they’re a whole new way of working.
CSA’s AR glasses allow technicians to edit and reorganize their to-do list, changing the information they see and how they want it to be displayed. Their displays can be customized by aircraft, season and even individual preference. They provide the engineers with step-by-step multimedia support and immersive experiences while performing the tasks, including AI object recognition and collaboration with a third-party expert.
“Combined with what [artificial intelligence], the AR glasses can really make our job a lot easier,” said an MRO engineer. “I can now point my fingers at a place, say a lube oil cap, and it automatically recognizes the object or key parts and tells me it’s open but should be closed. It can also show me, in a photo or a short video, what the object looked like in normal condition or in its last service.” When the job is done, technicians can even sign out by voice or even gestures, if it’s too noisy on the tarmac to use a voice command.
Instead of lugging manuals the size of unabridged dictionaries or spending precious time walking to an office to consult one, technicians have instant access to the information they need through the glasses. “I no longer have to look for the maintenance manual, it can be an hour of walking back and forth. The manual is now coming to me, before my very eyes!” an engineer told us. The AR glasses even make it possible for experts to advise mechanics on the asphalt in real time and provide them with photos, videos, voice advice and graphs.
The goggles also encourage more standardized performance. “It knows where I am in the process and points me where to go next. Everyone follows the same process in the same order,” explains another engineer.
Welcome to the Augmented Operations
Wide awake engineers, better compliance, a visual diary of the life of each component and ultimately safer flights are all benefits of this single pilot project in the airline with 850 aircraft. The AR glasses optimize performance not only by bringing more knowledge closer to the machines, but also by keeping an eye on MRO. Like most previous forms of digitization, the CSA’s experience suggests that augmented operations are less likely to displace people than increase their capabilities — a win for businesses, workers and travelers.
Today, the CSA’s first improved operational system is still a work in progress, not so much in its ability to transfer data to or from the individual – although that poses challenges – as in adapting the technology to meet to the capacities of human cognition. The AR smart glasses must meet industry safety standards as well as key privacy, comfort, display, connectivity, ergonomics, battery life, noise reduction, multimedia interactivity, immersive experience combined with transparency, infrastructure requirement ( 5G , edge computing), and a knowledge graph that can provide deeper AI-enabled support.
only the beginning
And that’s just one application in one industry — imagine the many other ways the technology could be used. Thousands of companies around the world are already experimenting with different aspects of AR technologies. And we believe that number will increase dramatically once we understand more about the best ways to manage the user interface on all those smart glasses, and awareness of this new and highly customizable technology grows. It’s no different than when something called a website appeared on our desktops or a decade later when it became clear that apps were the smartphone’s killer app.
When the business use of AR technologies has its own Netscape moment, we think many industries will see the dawn of a huge new opportunity. For example, airlines will be able to understand their cost structure in much more detail than they do now, right down to the component. Ultimately, this cognitive shift could change the balance of power within the aviation industry, from sales and the front office to the back office and maintenance hangar (particularly as carbon footprint becomes more integrated into price).
And this is just the beginning. As the CSA project has shown, virtualization knows no boundaries. Any person or object in aviation operations, from mechanics to the planes or the entire airport – can be virtualized, with enough data and enough modeling. By creating a virtual representation of a physical object, plus a continuous stream of new information about its status, digital twins of physical objects and even humans can give airlines an unprecedented opportunity to see how something is currently performing and simulate or predict how it might perform in the future.
The success of CSA suggests that AR is finally becoming part of our work reality. But there are still many unanswered questions. For example, in our work for CSA, we’ve asked a lot of questions about how to get the best out of people. When do people need reminders? What are the signs that their attention is beginning to wane? What is the most efficient way for mechanics to communicate through their glasses with an expert who can guide them through a complex repair? At the moment the questions are piling up, but luckily so are the answers.
This post How one airline is using AR to improve operations was original published at “https://hbr.org/2022/04/how-one-airline-is-using-ar-to-improve-operations”
