Augmented Reality Takes Agencies to Unexpected Places

Exciting Changes Across the Federal Government

Across the federal government, agencies already use existing commercial technology to bring AR to life in a range of emerging use cases. In 2016, for example, NASA used HTC Vive to help build an accurate simulation of the International Space Station.

Schmid envisions using AR to facilitate private medical or psychological conferences from a distance. It might enable Earth-based family members to “visit” astronauts in space and could open up new experiences for VIPs and the general public.

“It’s a new form of human exploration where you can be in two places at the same time, even in a place that’s very extreme and remote,” he says.

The U.S. Army, no stranger to extreme locations, recently gave a big vote of confidence to augmented reality. It approved a task order with Microsoft for continued development of an AR goggle based on the company’s commercial HoloLens, as part of what is expected to be a $22 billion program.

“We must modernize the Army, and this requires transformational change,” says Gen. James C. McConville, chief of staff of the Army. “Augmented reality is going to fundamentally change the way our soldiers operate on the battlefield.”

In early demonstrations of AR, the Army had users drop points on a virtual map to help them track their pathways and simultaneously communicate with other team members. The Army also enabled users to build virtual maps to model complex terrain.

“In training and in execution, AR gives us the ability to see the battlefield and understand it in ways we have not been able to before,” McConville says.

The Integrated Visual Augmentation System (IVAS), as the program is known, will enable soldiers “to visualize, train, rehearse and operate in an environment in real time.

“It also gives the ability to provide information to our soldiers from integrated battle command centers, passing data quickly between ground, air and other sensors and giving us better awareness of the battlefield — more than we’ve ever seen before,” he adds.

For the Occupational Safety and Health Administration, the technology also supports new modes of training. “Whenever you teach somebody something, it’s one thing to tell them, it’s one thing to show them, and it’s another thing to let them do it,” says OSHA spokesperson Denisha Braxton.

The agency uses AR to protect workers tasked with inspecting unsafe, high-hazard areas. “For example, trench collapses are a leading cause of death in the construction industry,” she says. With AR, “we can actually go into a trench and excavation environment that’s as close as they’re going to get in the field.”

“HoloLens’ realistic and immersive graphics help inspectors evaluate variables at a trench worksite — such as ground conditions, hazards, safety equipment and best practices for mitigating risk — without jeopardizing their own safety,” she says.

The VA Is Leaning into Tech for More Advanced Training

The Department of Veterans Affairs has used the HoloLens 2 headset and specialized anatomy software to help medical trainees get a simulated hands-on experience.

“Imagine you’re a medical student or a nursing student trying to understand the complexities of three-dimensional anatomy from a flat textbook. It’s not easy to do,” says Dr. Thomas Osborne, the director of the VA’s National Center for Collaborative Healthcare Innovation.

With AR, “we have this very complex, very detailed, three-dimensional interactive holographic atlas of human anatomy, which has all the layers, and you can take them apart and put them back together again,” he says.

“You can also make the hologram super big, two stories tall, which allows you to walk into structures like the heart and look around.”

Virtual presence promises to dramatically expand learning opportunities. “We have had people in different parts of the country collaborating in the same virtual room at the same time,” Osborne says.

AR can also make available a wider range of learning experiences. Trainees don’t always have access to a cadaver lab to learn from an actual body, he says, and even when they do, they get only one shot at practicing a procedure. With AR, “you can take things apart, label them, put them back together, then reset it and start over again.”

Virtual collaboration became essential for scientists around the world who use Argonne National Laboratory’s (ANL) unique Advanced Photon Source facility. The COVID-19 pandemic threatened to bring their intricate experiments to a halt.

“Everything was shut down, there was no travel, yet the experiments had to be carried out,” says Lead Beamline Scientist Olaf Borkiewicz. Real-time, collaborative experiences were critical to ensuring the work continued. “These are very complex experiments, and we need users’ input.”

HoloLens offered “a live conversation and live video chat where they could see everything that we saw,” he says. “We used the device to talk to the users, to get their input on the experimental setup, to show them how the setup looks and to make adjustments.”

ANL has also tapped HoloLens for educational purposes, using the device to give tours to high school and college students and even Department of Energy officials, Borkiewicz says. “It is a great communication tool.”

LEARN MORE: How artificial intelligence is shaping remote work.

The VA Is Looking to the Future

At the VA, officials see a promising future for expanded AR efforts. “There’s opportunity for us to continue to grow and learn from other healthcare systems,” says Anne Lord Bailey, the agency’s immersive technology lead for the Office of Healthcare Innovation and Learning.

“The ability to provide a digital overlay onto your real world that you can interact with is really powerful and helps us continue to expand the work that we do,” she says.

NASA is looking to share its emerging AR expertise across other agencies. “Federal agencies have many different missions, and we can use our tools and our expertise and our vision to help them come up with new uses,” Schmid says.

“Our novel idea was to take AR somewhere no humans had been taken before, and we can do that within federal government and other federal agencies,” he says. In fact, he adds, there are about 15 federal agencies that want to team up with NASA for projects such as medical simulations and natural disaster response.

Agencies wanting to move the needle on AR adoption may need to consider not just headsets but also network upgrades.

For simple AR use cases and smaller images, “you can use this type of technology without needing advanced infrastructure like 5G,” Osborne says. However, for more complex uses, with more extensive data and processing demands, “you often need to have bigger connectivity pipes to move more data faster and more efficiently.”

Overall, the federal government “may want to explore taking steps to allocate the necessary resources such as time, money and bandwidth,” Microsoft’s Evers says.

The Army’s McConville says agencies will need to keep their eyes on the ball as they work to bring AR to life. “When you’re trying to do new things, it’s hard,” he says. “You must continue to push through new problems to find innovative solutions. You have to push forward through the challenges so you don’t lose momentum.”

EXPLORE: The Marine Corps is using the Metaverse for training purposes.