Bringing Registered AR to Soldiers, First Responders

CAMBRIDGE, MA—Megacities, defined as having a population of at least 10 million, offer a lot of benefits, but with more people comes congestion, infrastructure stressors and mobility issues. Introduce a natural disaster or terrorist attack, and densely populated cities can present real challenges for the average citizen and the professionals who serve them.

Recently Draper participated in a technology demonstration to address challenges encountered during humanitarian, first responder and military operations within urban and subterranean environments. The demonstration occurred in GPS-denied environments across New York City, featuring tunnels, transit hubs and large commercial buildings, in a program called Thunderstorm 18-2 that was sponsored by the Department of Defense and coordinated with local authorities.

Draper demonstrated an augmented reality (AR) system running on tablets that allows small soldier units to place digital elements in their view of the real world, as seen through their displays, and leave these elements as ‘virtual breadcrumbs’ visible to others using the same system. For example, after assessing a room or hallway, they could leave a ‘medical’ element in the room to alert medics, or a ‘biohazard’ element to trigger a visit from a haz-mat unit.

Augmented reality is growing rapidly and supports people in different fields such as education, design, navigation and medicine. The technology uses your phone or tablet to create a bridge between the digital world and the physical one. By superimposing artificial objects onto real-world scenes, AR creates a virtual world that can be used in a range of applications, from a decision-aid in surgery, to pop-up maps for a walking tour, or in games, like Pokémon Go.

Today’s AR technologies have limitations, however, mostly in that they aren’t designed to maintain accurate alignment, or registration, between real and computer-generated objects in unknown, umapped environments, according to Eric Jones, a human systems architect at Draper. “Registered AR means the system is fully aware of your 3D position in space, including your orientation, and is able to use that information to virtually and precisely superimpose information over the landscape. In Draper’s version, we are able to provide users tactically relevant information, fused with other data, in formats that reduce distractions and improve operational performance.”

Draper’s first demonstration of its system, called Monarch®, was demonstrated on the government-owned Android Tactical Assault Kit (ATAK). Monarch uses camera imagery and inertial measurements to calculate changes in position and orientation as the unit moves, also referred to as a form of vision‐aided navigation. As a result, accurate registration is maintained across large, dynamic areas without any prior knowledge of where or how the system will be used.

Among soldiers and first responders, there is growing interest in harnessing the benefits of rapidly advancing augmented reality capabilities. Monarch helps by keeping the user focused on their work. When needed information is in view, they do not need to refer back to paper drawings, checklists or other resources. When digital augmentations show them exactly what needs to be done, they do not need to translate instructions into action. Monarch also helps to reduce workload because it is easier to access and interpret information. Tasks take less time to learn and perform, fewer errors are made, performance gains are achieved faster and situational awareness is maintained with less effort.

Draper developed Monarch as part of an effort to help troops navigate without access to GPS signals. Monarch enables registered augmented reality over large areas of operation without GPS, active sensing or the need for known points of reference.