Augmented reality for first responders nets $1.8M grant

Dr. Regis Kopper has received $1.8 million in National Institutes of Standards and Technology (NIST) funding to develop augmented reality (AR) interfaces for use by first responders. 

“AR displays information and images over a person’s real-world vision – think Google Glass meets Pokemon Go – and is the next technological frontier for improving public safety,” says Kopper.

“It will give EMS personnel, firefighters, and law enforcement officers access to critical data when they need it the most, while they are in action in the field.”

As AR technology develops, the government and public safety sector are preparing for a future in which AR devices will be accessible to first responders. Those preparations have included a significant stream of funding to support the development of AR public safety user interfaces, to use with the coming technology. 

In previous NIST-funded projects, Kopper and his collaborators have developed user interfaces that Wake County EMS, Durham Fire Department, and Hillsborough Police Department personnel tested in virtual reality environments, which simulated soon-to-be-available AR technology. The tests included interfaces for firefighters who need directions in a burning building, for police officers accessing driver information during traffic stops, and for EMS personnel facing complex medical cases. 

Now, Kopper’s team will adapt these public safety interfaces – and develop more – for use in current AR devices. 

“Some of the devices we’ll test are not yet available commercially,” says Kopper. “We’ll be working with AR tech that uses spatial data, instead of heads-up units that simply display 2D information. This spatial AR tech is not yet ruggedized enough for widespread use, but prototypes are available for testing. For example, the manufacturing and logistics industries have begun experimenting with some in the form of hands-free devices that help employees conduct their tasks in real time.”

For the new three-year project, Kopper’s team will work closely with public safety personnel and consultants to create a database of common public safety tasks that can be improved by AR tech.

They researchers will use that data to develop a high-value series of AR public safety “modules,” which they will test with the assistance of first responders.

“We want to create plug-and-play modules that work across a variety of situations first responders encounter, to increase situational awareness, reduce uncertainty, and improve decision making,” says Kopper. 

“The aim is to decrease response times, costs, and risks to personnel and the public.” 

Kopper’s partners on the FirstModulAR project are UNCG computer science colleague Dr. Jeronimo Grandi and NextGen Interactions, a Raleigh-based company specializing in virtual reality and augmented reality experiences. Other collaborators include the Hillsborough Police Department and the NC Department of Information Technology’s FirstTech program.  

Article by Sangeetha Shivaji and Elizabeth Keri, photography by Martin W. Kane