10/16/2018 09:05 AM EDT

Imagine a first responder answering the call to a natural disaster, a house fire, or an active shooter incident where there may be multiple injuries and unknown casualties. The information the responder needs to fulfill the mission is immeasurable. When you also consider the volume of data they receive from other responders, dispatch, command centers, victims, and onlookers while receiving and relaying information to medical personnel, it becomes clear that responders have to synthesize a large amount of life-saving information in a short amount of time. This can lead to information overload.

The Department of Homeland Security (DHS) Science and Technology Directorate (S&T)’s Next Generation First Responder Apex program (NGFR) partnered with the NASA Jet Propulsion Laboratory (JPL) to address this capability gap and develop the Assistant for Understanding Data through Reasoning, Extraction, and Synthesis (AUDREY). AUDREY is a state-of-the-art human-like reasoning system designed to assist first responders in synthesizing high-level data while at the scene of an emergency. DHS S&T began the AUDREY software pilot at the Multi Agency Communications Center (MACC) in Grant County, Washington, in the fall of 2017 and will pilot AUDREY at the Hastings-Quinte Paramedic Services in Ontario, Canada in early 2019.

Similar to the voice-activated device on a smartphone or the voice-controlled intelligent personal assistant available to the public, AUDREY is personalized to the individual responder and has the capability to recognize first responder specialized language. However, unlike the artificial general intelligence (AGI) systems available to the general public, AUDREY uses bio-inspired neural symbolic processing for cognitive reasoning. In other words, AUDREY leverages human intelligence and collects data to achieve better machine intelligence and provides insight that first responders may not have in the crucial moments of an emergency.

“AUDREY learns, analyzes, reasons, predicts, collaborates, and provides data fusion to provide direction for first responders on the scene,” said Dr. Edward Chow, manager of the Civil Programs Office at NASA JPL. “In other words, AUDREY has the potential to serve as a sort of guardian angel for first responders while responding to an emergency.”

As demonstrated in Next Generation First Responder video released last spring, AUDREY provides situational awareness during an incident, connecting first responders across different agencies with vital information right at their fingertips.

“AUDREY provides the kind of information at an incident that may not be readily apparent to even the most seasoned first responder,” said S&T’s NGFR Director John Merrill, DHS S&T AUDREY program manager.

“AUDREY’s purpose is to aid responders in taking all of the pertinent data related to an incident and making quicker decisions. In turn, this not only helps first responders save lives, but also keeps them better protected,” Merrill concluded.

At the MACC in Grant County, Washington, Technical Services Manager Dean Hane anticipates AUDREY will aid in caller and data information gathering as well as serve as a tool in synthesizing caller information from text-to-911.

“The major priority for dispatchers across the nation is to get a call out faster and more accurately to our first responders out in the field,” said Hane. “We believe AUDREY will be a tool we’ll use to gather information and data collection processes and to speed up our response.”

With the onset of text-to-911 services available in many dispatch centers across the country, many in the industry believe the future of 911 dispatch will be phone applications developed for the public to report more detailed information from an incident. Currently, the MACC offers the text-to-911 service and anticipates that evolution.

“Certainly, there will be a variety of unanticipated standards and data that will be collected from that kind of technology that we would need a tool like AUDREY to synthesize,” said Hane. “AUDREY will be that peek into the future – and we’re excited that we’ll be at the onset to crack open that door.”

Hane explained that while many are excited about the prospect of AUDREY’s ability to streamline dispatch, some are apprehensive that AGI may eventually take their jobs. However, Hanes assures, “Dispatchers don’t need to worry about AGI taking their jobs.

“Nothing can ever take the place of human intuition. There are instances where a caller may disguise their distress, in the example of domestic abuse. Trained dispatchers are able to distinguish cues that AI cannot,” Merrill stressed.

“AUDREY was created to learn with first responders and supplement their decisions while out in the field. There is no replacement for human intuition,” said Merrill.

Currently, the MACC is in the midst of gathering data to input into the AUDREY platform for their demonstration in the fall.

For Doug Socha, chief paramedic with the Hastings-Quinte Paramedic services, piloting a system like AUDREY seems like the natural next step in next generation first responder technology.

“Our services have been very progressive in advancing technology and trying to support paramedics in providing the best patient outcome possible. We’re always looking to try to advance tools and give paramedics the ability to do their job in those highly critical situations,” said Socha.

Socha was introduced to AUDREY through the established partnership between DHS S&T and Defence Research and Development Canada (DRDC) during the CAUSE IV interoperability experiment.

“When I first heard about AUDREY’s capabilities to support responders with instant decision-making, I saw an opportunity where this type of AI can benefit paramedics from a health care system point of view,” said Socha. “The ability to have paramedics check on medication or relay information to hospitals that could save them time, can help paramedics focus more on patient care.”

Socha explained that often, paramedics’ ability to treat patients on the spot is limited because they must call and consult physicians before taking the next steps in treatment.

“We lose crucial seconds when we could be saving lives,” said Socha.

Grant County’s Multi Agency Communication Center (MACC) will provide feedback and demo AUDREY’s integration with first responder communications by late fall 2018. The AUDREY pilot with Hastings-Quinte paramedic services will begin in early 2019.

Topics: First Responders, Science and Technology
Keywords: Artificial Intelligence, data, First Responder, News, R&D, Science and Technology