- This project aims to develop a framework for detecting and identifying adversarial intentions performed by groups of multiple coordinated agents (red) against an own team (blue), consisting of either one or multiple agents. Assuming an environment with both adversarial as well as neutral agents, a key challenge is to properly disambiguate between malicious and benign intentions. In addition, maneuvers that may initially appear benign could actually be deceptive actions aimed at diverting attention from a malicious intent. In prior ONR-funded research, we developed Hidden Markov Model (HMM)-based capabilities for recognizing intentions of single agents, from overt behaviors, with agents reduced to a simplistic point form representation. In this project, we aim to expand our work along several directions in order to: 1) design a more realistic agent model that includes systems for sensing and defense, and their structured coverage areas, 2) recognize intent in the presence of coordinated groups/swarms of adversarial agents, 3) enable detection of both overt and deceptive intent, and 4) provide recommendations for actions that minimize/reduce potential threats.
- Our solution relies on building and maintaining a dynamic threat heatmap which represents future potential threats to the blue team, computed as described below. Blue agents are equipped with a set of sensor and defense systems and their structured coverage areas, which can be dynamically enabled or (re-)positioned during the mission. A particular state of these systems determines varying levels of safety around the agent (e.g. areas covered/not covered by sensors/defenses). Additional input, such as terrain layout and obstacles, may also be integrated seamlessly to model safe/unsafe zones. We plan to encode these in a vulnerability heatmap where high values indicate raised levels of weakness.
- We assume that adversarial agents act in order to exploit vulnerable zones of the blue team. Based on observed position, speed, and orientation, we compute, for all non-blue agents in a local neighborhood, the closest point of approach (CPA) with respect to the blue team (including CPA position and CPA time). This information is used to update an action heatmap as follows: for every cell in the map, each detected CPA provides a contribution to the heat map, discounted with distance and time. The resulting map encodes hot spots around the blue team where potentially threatening intentions may occur at a future time. The map also records the specific agents that contributed to each cell in the map, later helping with the investigation of potential threats and their sources.
- We integrate the vulnerability and action heatmaps in a threat heatmap that provides a compact and scalable model, where high values indicate a raised potential level of threat. The model takes into account coordinated future presence of agents in certain zones, the vulnerability of those zones and the severity of potential threats adjusted by spatial (distance) and temporal (near vs. distant future) factors. The approach can be seamlessly extended to a team of blue agents by combining their individual heatmaps. We will design tools to analyze the threat heatmap in order to 1) detect areas of potential threats from heatmap hotspots, and 2) identify relevant adversarial agents from current/past contributions to the heatmap, along with their specific intents inferred from individual navigation behaviors modeled as HMMs.
- The blue agents are equipped with two policies: 1) a mission control policy that specifies the goals to be achieved (e.g., navigation to specific goal, formation, etc.) and 2) an action response policy that provides standard responses to detected threats/abnormal situations (e.g., allocate resources to sensors/defenses, exhibit warnings, change maneuvers, re-organize the team). Different potential actions from the blue response policy can be evaluated to assess their outcomes and recommend actions that minimize threat: different response actions would result in different threat heatmaps, thus exposing potentially deceptive actions in which taking a response (as expected by the adversary) may lead to higher vulnerability or threat.
- Coming soon, we just started.
- Intent Recognition for Adversarial Groups using Dynamic, Predictive Threat Heatmaps, Office of Naval Research, PI (Co-PI: Mircea Nicolescu), Amount: $585,858, April 2021 - March 2024