- Understanding intent is an important aspect of communication among people and is an essential component of the human cognitive system. This capability is particularly relevant for situations that involve collaboration among multiple agents. In particular, for collaboration between umans and robots, having the robots detect the intent of human actions enables the use of implicit communication, typical for collaborations between humans, and enhances the quality of humans' interaction with robots.
- The goal of this project is to design an integrated system for automatic behavior modeling, which provides effective detection of intentions, before any actions have been finalized. Our approach relies on a novel formulation of Hidden Markov Models (HMMs), which allows a robot to understand the intent of others by virtually assuming their place and projecting their potential intentions based on the current situation. In contrast with the standard use of HMMs, which solely model transitions between discrete states, in this work the HMMs also model the rate and direction of change for a set of relevant parameters. This approach allows the system to recognize the intent of observed actions before the action has been completed, thus enabling the system to take appropriate actions to enhance the collaboration. The system's capability to observe and analyze the current scene employs novel vision-based techniques for target detection and tracking, using a non-parametric recursive modeling approach.
- Mohammad Saffar, Mircea Nicolescu, Monica Nicolescu, Banafsheh Rekabdar, “Intent Understanding Using an Activation Spreading Architecture,” Robotics – Special Issue on Representations and Reasoning for Robotics, vol. 4, no. 3, pages 284-315, July 2015.
- Mohammad Saffar, Mircea Nicolescu, Monica Nicolescu, Banafsheh Rekabdar, "Context-Based Intent Understanding Using an Activation Spreading Architecture", Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1-8, Hamburg, Germany, September 2015.
- Richard Kelley, Alireza Tavakkoli, Chris King, Amol Ambardekar, Liesl Wigand, Monica Nicolescu, Mircea Nicolescu, "Intent Recognition for Human-Robot Interaction", in Plan, Activity, and Intent Recognition, Gita Sukthankar, Christopher Geib, Hung Hai Bui, David Pynadath and Robert Goldman Editors, Elsevier, pp. 343-365, 2013. [Link]
- Richard Kelley, Alireza Tavakkoli, Chris King, Amol Ambardekar, Monica Nicolescu, Mircea Nicolescu, "Context-Based Bayesian Intent Recognition", in IEEE Transactions on Autonomous Mental Development, 4(3), 215-225, 2012. [PDF] [LINK]
- Richard Kelley, Liesl Wigand, Brian Hamilton, Katie Browne, Monica Nicolescu, Mircea Nicolescu, "Deep Networks for Predicting Human Intent with Respect to Objects", in Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, Boston, MA, 2012. [Link]
- Richard Kelley, Amol Ambardekar, Liesl Wigand, Monica Nicolescu, Mircea Nicolescu, "Point Clouds and Range Images for Intent Recognition and Human-Robot Interaction", in Proceedings of the Advanced Reasoning with Depth Cameras Workshop (in conjunction with the Robotics: Science and Systems Conference), Los Angeles, California, June 2011. [PDF]
- Richard Kelley, Christopher King, Amol Ambardekar, Monica Nicolescu, Mircea Nicolescu and Alireza Tavakkoli, "Integrating Context into Intent Recognition Systems", in the "7th International Conference on Informatics in Control, Automation and Robotics", pages 315-320, Funchal, Madeira, Portugal, June, 2010. [PDF]
- Richard Kelley, Alireza Tavakkoli, Christopher King, Monica Nicolescu, Mircea Nicolescu, "Understanding Activities and Intentions for Human-Robot Interaction", in Advances in Human-Robot Interaction, Daisuke Chugo (editor), In-Tech, pages 288-305, February 2010. [PDF]
- Richard Kelley, Monica Nicolescu, Mircea Nicolescu, Sushil Louis, "An Evolutionary Approach to Maximum Likelihood Estimation for Generative Stochastic Models", the 40th International Symposium on Robotics, Barcelona, Spain, March 10-13, 2009. [PDF]
- Richard Kelley, Christopher King, Alireza Tavakkoli, Mircea Nicolescu, Monica Nicolescu, George Bebis, "An Architecture for Understanding Intent Using a Novel Hidden Markov Formulation", International Journal of Humanoid Robotics, Special Issue on Cognitive Humanoid Robots, vol. 5, no. 2, pages 1-22, 2008. [PDF]
- Richard Kelley, Alireza Tavakkoli, Christopher King, Monica Nicolescu, Mircea Nicolescu, George Bebis, "Understanding Human Intentions via Hidden Markov Models in Autonomous Mobile Robots", Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, pages 367-374, Amsterdam, Netherlands, March 2008. [PDF]
- Alireza Tavakkoli, Richard Kelley, Christopher King, Mircea Nicolescu, Monica Nicolescu, George Bebis, "A Vision-Based Architecture for Intent Recognition", Proceedings of the International Symposium on Visual Computing, pages 173-182, Lake Tahoe, Nevada, November 2007. [PDF]
Intent disambiguation from context and temporal constraints [PR2 robot collaboration on making tea]
Intent disambiguation from context and temporal constraints [Baxter robot collaboration on making tea]
Intent disambiguation from context [robot responses based on perceived intent; preparing tea]
Intent disambiguation from context [robot responses based on perceived intent; preparing salad]
Intent disambiguation from context in an activation spreading architecture [robot responses based on perceived intent]
Robot interacting with a human user based on detected intentions in a "homework" scenario
Robot interacting with a human user based on detected intentions in an "eating" scenario
Robot interacting with a human user based on detected intentions (history of past events is used as context)
- Robot interacting with a human user based on detected intentions (history of past events is used context for disambiguation of high-level intentions) [MOV] (48.9Mb)
- Detecting a human's intentions in a "homework" scenario [MOV] (9.8Mb)
- Detecting a human's intentions in an "eating" scenario [MOV] (10.3Mb)
- Robot responds to detecting a theft [MOV] (4.4Mb)
- Robot responds to detecting a person leaving an unattended baggage [MOV] (2.4Mb)
- Robot interacting with a human user based on detected intentions [MOV] (66.3Mb)
- Detecting meeting, passing, following of two people; the system detects the transitions between the activities (sequence 1) [MOV] (4.6Mb)
- Detecting meeting, passing, following of two people; the system detects the transitions between the activities (sequence 2) [MOV] (4.9Mb)
- Understanding Intent Using an Activation Spreading Architecture, Office of Naval Research, PI (Co-PI: Mircea Nicolescu, Sushil Louis), Amount: $590,992, July 1, 2012 - June 30, 2015.
- Context-Based Intent Understanding for Autonomous Systems in Naval and Collaborative Robotics Applications, Office of Naval Research, PI (Co-PI: Mircea Nicolescu, Sushil Louis), Amount $543,280, August 1, 2009 - July 31, 2012.
- Understanding Intent Using a Novel Hidden-Markov Representation, Office of Naval Research (ONR), PI, (Co-PI: Mircea Nicolescu), Amount: $619,584, June 1, 2006 - May 31, 2009.