- Spatio-temporal pattern recognition is an important problem for all biological systems. Activities like gestures, movements and detecting them by observation contain both spatial and temporal information. This information is essential in communication, collaboration and learning by observation and demonstration. Spatio-temporal patterns and gestures are extensively used in social interactions, as they carry intentional meanings of a person's own goals. While people can quickly recognize such gestures and predict others' intentions based on the observed movement patterns, the same task is more challenging for an autonomous robot system. In order to facilitate and support such interactions, which naturally happen between people, for domains in which humans and robots interact with each other, the ability for real-time, early recognition of human gestures becomes of key importance. Existing approaches to learning spatio-temporal patterns are typically supervised, offline, rely on extensive amounts of training data, require the observation of the entire pattern for recognition, cannot process variable sized input patterns, can only handle patterns in a fixed frame and they are not scale/translation invariant. In contrast, the proposed methods are mostly-unsupervised and are robustly trainable on a small training set with limited number of samples, they are able to classify variable sized inputs, at different scales/locations, they are able to early classify patterns.
- Spike timing neural networks (SNNs) are suitable methods to model spatio-temporal patterns. This projects presents an unsupervised approach for learning, recognizing and early classifying spatio-temporal patterns using spiking neural networks for human-robotic domains. The main contributions of this work are as follows: i) it requires a very small number of training examples, ii) it enables early recognition from only partial information of the pattern, iii) it learns patterns in an unsupervised manner, iv) it accepts variable sized input patterns, v) it is invariant to scale and translation, vi) it can recognize patterns in real-time and, vii) it is suitable for human-robot interaction applications and has been successfully tested on a PR2 robot.
- Banafsheh Rekabdar, Luke Fraser, Monica Nicolescu, Mircea Nicolescu, "A Real-Time Spike-Timing Classifier of Spatio-Temporal Patterns", Neurocomputing, vol. 311, pages 183-196, October 2018.
- Banafsheh Rekabdar, Monica Nicolescu, Mircea Nicolescu, Sushil Louis, "Using Patterns of Firing Neurons in Spiking Neural Networks for Learning and Early Recognition of Spatio-Temporal Patterns", Neural Computing and Applications – Special Issue on Computational Intelligence for Vision and Robotics, vol. 28, nr. 5, pages 881-897, May 2017.
- Banafsheh Rekabdar, Monica Nicolescu, Mircea Nicolescu, Mohammad Taghi Saffar, Richard Kelley, "A Scale and Translation Invariant Approach for Early Classification of Spatio-Temporal Patterns using Spiking Neural Networks", Neural Processing Letters, vol. 43, no. 2, pages 327-343, April 2016.
- Banafsheh Rekabdar, Monica Nicolescu, Richard Kelley, Mircea Nicolescu, "An Unsupervised Approach to Learning and Early Detection of Spatio-Temporal Patterns Using Spiking Neural Networks", in Journal of Intelligent and Robotic Systems, January 2015. [LINK]
- Banafsheh Rekabdar, Luke Fraser, Monica Nicolescu, Mircea Nicolescu, “Spiking Neural Network for Human Hand Gesture Recognition: A Real-Time Approach”, Proceedings of Model Learning for Human-Robot Communication – RSS Workshop, pages 1-3, Ann Arbor, Michigan, June 2016.
- Banafsheh Rekabdar, Monica Nicolescu, Mircea Nicolescu, Richard Kelley, "A Biologically Inspired Approach to Learning Spatio-Temporal Patterns", Proceedings of the International Conference on Development and Learning and on Epigenetic Robotics, pages 291-297, Providence, Rhode Island, August 2015.
- Banafsheh Rekabdar, Monica Nicolescu, Mircea Nicolescu, Richard Kelley, "Scale and Translation Invariant Learning of Spatio-Temporal Patterns using Longest Common Subsequences and Spiking Neural Networks", Proceedings of the International Joint Conference on Neural Networks, pages 3659-3666, Killarney, Ireland, July 2015.
- Banafsheh Rekabdar, Monica Nicolescu, Mircea Nicolescu, "An Unsupervised Learning Approach for Classifying Sequence Data for Human Robotic Interaction Using Spiking Neural Networks", Proceedings of the HRI Pioneers Workshop (in conjunction with the ACM/IEEE International Conference on Human-Robot Interaction), pages 213-214, Portland, Oregon, March 2015.
- Banafsheh Rekabdar, Monica Nicolescu, Richard Kelley, Mircea Nicolescu, "Unsupervised Learning of Spatio-Temporal Patterns Using Spike Timing Dependent Plasticity", to appear in the Proceedings of the Seventh Annual Conference on Artificial General Intelligence, Quebec City, Canada, August 2014. [Link]
Detecting a 0
Detecting a 1
Detecting a 2
Detecting a 3
Detecting a 4
Detecting a 5
Detecting a 6
Detecting a 7
Detecting an 8
Detecting a 9
- Understanding Intent Using an Activation Spreading Architecture, Office of Naval Research, PI (Co-PI: Mircea Nicolescu, Sushil Louis), Amount: $590,992, July 1, 2012 - June 30, 2015.