Intelligent Traffic Intersection
Video-based target detection at signized intersections has many advantages over traditional inductive loops, such as easy set up, easy definition of detection zones, easy modification of detector settings, and lower maintenance cost. However, several common problems need to be resolved, including low lighting conditions due to adverse weather conditions. The main goal of this project is to design the video-based target detection algorithm by fusing visible with infared (IR) data to improve both accuracy and robustness, especially under adverse conditions (i.e., night-time, snow, rain, fog, etc.). We describe a traffic scene background model, a background subtraction process and on-line background update process that we have developed based on the support vector regression (SVR). Following an incremental support vector regression, the developed background model is on-line updated whenever a training input of intensity values is added to the training set. Such scheme enables the developed background model to be adaptive to the intensity variance of images caused by the changes of outdoor environment (i.e. illumination changes for visible image and variance of the thermodynamic and atmospheric conditions for infrared image.)

 

Automatic Background Modeling for Visual Surveillance
The final goal for many visual surveillance systems is automatic understanding of events in a site. Higher level processing on video data requires certain lower level vision tasks to be performed. One of these tasks is the segmentation of video data into regions that correspond to objects in the scene. Issues such as automation, noise robustness, adaptation, and accuracy of the model must be addressed. Current background modeling techniques use heuristics to build a representation of the background, while it would be desirable to obtain the background model automatically. In order to increase the accuracy of modeling it needs to adapt to different parts of the same scene and finally the model has to be robust to noise.

 

Vehicle Detection  Robust and reliable vehicle detection is one of the most important issues in any in-vehicle optical system, with applications to driver assistance systems or autonomous, self-guided vehicles. Several factors make on-road vehicle detection very challenging including variability in scale, location, orientation, and pose. The focus of this project, which is funded by Ford Motor Company, is to develop a reliable real-time on-road vehicle detection system.

 

Automatic Target Recognition Using Algebraic Functions of Views  The main goal of this project is to improve the performance of Automatic Target Recognition (ATR) by developing a more powerful ATR frame work which can handle changes in the appearance of a target more efficiently and robustly. The new framework will be built around a hybrid model of appearance by integrating (1) Algebraic Functions of Views (AFoVs), a powerful mathematical model of geometric appearance, with (2) eigenspace representations, a well known empirical model of appearance which has demonstrated significant capabilities in recognizing complex objects under no occlusion. This project is sponsored by The office of Naval Research (ONR).

Surveillance  In this project, we have been developing a system that acquires and processes images through one or multiple Camera Units monitoring certain area(s) via a Local Area Network (LAN) and is capable of combining information from multiple Camera Units to obtain a consensus decision. The system can be trained to detect certain type of intrusions, for example pedestrians, a group of pedestrians, vehicles, pets etc., and minimizes false alerts due to other non-interested intrusions. As a case study, we are using this system to detect Pedestrian/Vehicle in an observation area.

 
Computer Vision Technologies for Effective Human-Computer Interaction in Virtual Environments   Developing efficient and effective virtual environment technologies to provide sophisticated training sessions for astronauts and to help ensure safety is a high priority to NASA. Our main goal in this project is to Advance NASA's virtual environment technologies by developing and demonstrating computer vision non-contact techniques for efficient and effective human-computer interaction in virtual environments. This project is funded by NASA and is a joint effort with BioVIS Lab at NASA Ames Research Center.

 

Face Detection in the Infrared Spectrum  
Face detection is an important prerequisite step for successful face recognition. The performance of previous face detection methods reported in the literature is far from perfect and deteriorates ungracefully where lighting conditions cannot be controlled. We propose a method that outperforms state-of-the-art face detection methods in environments with stable or variable lighting conditions. The approach capitalizes upon the near-IR phenomenology for skin detection.

 

Gender Classification  Computer vision systems for surveillance and human computer interaction must be able to process human faces in a variety of ways. Gender information can be used to enhance existing Human Computer Interaction (HCI) systems but also can serve as a basis for passive surveillance and control in "smart buildings" (e.g., restricting access to certain areas based on gender) and collecting valuable demographics (e.g., the number of women entering a retail store on a given day). The goal of this project is to develop a robust gender classification system. The key idea of our approach is using subset feature selection.

 

Understanding Intent Using a Novel Hidden Markov Model Representation Understanding intent is an important aspect of communication among people and is an essential component of the human cognitive system. This capability is particularly relevant for situations that involve collaboration among multiple agents or detection of situations that can pose a particular threat. For surveillance or military applications, it is highly important to enable understanding the intent of relevant agents in the environment, from their current actions, before any attack strategies are finalized. The approach relies on a novel formulation of Hidden Markov Models (HMMs), which allows a robot to understand the intent of other agents by virtually assuming their place and detecting their potential intentions based on the current situation. This allows the system to recognize the intent of observed actions before they have been completed, thus enabling preemptive actions for defense. The system's capability to observe and analyze the current scene employs novel vision-based techniques for target detection and tracking, using a non-parametric recursive modeling approach.

Relevant Publications
  • Richard Kelley, Christopher King, Alireza Tavakkoli, Mircea Nicolescu, Monica Nicolescu, George Bebis, "An Architecture for Understanding Intent Using a Novel Hidden Markov Formulation", International Journal of Humanoid Robotics - Special Issue on Cognitive Humanoid Robots, vol. 5, no. 2, pages 243-264 (in print), 2008.
  • Richard Kelley, Alireza Tavakkoli, Christopher King, Monica Nicolescu, Mircea Nicolescu, George Bebis, "Understanding Human Intentions via Hidden Markov Models in Autonomous Mobile Robots", Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, pages 367-374, Amsterdam, Netherlands, March 2008.
  • Alireza Tavakkoli, Richard Kelley, Christopher King, Mircea Nicolescu, Monica Nicolescu, George Bebis, "A Vision-Based Architecture for Intent Recognition", Proceedings of the International Symposium on Visual Computing, pages 173-182, Lake Tahoe, Nevada, November 2007.
    • Support
    This work was supported by the Office of Naval Research award N00014-06-1-0611 and by the National Science Foundation EPSCoR Ring True III award EPS0447416.

    Segmentation for Videos with Quasi-Stationary Backgrounds - A Non-Parametric Approach Video segmentation is one of the most important tasks in high-level video processing applications. Background modeling is the key to detection of foreground regions (such as moving objects - e.g., people, cars) in videos where camera is assumed to be stationary. However, possible changes in the background of the video such as waving flags, fluctuating monitors, water surfaces, etc. make it difficult to detect objects of interest in the scene. Due to the diverse nature of video applications it has been a main concern for researchers to design a general, scene-independent system. In this project we first propose a novel adaptive statistical method as a baseline system that addresses this issue. After investigating its performance we introduce a universal statistical technique which aims to overcome the weaknesses of its predecessor in modeling slow changes in the background. Finally, a new analytical technique is proposed in this work that approaches the problem of background modeling in a different direction. This technique is introduced in order to solve the limitations of statistical techniques which are bound to the accuracy of the probability density estimation. The performance of each of the proposed methods is studied and scenarios in which each of them leads to a better performance are investigated.

    Relevant Publications
  • Alireza Tavakkoli, Mircea Nicolescu, George Bebis, Monica Nicolescu, "A Support Vector Data Description Approach for Background Modeling in Videos with Quasi-Stationary Backgrounds", International Journal on Artificial Intelligence Tools, vol. 17, no. 4, pages 635-658, August 2008.
  • Alireza Tavakkoli, Mircea Nicolescu, George Bebis, Monica Nicolescu, "Non-Parametric Statistical Background Modeling for Efficient Foreground Region Detection", Machine Vision and Applications, pages 1-15, Springer-Verlag, April 2008.
  • Alireza Tavakkoli, Mircea Nicolescu, George Bebis, "A Novelty Detection Approach for Foreground Region Detection in Videos with Quasi-stationary Backgrounds", Proceedings of the International Symposium on Visual Computing, pages 40-49, Lake Tahoe, Nevada, November 2006.
  • Alireza Tavakkoli, Mircea Nicolescu, George Bebis, "Robust Recursive Learning for Foreground Region Detection in Videos with Quasi-Stationary Backgrounds", Proceedings of the International Conference on Pattern Recognition, pages 315-318, Hong Kong, August 2006.
  • Alireza Tavakkoli, Mircea Nicolescu, George Bebis, "Automatic Robust Background Modeling Using Multivariate Non-Parametric Kernel Density Estimation for Visual Surveillance", Proceedings of the International Symposium on Visual Computing, pages 363-370, Lake Tahoe, Nevada, December 2005.
    • Support
    This work was supported in part by a grant from the University of Nevada Junior Faculty Research Grant Fund and by NASA under grant NCC5-583.

    Visual Awareness and Long-Term Autonomy for Robotic Assistants A major challenge in deploying robots into the real world is the design of an architectural framework which can provide long-term, natural and effective interactions with people. Within this framework, key issues that need to be solved relate to the robots' ability to engage in interactions in a natural way, to deal with multiple users, and to be constantly aware of their surroundings. We propose a control architecture that addresses these issues. First, we endow our robot with a visual awareness mechanism, which allows it to detect when people are requesting its attention and try to engage it in interaction. Second, we provide the robot with flexibility in dealing with multiple users, such as to accommodate multiple user requests and task interruptions, over extended periods of time. In support of our robot awareness mechanism, we develop visual capabilities that allow the robot to identify multiple users, with multiple postures, in real-time, in dynamic environments in which both the robot and human users are moving. To enable long-term interaction, we design a control architecture which enables the representation of complex, sequential and hierarchical robot tasks.

    Relevant Publications
  • Christopher King, Xavier Palathingal, Monica Nicolescu, Mircea Nicolescu, "A Control Architecture for Long-Term Autonomy of Robotic Assistants", Proceedings of the International Symposium on Visual Computing, pages 375-384, Lake Tahoe, Nevada, November 2007.
  • Christopher King, Xavier Palathingal, Monica Nicolescu, Mircea Nicolescu, "A Vision-Based Architecture for Long-Term Human-Robot Interaction", Proceedings of the International Conference on Human-Computer Interaction, pages 1-6, Chamonix, France, March 2007.
    • Support
    This work was supported by the Office of Naval Research award N00014-06-1-0611.



    An Automatic Framework for Figure-Ground Segmentation in Cluttered Backgrounds Grouping processes, which "organize" given data by eliminating irrelevant items and sorting the rest into groups, each corresponding to a particular object, can provide reliable pre-processed information to higher level vision functions, such as object detection and recognition. Here we consider the problem of grouping oriented segments in highly cluttered images. We developed a general scheme based on an iterative tensor voting approach which has been shown to improve segmentation results considerably. Segments are represented as second-order tensors and communicate with each other through a voting scheme that incorporates the Gestalt principles of visual perception. The key idea of our approach is to conservatively remove background segments using multi-scale analysis, and re-vote on the retained segments. This process results in better quality segmentations, especially under severe background clutter. Particularly remarkable, our experiments reveal that using our approach as a post-processing step to the boundary detection methods evaluated with the Berkeley dataset improves the results in 84% of the grayscale test images from this benchmark.

    Relevant Publications
  • Leandro Loss, George Bebis, Mircea Nicolescu, Alexei Skurikhin, "An Iterative Multi-Scale Tensor Voting Scheme for Perceptual Grouping of Natural Shapes in Cluttered Backgrounds", Computer Vision and Image Understanding, 24 pages (in print), Elsevier, 2008.
  • Leandro Loss, George Bebis, Mircea Nicolescu, Alexei Skurikhin, "Investigating How and When Perceptual Organization Cues Improve Boundary Detection in Natural Images", Proceedings of the IEEE Computer Society Workshop on Perceptual Organization in Computer Vision (in conjunction with the IEEE Conference on Computer Vision and Pattern Recognition), pages 1-8, Anchorage, Alaska, June 2008.
  • Leandro Loss, George Bebis, Mircea Nicolescu, Alexei Skurikhin, "An Automatic Framework for Figure-Ground Segmentation in Cluttered Backgrounds", Proceedings of the British Machine Vision Conference, pages 1-10, University of Warwick, UK, September 2007.
  • Leandro Loss, George Bebis, Mircea Nicolescu, Alexei Skourikhine, "Perceptual Grouping Based on Iterative Multi-scale Tensor Voting", Proceedings of the International Symposium on Visual Computing, pages 870-881, Lake Tahoe, Nevada, November 2006.

    • Efficient Vehicle Tracking and Classification for an Automated Traffic Surveillance System We propose a new traffic surveillance system that works without prior, explicit camera calibration, and has the ability to perform surveillance tasks in real time. Camera intrinsic parameters and its position with respect to the ground plane are derived using geometric primitives common to any traffic scene. We use optical flow and knowledge of camera parameters to detect the pose of a vehicle in the 3D world. This information is employed by a model-based vehicle detection and classification technique used by our traffic surveillance application. The object (vehicle) classification uses two new techniques - color contour-based matching and gradient-based matching. Our experiments on several real traffic video sequences demonstrate good results for our foreground object detection, tracking, vehicle detection and vehicle speed estimation approaches.

    Relevant Publications
  • Amol Ambardekar, Mircea Nicolescu, George Bebis, "Efficient Vehicle Tracking and Classification for an Automated Traffic Surveillance System", Proceedings of the International Conference on Signal and Image Processing, pages 1-6, Kailua-Kona, Hawaii, August 2008.

    • Voting-Based Computational Framework for Motion Analysis Our research addresses the problem of visual motion analysis, by formulating it as a motion layers inference from a noisy and possibly sparse point set in a 4-D space. Our approach is based on a layered 4-D representation of data and a voting scheme for token communication, within a tensor voting computational framework. From a possibly sparse input consisting of identical point tokens in two frames, the image position and potential velocity of each token are encoded into a 4-D tensor. Within this 4-D space, moving regions are conceptually represented as smooth surface layers, and are extracted through a voting process that enforces the smoothness constraint while preserving motion discontinuities.
    The key features of this approach are: 1) inference of a dense representation in terms of accurate velocities, motion boundaries and regions, without any a priori knowledge of the motion model, based on the smoothness of motion only; 2) consistent handling of both smooth moving regions and motion discontinuities; 3) integration of motion and monocular (intensity) cues for accurate segmentation; 4) a 4-D layered representation that allows for spatial separation of the points according to both velocities and image coordinates, thus letting tokens from the same layer to strongly support each other, while inhibiting influence from other layers, or from isolated tokens; 5) a non-iterative voting scheme, which does not require initialization and does not suffer from local optima or poor convergence problems, and whose only free parameter is the scale of analysis, an inherent characteristic of human vision. [Details]

    Relevant Publications
    Support
    This research has been funded in part by the Integrated Media Systems Center (IMSC), a National Science Foundation Engineering Research Center, Cooperative Agreement No. EEC-9529152, and by National Science Foundation Grant 9811883.

    GlobeAll: Panoramic Video for an Intelligent RoomThis project is targeted at a real-time modular system for vision-based intelligent environments. We designed and developed GlobeAll, a modular prototype based on an electronic pan-tilt-zoom camera array. The visual input is acquired by a multiple-camera system, which generates a composite view of the scene with a wide field of view (as a planar mosaic) and a view of the desired region of interest (as an electronically-controlled virtual camera). By maintaining an adaptive background model in mosaic space, the system segments the foreground objects as planar layers. Among them, targets are selected and tracked by redirecting the virtual camera. An interpretation module analyzes the generated models (segmented objects, trajectories), allowing for the detection of simple events.
    Compared to other solutions, the key features of our system are: 1) acquisition of a large field of view, while also capturing enough resolution for focusing on a certain region of interest; 2) ability to perform pan-tilt-zoom operations electronically rather than mechanically; 3) better precision and response time in redirecting the region of interest; 4) low cost and high robustness, since it involves a digital solution, instead of using expensive and fragile mechanical or optical components. [Details]

    Relevant Publications
  • Mircea Nicolescu, Gerard Medioni, "GlobeAll: Panoramic Video for an Intelligent Room", Proceedings of the International Conference on Pattern Recognition, vol. I, pages 823-826, Barcelona, Spain, September 2000.
  • Mircea Nicolescu, Gerard Medioni, "Electronic Pan-Tilt-Zoom: A Solution for Intelligent Room Systems", Proceedings of the International Conference on Multimedia and Expo, pages 1581-1584, New York, NY, July 2000.
  • Mircea Nicolescu, Gerard Medioni, Mi-Suen Lee, "Segmentation, Tracking and Interpretation Using Panoramic Video", Proceedings of the IEEE Workshop on Omnidirectional Vision, (in conjunction with the IEEE Conference on Computer Vision and Pattern Recognition) pages 169-174, Hilton Head Island, SC, June 2000.
    • Support
    This research has been funded in part by the Integrated Media Systems Center (IMSC), a National Science Foundation Engineering Research Center, Cooperative Agreement No. EEC-9529152, with additional support from the Annenberg Center for Communication at the University of Southern California and the California Trade and Commerce Agency. The support of the Philips Multimedia Center is also gratefully acknowledged.

    Home | People | Faculty | Students | Postdocs | Visiting Scientists
    External Collaborators | Alumni | Research | Current Projects
    Past Projects | Software/Databases | Sponsors | Publications
    Courses | Announcements | Vision Resources | Contact Us