Text:

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd edition, Wiley-Interscience. ISBN 0-471-05669-3
Download Revised Section 2.11 and Errata

Optional Texts (not required):

• B. Ripley, Pattern Recognition and Neural Networks (Cambridge University Press)
  
• K. Fukunaga, Statistical Pattern Recognition, 2nd edition, Morgan Kaufmann, 1990
  
• T. Hastie et al., The Elements of Statistical Learning, Springer-Verlag, 2001.
  
• D. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge Univ Press, 2003.
  
• C. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995.
  
  

Useful Links

• Andrew Moore's Data Mining Tutorials
• Pattern Recognition Home page
• Bibliography Image Processing, Pattern Recognition & Neural Networks
• Review Papers on Statistical Pattern Recognition, Neural Networks and Learning
• Pattern Recognition Journals
• Computer Vision Home Page
• Face Detection Home Page
• Face Recognition Home Page
• Gesture Recognition Home page (more info can be found here )
• Biometrics Research
• CMU Artificial Intelligence Repository
• Links to Mathematics, Statistics, and Geometry resources
• Machine Learning Databases
• MNIST database of handwritten digits

Some useful software:

• Image Formats and Viewers
  
• C Routines to read/write .pgm images ReadImage.c WriteImage.c and an example Threshold.c
  
• C++ Routines to read/write .pgm images ReadImage.cpp WriteImage.cpp and an example Threshold.cpp
  
• Statistical Pattern Recognition Toolbox
  
• The R Project
  

Course Description

Course Outline (tentative)

Course Prerequisites

Exams and Assignments

Grading will be based on two exams, homework assignments, programming assignments, and a paper presentation. Specifically, there will be a midterm and a final exam. The material covered by the exams will be drawn from the material covered in class and the homework. Both exams will be closed books, closed notes. Homework problems will be assigned on a regular basis and will be collected at the beginning of the class on the due date. Also, there will be several programming assignments. Finally, each student will be also required to present a paper to the rest of the class. The topic of the paper should be related to the application of a pattern recognition technique (i.e., preferably among those discussed in class but not exclusive) to a computer vision problem. Each presentation should be professional as if it was presented in a formal conference (i.e., slides/projector).

Late homework and/or programming assignments will not be accepted. If you are unable to hand in an assignment by the deadline, you must discuss it with me before the deadline. If you are unable to attend an exam you must inform me in advance. A missed exam may be made up only if it was missed due to an extreme emergency. Please, note that good programming skills and a lot of motivation would be essential in order to complete the programming assignments.

Disability Statement

Syllabus

• Spring 2006 syllabus (pdf)

Exam Related Material

• Sample Midterm Exam
• Midterm Cheat Sheet

Useful Material

• Jain, A.K, Murty M.N; Flynn P.J., Data Clustering: A Review, ACM Computing Surveys, vol. 31, no. 3, pp. 264-323, 1999.
  

Lectures and Reading Assignments

• Introduction to Pattern Recognition
  • Jain, A.K.; Duin, P.W.; Jianchang Mao, Statistical pattern recognition: a review, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 22, no 1, pp. 4-37, Jan. 2000.
    
  • T. Dietterich Machine Learning Research: Four Current Directions, AI Magazine, vol 18, no. 4, 1997.
• Review of Probability
  • Everything I need to know about Probability
    
  • Generating Gaussian Random Numbers
• Bayesian Decision Theory
  • Bayesian Decision Theory with Gaussian Distributions
• Bayesian Decision Theory - Case Studies
  • A. Madabhushi and J. Aggarwal, A bayesian approach to human activity recognition, 2nd International Workshop on Visual Surveillance, pp. 25-30, June 1999.
  • J. Yang and A. Waibel, A Real-time Face Tracker, Proceedings of WACV'96, 1996.
• Bayesian Belief Networks
  • E. Charniak, Bayesian Networks without Tears, AI Magazine, 1991.
    
  • A Brief Introduction to Graphical Models and Bayesian Networks
    
  • D. Heckerman, A Tutorial on Learning with Bayesian Networks , Technical Report, Microsoft, 1995.
    
  • W. Buntine, A guide to the literature on learning probabilistic networks from data , IEEE Transactions of Knowledge and Data Engineering, vol. 8, no. 2, pp. 195-210, 1996..
• Bayesian Estimation - Part 1
  • In Jae Myung, Tutorial on Maximul Likelihood Estimation, Journal of Mathematical Psychology, vol. 47, pp. 90-100, 2003.
• Bayesian Estimation - Part 2
• Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) (with case studies)
  • M. Turk and A. Pentland, Eigenfaces for Recognition, Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
    
  • Daniel L. Swets, John (Juyang) Weng Using Discriminant Eigenfeatures for Image Retrieval , IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 18, No 8 , pp. 831-836, 1996.
  • A. Martinez and A. Kak PCA versus LDA, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 23, No 2, pp. 228 -233, 2001.
• EM Algorithm
  • Todd Moon, The Expectation-Maximization Algorithm, IEEE Signal Processing Magazine, November 1996 (Warning: it contains many typos!).
    
  • Michael Collins, The EM Algorithm
    
  • Jeff Bilmes, A Gentle Tutorial of the Expectation-Maximization Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models, International Computer ScienceInstitute.
• EM Case Studies
  • S. McKenna, Y. Raja, and S. Gong, Tracking color objects using adaptive mixtur e models, Image and Vision Computing, vol. 17, pp. 225-231, 1999.
    
  • C. Stauffer and E. Grimson, Adaptive background mixture models for real-time tracking, IEEE Computer Vision and Pattern Recognition Conference, Vol. 2, pp. 246-252,
• Hidden Markov Models
  • L. Rabiner, An Introduction to Hidden , IEEE ASSP Magazine, January 1986.
    
  • R. Dugad and U. B. Desai, An Introduction on Hidden Markov Models.
• Non-parametric Estimation
  • M. Jones and J. Rehg, Statistical color models with application to skin detection, International Journal of Computer Vision Vol. 46, No. 1, 2002. (powerpoint)
    
  • A. Elgammal et al, Background and Foreground Modeling Using Nonparametric Kernel Density Estimation for Visual Surveillance, Proceedings of the IEEE, vol. 90, no. 7, pp. 1151-1163, 2002. (powerpoint)
    
  • A. Moore, An introductory tutorial on kd-tress, Carnegie Mellon University.
  
  
• Linear Discriminant Functions
  
  
• Dr. Salil Prabhakar, Fingerprint Recognition: Future Directions, DigitalPersona Inc., CA
  
  
• Support Vector Machines
  • C. Burges, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, Kluwer Academic Publishers, 1998.
    
  • M. Pontil and A. Verri, Support vector machines for 3D object recognition, IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 20, no. 6, pp. 637-646, 1998.
    
  • A. Mojan, C. Papageorgiou and T. Poggio, Example-based object detection in images by components, IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 23, no. 4, pp. 349-361, 2001.
    
  • B. Moghaddam and M. Yang, Gender Classification with SVM, IEEE Conference on Face and Gesture Recognition, pp. 306-311, 2000.

Homework Assignments

Homework Assignment #1 (Due on 2/14/2006)
Homework Assignment #2 (Due on 2/21/2006)
Homework Assignment #3 (Due on 3/7/2006)
Homework Assignment #4 (Due on 4/25/2006)

Programming Assignments

Programming Assignment #1 (Due on 2/28/2006)
Programming Assignment #2 (Due on 3/28/2006)
Programming Assignment #3 (Due on 5/2/2006)

Papers for Student Presentations

• N. Zeidat et al., Dataset editing techniques: a comparative study, Department of Computer Science, University of Houston.
  
• P. Domingos and M. Pazzani, On the Optimality of the Simple Bayesian Classifier Under Zero-One Loss Machine Learning, vol. 29, pp. 103-130, 1997.
  
• R. Bolle et al, VeggieVision: A Produce Recognition System IEEE Workshop on Applications of Computer Vision, 1996. ( powerpoint)
  
• F. Samaria, Face Segmentation for Identification Using Hidden Markov Models, British Machine Vision Conference, 1993.
  
• Y. Wu, Q. Tian, and T. Huang, Discriminant-EM algorithm with application to image retrieval, CVPR 2000.
  
• H. Schneiderman and T. Kanade, A statistical method for 3D object detection applied to faces and cars, CVPR 2000 ( powerpoint).
  
• S. Mahamud, M. Hebert, and J. Shi, Object Recognition Using Boosted Discriminants, CVPR 2001 ( powerpoint).
  
• M. Brand and V. Kettnaker, Discovery and Segmentation of Activities in Video, IEEE PAMI, vol. 22, no. 8, pp. 844-851, 2000.
  
• I. Gath and B. Geva, Usupervised Optimal Fuzzy Clustering, IEEE PAMI, vol. 11, no. 7, pp. 773-781, 1989. ( powerpoint)
  
• Anil K. Jain, Arun Ross and Sharath Pankanti, Prototype Hand Geometry-based Verification System,2nd Int'l Conference on Audio and Video-based Biometric Person Authentication, Washington D.C., pp.166-171, 1999. ( powerpoint)