CS493/790 (X) Advanced Topics in Robotics

Fall 2004


General Information Course Description Project Syllabus Grading Announcements

Instructor: Monica Nicolescu

E-mail: monica@cs.unr.edu
Office: SEM 239
Phone: (775) 784-1687
Office hours: Tuesday, Thursday 2:30pm-3:30pm

Teaching/laboratory assistant: TBA

Phone: TBA
Office hours: TBA

Time and Place

Tuesday & Thursday: 9:30-10:45am; PE 205

Recommended Textbooks

There is no required textbook for this course. The books below constitute a good source of additional background and in-depth readings on topics relevant to the course.


The Robotics Primer, 2001.
Author: Maja Mataric' (available in draft form at the beginning of the semester)
Robotic Explorations: An Introduction to Engineering Through Design, 2001.
Author: Fred G. Martin
Behavior-Based Robotics, 2001.
Author: Ron Arkin

Course description

This is an advanced level, seminar-style course, which will examine representative approaches to robot control, learning, coordination and cooperation between multiple robots and human-robot interaction. Students will learn about the development of the robotics field and the main directions of research in this area. Each week all the students will read all of the assigned readings. Each of the assigned readings will be presented by a student, and discussed and critiqued by all others.



CS 491/691 (X) - Robotics or CS 476/676 (Artificial Intelligence). Good programming skills are essential. If you have not taken any of these classes you should purchase "The Robotics Primer" book for background reading.


Each student will complete an individual research project, on a topic covered in class. Teams up to two students are possible, however the complexity of the selected topic must justify it.

Project topics: The projects should be an implementation of either: a single robot system (involving complex behavior and demonstrated on a physical robot) or a multi-robot system (involving cooperation/communication/coordination between robots and demonstrated in simulation).

Test-beds: The following simulation environments and physical robots will be available for the project:

  • The Player-Stage-Gazebo simulator (playerstage.sourceforge.net). Player is a general purpose language-indepedent network server for robot control. Stage is a Player-compatible high-fidelity indoor multi-robot simulation testbed. Gazebo is a Player-compatible high-fidelity 3D outdoor simulation testbed with dynamics. Using Player/Stage/Gazebo allows for direct porting to Player-compatible physical robots.
  • One Player-compatible ActivMedia Pioneer 1 AT (all terrain) robot. The robot is equipped with 7 sonar sensors and requires the use of a laptop (not provided).
  • One Player-compatible ActivMedia Pioneer 1 indoor robot. The robot is equipped with 7 sonar sensors and requires the use of a laptop (not provided).
  • 16 LEGO robot kits. The robots are equipped with a Handy Board microcontroller that can be programmed using Interactive C. A reference on the Handy Board and Interactive C can be found here.

Project report: For each project students should prepare a final project report, which should include the following:

  • Title, author
  • Abstract
  • Introduction and motivation
  • Problem definition, including project goals, assumptions, constraints, and evaluation criteria
  • Details of proposed approach
  • Results and objective experimental evaluation and/or sound theoretical proof of the proposed approach
  • Strengths and weaknesses
  • Review of relevant literature and previous research and how it relates to the project
  • Discussion and conclusion
  • References
  • Appendix (relevant code or algorithms)

Timeline: The project development should adhere to the following timeline:

  • Project proposal - outlines the specific goals, implementation platform and the proposed approach - due about four weeks from the beginning of the semester (see the syllabus).
  • Project status report - describes the current status of the project, what has been done, what is still to be done - due around mid-semester (see the syllabus).
  • Project presentation (live demos are highly encouraged) - during one of the last two lectures
  • Project report - due on Dec 15.



Following are the topics that will be discussed, listed in the approximate order in which they will be covered.
  • Introduction, development of the robotics field
  • Reactive systems
  • Hybrid systems
  • Behavior-based systems
  • Navigation and mapping
  • Learning: supervised learning, reinforcement learning, genetic/evolutionary approaches, imitation and learning from demonstration
  • Multi-agent control
  • Multi-robot control
  • Multi-robot learning
  • Emergent behavior
  • Biologically inspired robotics
  • Human-robot interaction

Class schedule

Paper reports: During each lecture we will discuss a numbers of papers on a specific research topic. Students must read the papers before class and submit, for each of the papers presented that day, a brief report for each paper. The reports should be submitted at the beginning of the class when they are discussed and must contain:

  • Student's name
  • Title and authors of the paper
  • A short paragraph summarizing the contributions of the paper
  • A critique of the paper that addresses the strengths and weaknesses of the paper

All reports should be typed - no hand-written reports will be accepted.

Paper presentations: During the semester, each student is required to present several papers to the rest of the class. Each presentation should take about 25-30 minutes and must be prepared as if presented in a formal conference (i.e., slides, projector). This presentation should assume that the audience has read the paper, and not spend more than about 15 minutes summarizing it. The rest of the presentation should be spent on discussing the paper, its strengths, weaknesses, any points needing clarification. The presentation will be followed by class discussions in which all students will express their point of view and general comments on the paper.

The table below presents the tentative schedule of assigned readings.

Date Topic Papers Presentations

Aug 24

Introduction Monica Nicolescu

Aug 26

Robot control architectures, history of robotics Monica Nicolescu

Aug 31

Reactive architectures "Intelligence Without Reason", Rodney A. Brooks, Proceedings of 12th International Joint Conference on Artificial Intelligence (IJCAI-91), Sydney, Australia, pages 569-595, August 1991. Anil [ppt]
"Behavior-Based Artificial Intelligence", Pattie Maes. From Animals to Animats 2, Simulation of Adaptive Behavior, pages 2-10, 1993. Derak [ppt]

Sep 2

Reactive architectures "A Robust Layered Control System for a Mobile Robot", Rodney Brooks, MIT AI Lab Technical Report, Sept. 1986.
Michael [ppt]
"What are Plans For", Philip Agre and David Chapman, Robotics and Autonomous Sysmtems, vol. 6, Elsevier Science, pages 17-34, 1990 Tamer [ppt]

Sep 7

Reactive architectures "Elephants Don't Play Chess", Rodney, A. Brooks, Robotics and Autonomous Systems, Vol 6, pp. 3-15, 1990. Yan [ppt]
"Universal Plans for Reactive Robots in Unpredictable Environments", Marcel Schoppers, Proceedings of the 10th International Joint Conference on Artificial Intelligence (IJCAI-87), Milan, Italy, pages 1039-1046, August 1987. Javier [ppt]

Sep 9

Hybrid architectures "On Three-Layer Architectures", Eran Gat, Artificial Intelligence and Mobile Robotics, in D. Kortenkamp, R. P. Bonnasso and R. Murphy (eds.), AAAI Press, pages 195-210, 1998. Ekkasit [ppt]
"AuRA: Principles and Practice in Review", Ron Arkin and Tucker Balch, Journal of Experimental and Theoretical Artificial Intelligence, 9(2-3), pages 175-189, April 1997. Jirakhom [ppt]

Sep 14

Hybrid architectures "SSS: A Hybrid Architecture Applied to Robot Navigation", Jon Connell, Proceedings, IEEE International Conference on Robotics and Automation (ICRA-92), Nice, France, pages 2719-2724, May 12-14, 1992. Kai [ppt]
"Structured Control for Autonomous Robots", Reid G. Simmons, IEEE Transactions on Robotics and Automation, Vol 10, No. 1, pages 34-43, February 1994. Uday [ppt]

Sep 16

Hybrid architectures "Experiences with an Architecture for Intelligent Reactive Agents", Peter Bonasso, James Firby, Erran Gat, David Kortenkamp, David Miller, Marc Slack, Journal of Experimental and Theoretical Artificial Intelligence , Vol. 9, No. 1, 1997. Tony [ppt]
Behavior based systems "Behavior-Based Control: Examples from Navigation, Learning, and Group Behavior", Maja J Mataric´, Journal of Experimental and Theoretical Artificial Intelligence, special issue on Software Architectures for Physical Agents, 9(2-3), pages 323-336, 1997. Jeff [ppt]

Sep 21

Behavior based systems "Integration of Representation Into Goal-Driven Behavior-Based Robots", Maja J Mataric´, IEEE Transactions on Robotics and Automation, 8(3), pages 304-312, June 1992. Andy [ppt]
"Behavior Coordination Mechanisms: State-of-the-Art", Paolo Pirjanian, Technical Report IRIS-99-375, Institute for Robotics and Intelligent Systems, School of Engineering, University of Southern California, October 1999. Chris Martin [ppt]

Sep 23

Project topic presentations.    

Sep 28

Behavior based systems "A Formal Model of Computation for Sensory-Based Robotics", Damian Lyons, Michael Arbib, IEEE Transactions on Robotics and Automation, Vol. 5, No. 3, pages 280-293, June 1989. Steve [ppt]
"Internalized Plans: A Representation for Action Resources", David Payton, Robotics and Autonomous Systems, Vol. 6, pages 89-103, 1990. Ryan

Sep 30

Navigation & mapping "Motor Schema Based Navigation for a Mobile Robot: An Approach to Programming by Behavior", Ronald Arkin, Proceedings of the IEEE Conference on Robotics and Automation (ICRA-87), Raleigh, NC, pages 264-271, March 31 - April 3, 1987. Chris Miles [ppt]
"Behavior Control for Robotic Exploration of Planetary Surfaces", Eran Gatt, Rajiv Desai, Robert Ivlev, John Loch, David Miller, IEEE Transactions on Robotics and Automation, Vol. 10, No 4, Aug 1994. Tony [ppt]

Oct 5

Navigation & mapping "DAMN: A Distributed Architecture for Mobile Navigation", Julio K. Rosenblatt, Journal of Experimental and Theoretical Artificial Intelligence, Vol. 9, No. 2 / 3, 1997, pp. 339 – 360. Chris Miles [ppt]
"Sonar-Based Real-World Mapping and Navigation", Alberto Elfes, IEEE Journal of Robotics and Automation, Vol. 3, pages 249-265, 1987. Uday [ppt]

Oct 7

Learning: supervised learning and reinforcement learning "ALVINN: An Autonomous Land Vehicle In a Neural Network", Dean Pomerleau, Advances in Neural Information Processing Systems 1, Morgan Kaufmann, 1989. Ryan [pdf]
"Learning to Coordinate Behaviors", Pattie Maes and Rodney A. Brooks, Proceedings, 8th National Conference on Artificial Intelligence (AAAI-90), AAAI Press/MIT Press, pages 796-802, 1990. Javier [ppt]
Oct 12 Reinforcement learning "Automatic Programming of Behavior-Based Robots Using Reinforcement Learning", Sridhar Mahadevan and Jon Connell, Artificial Intelligence , 55(2-3), 311-365, 1992. Anil
"Learning to use selective attention and short-term memory in sequential tasks", A. McCallum, From Animals to Animats, Fourth International Conference on Simulation of Adaptive Behavior, September, 1996. Kai
Oct 14 Learning by imitation "A Robot Controller Using Learning by Imitation", Gillian Hayes and John Demiris, Proceedings of the 2nd International Symposium on Intelligent Robotic Systems, pp. 198-204, Grenoble, France, July 1994. Chris Martin
"Seeing, understanding and doing human task",  Kuniyoshi, Y.; Inaba, M.; Inoue, H., Robotics and Automation, 1992. Proceedings., 1992 IEEE International Conference on , 12-14, pages:2 - 9 vol.1, May 1992. Steve
Oct 19 Genetic/evolutionary approaches to robot learning "Evolving 3D Morphology and Behavior by Competition", Karl Sims, Proceedings, Artificial Life IV, R. Brooks and P. Maes (eds.), MIT Press/Bradford Books, pages 28-39, 1994. Ekkasit
"RoboShepherd: Learning a Complex Behavior" by A. Schultz J.J. Grefenstette and W. Adams, RoboLearn '96, The Robots and Learning Workshop at FLAIRS, May 20, 1996. Yan
Oct 21 Genetic/evolutionary approaches to robot learning "Automatic Programming of Robots Using Genetic Programming", John Koza and James Rice, AAAI , pages 194- 201, 1992. Jirakhom
"Evolving Electronic Robot Controllers that Exploit Hardware Resources", Arian Thompson, F. Moran, A. Moreno, J. J. Merelo, and P. Chacon (eds.), Advances in Artificial Life: Proceedings of the Third European Conference on Artificial Life, 929, Springer-Verlag, pages 640-656, 1995. Chris Miles
Oct 26 Multi-agent control "Agent Architectures for Flexible, Practical Teamwork",  Milind Tambe, AAAI 1997. Ryan
"Flocks, Herds, and Schools: A Distributed Behavioral Model, in Computer Graphics", Reynolds, C. W, 21(4), SIGGRAPH '87 Conference Proceedings, pages 25-34, 1987, Jeff
Oct 28 Invited presentation "The Player/Stage/Gazebo Project: Open Source Tools for Robotics Research" Dr. Brian Gerkey
Nov 2 Multi-robot control "Stigmergy, self-organization, and sorting in collective robotics", Owen Holland and Chris Melhuish, Artificial Life, 5(2), pages 173-202, 1999. Anil
"Swarm-Made Architectures", J. L. Deneubourg, First European Conference on Artificial Life, 1991. Uday
Nov 4 Project status presentations.    
Nov 9 Multi robot control "ALLIANCE: An Architecture for Fault Tolerant, Cooperative Control of Heterogeneous Mobile Robots", Lynne E. Parker, Proceedings of the 1994 IEEE/RSJ/GI International Conference on Intelligent Robots and Systems (IROS '94), September 1994: 776-783. Javier
"Cooperation without deliberation: A minimal behavior-based approach to multi-robot teams", Barry Werger, Artificial Intelligence, 110, pages 293-320, 1999. Andy
Nov 11      
Nov 16 Multi robot control "Cooperative transport by ants and robots" by C. Ronald Kube and Eric Bonabeau, Robotics and Autonomous Systems 30(1-2), pages 85-101, 2000. Jeff
"Emotion-Based Control of Cooperating Heterogeneous Mobile Robots", Robin Roberson Murphy, Christine Laetitia Lisetti, Russell Tardif, Liam Irish, and Aaron Gage, IEEE Transactions on Robotics and Automation, VOL. 18, NO. 5, OCTOBER 2002. Tony
Nov 18 Multi robot learning "Agents that Learn from Other Competitive Agents", Minoru Asada, Eiji Uchibe, Koh Hosoda, Proc. of the Machine Learning Conference Workshop on Agents that Learn from Other Agents, pages 1-7, 1995. Kai
"Using Communication to Reduce Locality in Multi-Robot Learning", Maja Mataric', AAAI, 1997. Derak
Nov 23 Multi robot learning "Learning to Behave Socially", Maja Mataric', From Animals to Animats 3, Third International Conference on Simulation of Adaptive Behavior Yan
"Reinforcement Learning in the Multi-Robot Domain", Maja Mataric', Autonomous Robots, Vol 4, pp 73-83, 1997. Andy
Nov 25      
Nov 30 Biologically inspired robotics "Towards a Theory of Emergent Functionality", Luc Steels, From Animals to Animats 1 Michael
"A Biological Perspective on Autonomous Agent Design", Randall Beer, Hillel Chiel, Leon Sterling, Robotics and Autonomous Systems, Vol. 6, pages 169-186, 1990. Chris Martin
Dec 2 Human robot interaction "Roball, the rolling robot", Francois Michaud, Serge Caron, Autonomous Robots, 12(2): 211-222, 2002. Jirakhom
"MINERVA: A second generation mobile tour-guide robot", Sebatian Thrun, Maren Bennewitz, Wolfram Burgard, Armin B. Cremers, Frank Dellaert, Dieter Fox, Dirk Haehnel, Charles Rosenberg, Nicholas Roy, Jamieson Schulte, and Dirk Schulz, Proceedings, IEEE International Conference on Robotics and Automation (ICRA), 10-15 May 1999. Steve
Dec 7 Final project preparation.    
Dec 13 2:30pm: Project final presentations and demonstrations. Location: SEM 246  

Assignments and grading

Evaluation for this class will be based on your paper presentations, participation in class discussions, summary reports for each of the papers and a final project. Regular class participation is required.

Grading policy (tentative, subject to change):

Paper reports: 20%
Participation in class discussions: 20%
Paper presentations: 20%
Final project: 40%

Late policy: No late submissions will be accepted.

Academic integrity: Students are encouraged to study together, however each student must individually prepare his/her solutions. Cheating or plagiarism are not permitted and will be sanctioned according with the UNR policy on Academic Standards. You should carefully read the section on Academic Dishonesty found in the UNR Student Handbook (copies of this section are on-line). Your continued enrollment in this course implies that you have read it, and that you subscribe to the principles stated therein.


Announcements regarding the assignments or other updates will be posted on the class web page and also sent by e-mail.

Created by: Monica NICOLESCU (e-mail:monica@cs.unr.edu)
Last update: 08/03/2004