4005-754 Image Understanding

Professor R. Canosa, Ph.D.

General Course Policies

This course explores the theory and methodologies used to automate the interpretation of images in terms of semantic content. Techniques from image processing are extended for the purpose of scene understanding using both a bottom-up and a top-down approach. Topics include human visual perception, knowledge representation, object recognition, contextual classification, clustering techniques, scene labeling, semantic segmentation, 3D models and matching, active vision, and automated reasoning about images. A programming project is required.

Course Outcomes:

Students will demonstrate a thorough understanding of key concepts pertaining to classical image understanding algorithms, including the advantages and disadvantages of each approach. Assessed by exams.
Students will implement several of the algorithms in software, and evaluate the effectiveness of the approach. Assessed by programming projects.
Students will read, explain, and discuss the current literature on image understanding and approaches. Assessed by homework, term paper, and presentation.

Readings Assignments:

A major component of this course consists of reading and understanding the current literature in this field, as well as presenting the findings to an audience. Reading assignments are due on the date indicated in the schedule below, before class begins. Late readings will be accepted with a penalty of a 25% deduction off the maximum possible score for each day late. Once class begins, the assignment is considered to be one day late.

A reading assignment consists of reading a selected publication in the area of image understanding, writing a summary of the paper (typed, 250-300 words - do not merely paraphrase the abstract), and 3 questions that you have about the paper. In addition, a student from the class will present the paper to the rest of the class. The presentation will consist of 15-20 PowerPoint slides or transparencies that give a good, general overview of the paper, highlighting the most important or interesting findings, and a list of points for discussion. The student presenting is expected to provide:

A general overview of the paper
Background information that may be needed to understand the issues
An outline of the experimental steps or the procedure used
An explanation of the results
An explanation of the author's conclusions based on the results
Discussion points - anything that was unclear, of questionable scientific merit, or particularly interesting

Your grade will be based not only on how well you present the topic paper when it is your turn to present, but also on how well you pose questions and participate in the discussion when it is not. You may want to make a copy of your paper summary and questions so you can hand in the original at the beginning of class, and have a copy with you during the classroom discussion.

You must send me an email message on or before Friday December 2nd (by midnight) with the paper that you would like to present. The schedule will be filled on a first-come first-served basis.

Paper Presentation Schedule

Project:

Every student will implement an image understanding project of their own choosing, subject to approval by the instructor. The project will be implemented in either Matlab, IDL, or OpenCV, and should be of sufficient scope to require an extended effort over six weeks. Each student will decide on a project topic and will present a short (5 minute) overview of the project, as well as the proposed approach during week 4 of the quarter. A 1-page summary of the proposed project, the proposed approach, and a list of at least 3 references is due at the time of the presentation during week 3.

The last week of class will be devoted to project presentations and demonstrations. Each student will prepare a 20 minute presentation of their project which will include a demonstration of the implementation. A final project write-up (at least 10 pages) is due at the end of the last week of classes.

Course Information
Course Page	www.cs.rit.edu/~rlc/Courses/ImageUnderstanding/
Instructor	R. Canosa, Ph.D. rlc at cs dot rit dot edu rlcvcs at rit dot edu
Highly Recommended	Sonka, Hlavac, an Boyle, Image Processing, Analysis, and Machine Vision, 3rd edition, 2008 Matlab companion book to Image Processing, Analysis, and Machine Vision by Svoboda, Kybic, and Hlavac
Suggested Resources	Computer Vision and Image Understanding Stephen H. Palmer, Vision Science: Photons to Phenomenology, MIT Press, 1999 Larry S. Davis, Foundations of Image Understanding, Kluwer International Series in Engineering and Computer Science: vol. 628. August, 2001. ISBN: 0-7923-7457-6. David A. Forsyth, Jean Ponce, Computer Vision: A Modern Approach, Prentice-Hall, Inc. 2003. ISBN: 0-13-085198-1. Gonzales, Woods, and Eddins, Digital Image Processing Using MATLAB, Prentice-Hall, Inc. 2004. Laura G. Shapiro, George C. Stockman, Computer Vision, Prentice-Hall, Inc. 2001. ISBN: 0-13-030796-3. MATLAB Image Processing Toolbox User's Guide Installing and Using OpenCV Using OpenCV with a Kinect OpenCV Library The Computer Vision Homepage maintained at Carnegie Mellon University Online Demos Companies that hire computer vision specialists maintained by Dave Lowe at the University of British Columbia Computer Vision Bibliography maintained by Keith Price at the University of Southern California Imaging Science Seminar Series
Prerequisites	4003-457 or 4005-757 Introduction to Computer Vision
Grading	14% Reading Assignments 6% Paper Presentation 25% Project 25% Midterm Exam 25% Final Exam 5% Participation

Class Schedule and Assignments
Week	Dates	Special Events	Topics	Notes
1	Nov 28	Email me your paper choice by midnight on Friday December 2	Human Visual Perception	Human Vision
2	Dec 5	If I Were You: Perceptual Illusion of Body Swapping by V. I. Petkova and H. H. Ehrsson for discussion on December 7	Knowledge Representation Shape Representation	Shape
3	Dec 12	Scalable Multi-Class Object Detection by Razavi et al. for discussion on Dec 14 Project Topic Due December 14	Classification Parametric Decision-Making Non-parametric Decision-Making	Classification
4	Jan 9	Tour the World: Building a Web-Scale Landmark Recognition Engine by Zheng et al. for discussion on January 11	Clustering Hierarchical Techniques Partitioning Techniques	Clustering
5	Jan 16	Recognition Using Visual Phrases by Sadeghi and Farhadi for discussion on January 18	Control Strategies Semantic Segmentation	Semantic Models
6	Jan 23	Midterm Exam January 25	Semantic Networks Constraint Propagation
7	Jan 30	Shape Estimation in Natural Illumination by Johnson and Adelson for discussion on February 1	3D from 2D Active Vision	3D
8	Feb 6	Real-Time Human Pose Recognition in Parts From Single-Depth Images by Shotton et al. for discussion on February 8	3D Models and Matching Geometric Models Relational Models Functional Models	3D Models
9	Feb 13	What Makes a Chair a Chair? by Grabner et al. for discussion on February 15	Reasoning About Images	Reasoning
10	Feb 20	Project Presentations Project Write-up Due Feb 26