This course extends image processing and computer vision concepts from the study of single static images which have been passively acquired to actively acquired image sequences involving motion and three-dimensionality. A first course on image processing and computer vision of single static images, such as CSE473 or CSE573, is a prerequisite to this course. The strengths and limitations of the dominant Marr paradigm for 3D and motion will be discussed in the context of a set of specific algorithms for low and higher-level processing. Newer approaches designed to overcome some of these limitations based on purposive, active and animate vision principles inspired by natural vision will be developed. We will work from research
papers and reports rather than a required textbook.
Course schedule
Office Bell 136 phone 645-3180x137, email: peter@cse.buffalo.edu.
Office hours 2:00PM-3:00PM MTW.
| Monday Lecture | Wednesday Lecture | Friday Lecture |
| 01/12
Introduction and overview of semester work |
01/14
Projective geom I |
01/16 Projective geom II |
| 01/19 MLK Day
No Class |
01/21 Steropsis I |
01/23
Steropsis II |
| 01/26 Shape from shading | 01/28 Other shape-from | 01/30 Correspondence |
| 02/02
Natural vision |
02/04 Natural Perception | 02/06 Purposive vision I |
| 02/09 Purposive Vision II | 02/11 Active egomotion | 02/13 Obstacle aviod I |
| 02/16
Visual servoing I |
02/18
Visual servoing II |
02/20
Homing |
| 02/23
Homing II |
02/25 Natural OR | 02/27 3DModel-based OR |
| 03/02 Midterm Exam | 03/04 2D View-based OR | 03/06
Indexing, occlusion |
| 03/09 Spring Break |
03/11 Spring Break |
03/13 Spring Break |
| 03/16 Optical Flow I | 03/18 Optical Flow II | 03/20 Optical Flow III |
| 03/23 Struc from motion | 03/25 Active OR I | 03/27 PCA |
| 03/30 Active
OR II |
04/01 Active
OR III |
04/03
Attentional segmentation |
| 04/06 Attentional gaze ctrl I | 04/08 Atten'l gaze cntrl II |
04/10 Systems overview |
| 04/13 Catchup |
04/15
Presentations Qiang Gao Jai Sharma Ning Zhang |
04/17 Presentations Kevin Keane Ashish Kulkarni Stephen Pfetsch |
| 04/20
Presentations Jeffrey Delmerico Niranjanganesh Kamat Yu Liu |
04/22
Presentations Dipankar Das Jaehan Koh Yongding Zhu |
04/24 Project report discussion, review |
| 04/27 Last class - Final exam prep |
04/29 Reading Day No Class |
Final Exam Tues May 5 11:45 OBrian 213 Project reports due |
Registration and meeting times: Registration number 267114. Course meets MWF 12:00-12:50pm Capen 260. First class meeting is Monday January 12."The following statement further describes the specific application of these general principles to a common context in the CSE Department environment, the production of source code for project and homework assignments. It should be thoroughly understood before undertaking any cooperative activities or using any other sources in such contexts.Prerequisites: CSE473, CSE573 or PI.
Course director: Peter Scott, Associate Professor Dept. Computer Science and Engineering, Rm. 136 Bell Hall, 645-3180 x 137, mailto:peter@buffalo.edu. Office hours 2:00PM-3:00PM MTW.
TA: none.
Required textbook: none. We will use published journal articles and conference papers.See Required and supplementary readings list below.
Required work: Midterm and final exams, project presentation, project report. The project may be used to satisfy C.S.E . Department M.S. capstone requirement. Exams 75 minute closed book, notes. Midterm covers Part I: Passive and active approaches to early vision for 3D and motion recovery; final exam covers Part II: Passive and active approaches to late vision for 3D and motion recovery. Projects will be done individually, will be research projects (as opposed to design projects as in CSE573) with project definition up to the student, but subject to agreement by the instructor. Projects can be executed in any computing environment the student selects (eg. Matlab on UNIX, Prolog on XP, etc.). Each student will do a 15min PowerPoint presentation of his project during the last two weeks of class.The PowerPoint presentations will be put on the course web page the evening before each student's scheduled presentation. Project reports will be submitted electronically, as pdf, doc, docx or txt files, and will be due at the Final Exam. The project reports will be checked using Turnin.com's web crawler which identifies suspicious similarities between the submitted text and sections of files available on the web or in its database.
Grading: 25% each: midterm exam, final exam, project presentation, project report.
Listserv newsgroup: sunyab.cse.668.
Academic integrity: The value of our courses, grades, degrees and research findings are dependent upon adherence to standardsof ethical conduct. Projects will be checked for originality by submitting to Turnitin.com, a web service which searches for duplication of report sections with material available on the web, or previously submitted to Turnitin.com for originality analysis.
Plagiarism and inappropriate collaboration willnot be tolerated. In this course we will adhere to the CSE departmental standard for academic integrity. We quote here this standard as it applies to coding assignments and projects:
All academic work must be your own. Plagiarism, defined as copying or receiving materials from a source or sources and submitting this material as one's own without acknowledging the particular debts tothesource (quotations, paraphrases, basic ideas), or otherwise representing the work of another as one's own, is never allowed. Collaboration, usually evidenced by unjustifiable similarity, is never permitted in individual assignments. Any submitted academic work may be subject to screening by software programs designed to detect evidence of plagiarism or collaboration.
It is your responsibility to maintain the security of your computer accounts and your written work. Do not share passwords with anyone, nor write your password down where it may be seen by others. Do not change permissions to allow others to read your course directories and files. Do not walk away from a workstation without logging out. These are your responsibilities. In groups that collaborate inappropriately, it may be impossible to determine who has offered work to othersin the group, who has received work, and who may have inadvertently madetheir work available to the others by failure to maintain adequatepersonal security In such cases, all will be held equally liable. "
Additional information on University-wide policies and procedures is contained in the UB Academic Integrity webpage and the Graduate School Grievance Policies and Procedures.
The current list of required and supplementary (optional) readings, which include chapters from books, journal and conference articles, and technical reports, is shown below. This list will likely be changed as the semester proceeds. I will try to make as many of the readings available electronically as possible. Most items on the list are linked to full-text electronic copies which may be printed out. Required readings for each topic will be announced in lecture as that topic is being discussed, and will be included in the lecture slides for that topic.
[ 1] Aloimonos, Introduction: active vision revisited, in Aloimonos (ed), Active Perception, Adv in CV V1 (1993) Erlbaum Associates 1-18.[ 2] Aloimonos, Weiss and Bandyophadyay, Active vision, Int J Comp Vision v1(1987) 333-356.
[ 3] Bakhtari, Eskandari, Naish and Benhabib, A multi-sensor surveillance system for active-vision based object localization IEEE Int Conf on Systems, Man and Cybernetics (2003), V1 , 1013 - 1018.
[ 4] Ballard, Animate vision, Artificial Intelligence v 48 (1991), 57-86.
[ 5] Barth and Tsuji, Egomotion determination through an intelligent gaze control strategy, IEEE Trans Systems, Man and Cybernetics, v 23 (1993), 1424-1432.
[ 6] Batlle et al, A review on strategies for recognizing natural objects in color images of outdoor scenes, Image and Vision Computing v18 (2000) 515-530.
[ 7] Beis and Lowe, Indexing without invariants in 3-D OR, IEEE Trans PAMI v21 (1999) 1000-1015.
[ 8] Borotschnig, Paletta, Prantl and Pinz, Appearance-based object recognition, Image and Vision Computing v18 (2000) 715-727.
[ 9] Caelli et al, 3-D object recognition: inspirations and lessons from biological vision, in Jain and Flynn (eds), Three-dimensional object recognition systems, Elsevier 1993.
[10] Camps et al, Operator theoretic methods for robust active vision problems, Conf on Decision and Control (2003) , V5, 4889 - 4895.
[11] Chakravorty and Saha, A hybrid approach to the simultaneous localization and mapping (SLAM) problem, 2009, unpublished manuscript.
[12] Chen and Li, 3D object reconstruction from multiple controlled viewpoints, Congress on Intelligent Control and Automation (2004). V5, 4645 - 4649.
[13] Davison, Mobile robot navigation using active vision, Ph. D. dissertation, University of Oxford (1998).
[14] Davison, Real-time simultaneous localisation and mapping with a single camera, Conf. on Computer Vision (2003), V2, 1403-1410.
[15] Davison, Modelling the world in real time: how robots engineer information, Phil. Trans. Soc. B, London A (2003), V361, 1-16.
[16] Dunn et al, Pareto optimal strategies for an active vision system, Congress on Evolutionary Computation (2004), 457-463.
[17] Feng et al, Moving object tracking research based on active vision, Congress on Intelligent Control and Automation (2004) V5, 3846-3849.
[18] Fermuller and Aloimonos, Vision and action, Image and Vision Computing v 13 (1995) 725-744.
[19] Fusiello, Uncalibrated Euclidean reconstruction, a review, Image and Vision Computing v 18 (2000) 555-563.
[20] Jaklic, Leonardis and Solina, Segmentation and recovery of superquadrics, Computational imaging and vision,v 20, Kluwer, 2000. Ch 2.
[21] Jiang, Sun and Jiang, Developmentally cognitive robot vision, Congress on Intelligent Control and Automation (2004) V5, 4691-4695.
[22] Kundur and Raviv, Active vision-based control schemes for autonomous navigation tasks, Pattern Recognition v 33 (2000), 295-308.
[23] Levine, Vision in man and machine, McGraw Hill 1985. Ch 3.
[24] Maki, Nordlund and Eklundh, Attentional scene segmentation: Integrating depth and motion from phase, Computer Vision and Image Understanding v 78 (2000), 351-373.
[25] Maki, Uhlin and Eklundh, Phase-based disparity estimation in binocular tracking, Proceedings of the Eight SCIA, (1993) 1145-1152.
[26] Marr, David, Vision: A Computational Investigation into the Human Representation and Processing of Visual Information, W. Freeman and Co. 1982.
[27] Matsuyama and Ukita, Real-time multi-target tracking by a cooperative distributed vision system, Proc. IEEE V90 (2002), 1136-1150.
[28] Murray, Reid and Davison, Steering without representation using active fixation, Perception, V26 (1997), 1519-1528.
[29] Nelson, From visual homing to object recognition, in Aloimonos (ed.) Visual Navigation, Lawrence Erlbaum Associates, 1997.
[30] Rakashekar, Cormack and Bovik, Image features that draw fixations, Conference on Image Processing (2003), V3 313-316.
[31] Scott, Roth and Rivest, View planning for automated 3-D OR and inspections, ACM Computing Surveys V35 (2003), 64-96.
[32] Shibata and Kawasumi, Solution for stereo correspondences on active stereo vision robot, IEEE Workshop on Advanced Motion Control (2004), 665 - 670.
[33] Sipe, Feature space trajectory methods for active computer vision, IEEE Trans PAMI v24 (2002), 1634-1643.
[34] Soyer et al, Attentional sequence-based recognition: Markovian and evidential reasoning, IEEE Trans SMC B V33 (2003), 937-952.
[35] Sonka et al, Image Processing, Analysis and Machine Vision, IPT Publishing 1999.
[36] Trucco and Verri, Introductory techniques for 3-D computer vision, Prentice Hall 1998. Chapter 8, Chapter 9.
[37] Trujillo-Romero et al, Modality control of an active camera for an object recognition task, Conference on Electronics, Communications and Computers (2004) 14-17.
[38] Vassallo et al,Visual servoing and appearance for navigation, Robotics and AutonomousSystems, v 31 (2000), 87-97.
[39] Zhou et al, Conditional feature sensitivity: a unifying view on active recognition and feature selection, Conference on Computer Vision V2 (2003), 1502-1509.
Most of the overheads used this semester in lecture will be available here. I will try to have them a few days in advance for you to look at and possibly photocopy, but there will be times they will not be available until after a given lecture.
Each registered student is expected to choose and complete a research project related to the content of this course, ie. a topic involving 3D or vision computing in either the passive or active paradigms. Some ideas for projects are available through the bulleted link near the top of this page, but these ideas are intended mostly to stimulate thought, your own ideas modified from or completely different from any of these suggestions are encouraged. Project reports are due at the time of the final exam. They should be submitted in electronic form, as pdf or doc files attached to an email addressed to peter@buffalo.edu. Reports will be graded on the basis of four elements: scholarship, inovation, completeness and quality of written presentation. There is no required format or length for the project report, but it should be comparable to a submission you might make for consideration for acceptance to a good conference. Projects graded B+ or better are qualified for use as the capstone project in the CSE MS degree program. Additional information on the project, and project ideas, will be posted from time to time during semester both here and in our newsgroup, sunyab.cse.668.
Projects will be done individually and all references to others' work properly cited. Any section of a report longer than a few words which is substantially identical to anothers' work must be cited as such. Failure to do so will be considered a violation of academic integrity and be subject to severe penalties. This means, for instance, that sections of a report which review some known method in the literature cannot be cut/pasted from other sources without those sources being cited immediately after the place of insertion and the cut/paste portion being set off in quotation marks or italics. As mentioned above, a software service Turnin.com will be used to monitor compliance. Your reports will be uploaded to Turnitin.com and a list of all matches reported back. For instance, if a paragraph is found to be substantially identical to a paragraph in Turnitin.com's database, that paragraph and its matching source will be reported back side-by-side. If there are more than one such partial matches, all will be reported back. Turnitin.com's database is very extensive, including most of the world wide web, plus many additional reports that have been uploaded to this tool in the past but never appeared on the web. Please do not jeapordize your career and reputation by taking any shortcuts. All writing in your report must be your own, excepting material set off in quotes or italics and properly cited.
Each student will present his project results to date in one of the four scheduled Project Presentation lecture meetings April 15-24. 15 min will be assigned to each presentation: 12 min for the prepared talk and 3 min for questions. I would encourage presenters to prepare their talk as a Powerpoint (.ppt) file with about 15-20 slides. An additional 1-5 backup slides containing details you don't expect to be able to get to might be prepared in case time happens to permit, or to help answer questions.
The presentations are understood to be reports of work in progress, not a presentation of a completed project, since the final project reports are not due until the day of the final exam May 5. Nevertheless, each presentation should contain some definite results to date as well as plans for how the presenter will complete his project. Each presenter should indicate specific tasks they plan to complete and discuss in their final report, and those they will do if time permits. They should also cite the references found most valuable in pursuing the project research and what results in those references are most significant. Presentations late in the cycle, eg. those given April 22, are expected to be more complete than those earlier, eg. April 15.The presentation checklist indicates the items on which the presentations will be graded. Each presenter will be sent an email within 24 hrs of his/her presentation indicating the grade for the presentation and comments on the presenter's plans for completing the project.
