CS 4476-B Computer Vision
Fall 2021, MW 12:30 to 1:45, CCB 16. Synchronous remote lecture on Bluejeans
Instructor: James Hays
TAs: Ben Wilson (head TA), Bharat Mamidibathula, Gunhyun Park, Jonathan Leo, Otis Smith,
Pranav Khorana, Sukriti Bhardwaj, Tony Zhang, Xueqing Li, Yash Kothari, Yoonwoo Kim
Course Description
This course provides an introduction to computer vision including fundamentals of image formation, camera imaging geometry, feature detection and matching, stereo, motion estimation and tracking, image classification and scene understanding. We'll explore methods for depth recovery from stereo images, camera calibration, automated alignment, tracking, boundary detection, and recognition. We'll use both classical machine learning and deep learning to approach these problems. The focus of the course is to develop the intuitions and mathematics of the methods in lecture, and then to learn about the difference between theory and practice in the projects.Learning Objectives
Upon completion of this course, students should be able to:- 1. Recognize and describe both the theoretical and practical aspects of computing with images. Connect issues from Computer Vision to Human Vision
- 2. Describe the foundation of image formation and image analysis. Understand the basics of 2D and 3D Computer Vision.
- 3. Become familiar with the major technical approaches involved in computer vision. Describe various methods used for registration, alignment, and matching in images.
- 4. Get an exposure to advanced concepts leading to object categorization and segmentation in images.
- 5. Build computer vision applications.
Prerequisites
No prior experience with computer vision is assumed, although previous knowledge of visual computing or signal processing will be helpful. The following skills are necessary for this class:- Data structures: You'll be writing code that builds representations of images, features, and geometric constructions.
- Programming: Projects are to be completed and graded in Python and PyTorch. All project starter code will be in Python. TA's will support questions about Python. If you've never used Python that is OK, as long as you have programming experience.
- Math: Linear algebra, vector calculus, and probability. Linear algebra is the most important and students who have not taken a linear algebra course have struggled in the past. If you want a refresher on linear algebra, check out Prof. Gilbert Strang's lectures.
Grading
Your final grade will be made up from- 80% 5 programming projects. First project 12% of grade, projects 2 through 5 17% of grade.
- 20% Problem sets / Quizzes.
You will lose 10% each day for late projects. However, you have six "late days" for the whole course. That is to say, the first 24 hours after the due date and time counts as 1 day, up to 48 hours is two and 72 for the third late day. This will not be reflected in the initial grade reports for your assignment, but they will be factored in and distributed at the end of the semester so that you get the most points possible.
These late days are intended to cover unexpected clustering of due dates, travel commitments, interviews, hackathons, etc. Don't ask for extensions to due dates because we are already giving you a pool of late days to manage yourself. In fact, we're doubling the pool of late days this semester because of the difficult circumstances.
Academic Integrity
Academic dishonesty will not be tolerated. This includes cheating, lying about course matters, plagiarism, or helping others commit a violation of the Honor Code. Plagiarism includes reproducing the words of others without both the use of quotation marks and citation. Students are reminded of the obligations and expectations associated with the Georgia Tech Academic Honor Code and Student Code of Conduct, available online at www.honor.gatech.edu. We will use tools to find code sharing in projects.You are expected to implement the core components of each project on your own, but the extra credit opportunties often build on third party data sets or code. That's fine. Feel free to include results built on other software, as long as you are clear in your handin that it is not your own work.
You should not view or edit anyone else's code. You should not post code to Canvas, except for starter code / helper code that isn't related to the core project.
Learning Accommodations
If needed, we will make classroom accommodations for students with documented disabilities. These accommodations must be arranged in advance and in accordance with the office of disability services. (disabilityservices.gatech.edu).Important Links:
- Canvas. For announcements, grades, and archived recordings of lecture
- Piazza. For discussion
- Gradescope. For project handin
Contact Info and Office Hours:
If possible, please use Piazza to ask questions and seek clarifications before emailing the instructor or staff.- James: hays[at]gatech.edu
- Ben Wilson: benjaminrwilson[at]gatech.edu
- James: Tuesday, 2pm, Bluejeans.
- TA hours: See pinned note in Piazza.
Projects |
| Optional Project 0: Test environment setup. Github |
| Project 1: Convolution and Hybrid images. Github, Handout pdf |
| Project 2: SIFT Local Feature Matching. Github, Handout pdf |
| Project 3: Camera Calibration and Fundamental Matrix Estimation with RANSAC. Github, Handout pdf |
| Project 4: Recognition with deep learning Github, Handout pdf |
| Project 5: Semantic Segmentation Github, Handout pdf |
Textbook
Readings will be assigned in "Computer Vision: Algorithms and Applications, 2nd edition" by Richard Szeliski. The book is available for free online or available for purchase.Syllabus
| Class Date | Topic | Slides | Reading | Projects |
| Mon, Aug 23 | Introduction to computer vision | pdf, pptx | Szeliski 1 | |
|
| ||||
| Wed, Aug 25 | Camera Projection and Image Filtering | pdf, pptx | Szeliski 2.1, especially 2.1.4 | |
| Wed, Aug 30 | Thinking in Frequency | pdf, pptx | Szeliski 3.2 and 3.4 | |
| Mon, Sep 1 | Thinking in Frequency, continued | |||
| Mon, Sep 6 | No classes, Institute holiday | |||
| Wed, Sep 8 | Light and Color and Sensors | pdf, pptx | Szeliski 2.2 and 2.3 | |
|
| ||||
| Mon, Sep 13 | Interest points and corners | pdf, pptx | Szeliski 7.1.1 and 7.1.2 | |
| Wed, Sep 15 | Local image features | pdf, pptx | Szeliski 7.1.3 | |
| Mon, Sep 20 | Model fitting, Hough Transform | pdf, pptx | Szeliski 7.4.2 and 2.1 | |
| Wed, Sep 22 | RANSAC and transformations | pdf, pptx | Szeliski 8.1 and 2.1 | |
|
| ||||
| Mon, Sep 27 | Stereo intro | pdf, pptx | Szeliski 12 and 11.2.1 | |
| Wed, Sep 29 | Camera Calibration, Epipolar Geometry | pdf, pptx | Szeliski 11 | |
| Mon, Oct 4 | Dense Stereo Correspondence | pdf, pptx | Szeliski 12 | |
| Wed, Oct 6 | Optical Flow | pdf, pptx | Szeliski 9.4 | |
|
| ||||
| Mon, Oct 11 | No classes, Institute holiday | |||
| Wed, Oct 13 | Machine learning crash course | pdf, pptx | Szeliski 5.1 and 5.2 | |
| Mon, Oct 18 | Machine learning crash course, part 2 and neural network intro | pdf, pptx | Szeliski 5.1 and 5.2 and 5.3 | |
| Wed, Oct 20 | Convolutional Neural Networks and Network Visualization | pdf, pptx | Szeliski 5.3 and 5.4 | |
| Mon, Oct 25 | Classical recognition techniques and Deeper Deep Architectures | pdf, pptx | Szeliski 6.2.1 | |
| Wed, Oct 27 | ResNet, Big Data | pdf, pptx | ||
| Mon, Nov 1 | Big Data, Crowdsourcing | pdf, pptx | Szeliski 6.3 | |
| Wed, Nov 3 | "Unsupervised" Learning and Colorization | pdf, pptx | ||
| Mon, Nov 8 | Semantic Segmentation | pdf, pptx | Szeliski 5.5 | |
| Wed, Nov 10 | Deep Object Detection and Structured Output from Deep Networks | pdf, pptx | Szeliski 5.4.7 | |
| Mon, Nov 15 | 3D Point Processing and Lidar | pdf, pptx | ||
| Wed, Nov 17 | Transformer architectures | pdf, pptx | ||
| Mon, Nov 22 | No lecture, work on Project 5 | |||
| Wed, Nov 24 | No lecture, Institute holiday | |||
| Mon, Nov 29 | Guest lectures: John Lambert (multiview geometry) and Amit Raj (neural rendering) | John Lambert pdf | ||
| Wed, Dec 1 | Guest lecture: Sean Foley (remote sensing) | |||
| Mon, Dec 6 | No lecture | |||
| Wed, Dec 8 | No lecture | |||
| Final Exam Period | Not used. No class or final exam | |||