FINAL PROJECT Choose your own Computer Vision adventure Solution

Description

The objective of the final project is to make use of what you have learned during this course, and to develop some creative new applications of computer vision. The project will be demo’ed for your fellow classmates during the last week of classes. You will work in groups of one or two. Our recommendation is for you to work in one of two project areas:

1. The Photo Sorter project

2. An interactive project of your own choice that includes either

1. • An object (or face) recognition/classification task, or

1. • An implementation of an algorithm from literature.

Grading will be based on the completeness of the project, the level of functionality attempted, the clarity and thoroughness of the report, and your in-class demo. Further specifics on grading are given for each project below.

Project Details – Option A. Photo Sorter

Over the last decade, digital cameras have overtaken traditional film cameras. As a result, we are amassing ever larger consumer archives of photos as well as large online repositories such as Instagram, Facebook, and Google Photos. Facebook is currently the largest photo online storage site, with over 300 million photos updated every day (https://zephoria.com/top-15-valuable-facebook-statistics/).

Digital cameras have tended to alter people’s photo-taking behavior. Since images are now “free”, most people take more pictures. They also take many duplicates in the hope of getting one “really good” shot. But, as these photos come off the camera, they tend to pile up on the drive without any useful sorting or clustring. This is where you come in.

The project’s objective is to develop some photo sorting tools. The project ground rules are as follows:

• You will be supplied with a folder of digital images. https://drive.google.com/drive/folders/0BxmcryMGzBBOVDZTVUlwdzRwUmc?usp =sharing

• Your job will be to detect common attributes among photos and use them to create “clusters” of similar photos. Optionally, you can also apply labels to the clustered photos or save them into subfolders.

• You will also create an application interface that can be demonstrated. The idea here is to have a menu/choice driven application with functionality such as:

o load the images set o sort images

o select only:

• • outdoor scenes

  • portraits (of one or a group of people)

• • text documents

• • images containing cars

  • images containing buildings, etc.

15. add image to the set and re-sort, etc.

Note: these are only ideas and suggestions; you are free to come up with your own functions and tasks

• You can use the programming language of your choice.

There are three levels of grading for this project:

1. Baseline: The baseline project is to detect duplicate photos and to group them. Duplicates may differ in small details, but the overall composition is the same. A project that does this well will get 75% credit.

2. Preferred: The preferred project will perform baseline operations, but also be able to detect common portions of images and group based on those. It will detect that scenes contain the same people or objects (but might differ in composition) and will group based on those objects. This does not need to be a general-purpose functionality. That is, it suffices, for example, to recognize some small set of people, and it is ok to assume the people have not altered their appearance. A project that exhibits these capabilities would get 100% credit.

3. Deluxe: A deluxe project would add additional functionality, a nice user interface, interactivity, or interfaces to other tools (Facebook, Flickr, Instagram etc.). Example: implement interactive tools where, for example, the user can click or circle a part of the image, and then use the properties of that image patch later in your algorithm. Credit for deluxe items will be given on a case-by-case basis.

Project Details – Option B. Interactive project of your choice

Goals: Over the course of the semester we have learned a lot of techniques that are used in computer vision today. With this set of tools you can actually solve some interesting and challenging problems. The final project is your chance to apply some of these tools to a problem that is of interest to you. There are a number of components to a good project:

• Choosing the project: It’s hard to choose something that is both interesting and doable in 2.5 weeks. Ask for help with this process.

• Formalizing the problem: How do you take an idea like “detecting mouths in a single image” and formalize it and then turning this into something that can be computed. This is connected with the algorithm development part.

• Data collection: You will need data with which to test your algorithm. The safest bet is to use data sets that others have already gathered and made available and that have been used for a similar task to yours. Ask us if you need help. It’s not always the best idea to collect new data because it can be time consuming and increase the challenges of your project.

• Algorithm development: You can use the programming language of your choice. Set incremental goals. Test your software in stages. Use good programming practices (don’t let Matlab/Python make you lazy). Try to break your code! If it works the first time don’t believe it; try strange/edge cases.

• Experimentation: You want to show us experiments that illustrate the ideas and the behavior of your method. Don’t overwhelm us with results but rather choose good illustrative examples (of success and failure).

• Write-up: A key part of any research project is effective communication of the results. Unlike most homework assignments you’ve done so far, your grade will be determined in a big part by your write-up.

Tips: Don’t get carried away. Choose something that seems small and then set intermediate goals. You want to have a fallback strategy if you don’t meet your ultimate goals. These projects should not be basic research. If nobody has ever done this before then the risks are too high for a final project. Choose something from the literature that seems doable and implement that.

Note: the Web is a great source of ideas, papers, data, and even code. If you do get data, code, or ideas from the web you MUST disclose this. Become a fanatic about appropriately citing work from any source. Your write-up should specify in detail your sources and your personal contribution to every task in your algorithm. If we find any portion of your solution code on a website, it means you have violated the Honor Code rules.

Submission Guidelines (for all projects):

You final report should document the following:

15. Your initial project idea and project goals.

15. Your algorithm/approach and your hypothesis (what will work, what do you

think you’ll be able to achieve).

15. The data analysis and the results: which goals you managed to achieve. o The computer vision methods you used to achieve these goals.

o If things did not work out, try to offer a reason for it.

o Known limitations of the methods you have used and possible future extensions.

o Your contributions to the project (should be separate for each team member) o The two or three things you learned during this project.

15. Advice you would give to next year’s vision students.

During the last week of classes each team is required to make a presentation of their project. You can choose to demo your project or (if you are a remote distance student) have a recorded video (screencast demo) of how your project works. Once again, focus on what your goals were at the beginning, challenges encountered and your final results.

Note1: If your project will make use of a neural network (CNN or otherwise), it is not sufficient to present just your results. You need to argument your choice of parameters for your network and explain how the training was done. Training from scratch won’t work very well and otherwise you’ll have to fine tune a pre-trained existing network. Keep in mind the purpose of this project is to put together everything you learned this semester, in order to build a classification pipeline.

Note2: If you are combining your course project with the project from another class, you must receive permission from both instructors, and clearly explain in the report the exact portion of the project that is being counted for CSCI 4831/5722. In this case you must prepare separate reports for each course and submit your final report for the other course as well.

You should submit:

1. Your solution code, as a zip file, via Moodle before Thursday, April 25th, by 11:55pm

2. Your written report, as a pdf file, via Moodle before Sunday, April 28th, by 11:55pm

3. (For remote distance students) Your video, via Google form before Tuesday, April 30th, by 11:55pm