Homework 7

Description

1. 4. Use the design parameters that you chose in the first part and train a neural network, but this time set early-stopping=True. Research what early stopping is, and compare the performance of your network on the test set with the previous network. You can leave the validation-fraction as the default (0.1) or change it to see whether you can obtain a better model. Remember to standardize your features. Report your R² on both training and test sets. (10 pts)

Note: there are a lot of design parameters in a neural network. If you are not sure how they work, just set them as the default of sklearn, but if you use them masterfully, you can have better models.

5. Optional Programming Assignment: (Deep) CNNs for Image Colorization. This part will not be graded.

5. 1. This assignment uses a convolutional neural network for image colorization which turns a grayscale image to a colored image.² By converting an image to grayscale, we loose color information, so converting a grayscale image back to a colored version is not an easy job. We will use the CIFAR-10 dataset. Downolad the dataset from http://www.cs.toronto.edu/_~kriz/cifar-10-python.tar.gz.

5. 2. From the train and test dataset, extract the class birds. We will focus on this class, which has 6000 members.

5. 3. Those 6000 images have 6000 ⇥ 32 ⇥ 32 pixels. Choose at least 10% of the pixels randomly. It is strongly recommended that you choose a large number or all of the pixels. You will have between P = 614400 and P = 6144000 pixels. Each pixel is an RGB vector with three elements.

5. 4. Run k-means clustering on the P vectors using k = 4. The centers of the clusters will be your main colors. Convert the colored images to k-color images by con-verting each pixel’s value to the closest main color in terms of Euclidean distance. These are the outputs of your network, whose each pixel falls in one of those k classes.³

5. 5. Use any tool (e.g., openCV or scikit-learn) to obtain grayscale 32 ⇥ 32 ⇥ 1 images from the original 32 ⇥ 32 ⇥ 3 images. The grayscale images are inputs of your network.

5. 6. Set up a deep convolutional neural network with two convolution layers (or more) and two (or more) MLP layers. Use 5 ⇥ 5 filters and a softmax output layer. Determine the number of filters, strides, and whether or not to use padding your-self. Use a minimum of one max pooling layer. Use a classification scheme, which means your output must determine one of the k = 4 color classes for each pixel in

your grayscale image. Your input is a grayscale version of an image (32 ⇥ 32 ⇥ 1) and the output is 32 ⇥ 32 ⇥ 4. The output assigns one of the k = 4 colors to each of the 32 ⇥ 32 pixels; therefore, each of the pixels is classified into one of the classes [1 0 0 0], [0 1 0 0], [0 0 1 0], [0 0 0 1]. After each pixel is classified into one of the main colors, the RGB code of that color can be assigned to the pixel. For example, if the third main color ⁴ is [255 255 255] and pixel (32,32) of an image has the one-hot encoded class [0 0 1 0], i.e it was classified as the third color, the (32,32) place in the output can be associated with [255 255 255]. The size of the output of the convolutional part, c₁ ⇥ c₂ depends on the size of the convolutional layers you choose and is a feature map, which is a matrix. That matrix must be flattened or reshaped, i.e. must be turned into a vector of size c₁c₂ ⇥1, before it is fed to the MLP part. Choose the number of neurons in the first layer of the MLP (and any other hidden layers, if you are willing to have more than one hidden layer) yourself, but the last layer must have 32 ⇥ 32 ⇥ 4 = 4096 neurons, each of which represents a pixel being in one of the k = 4 classes. Add a softmax layer⁵ which will choose the highest value out of its k = 4 inputs for each of the 1024 pixels; therefore, the output of the MLP has to be reshaped into a 32 ⇥ 32 ⇥ 4 matrix, and to get the colored image, the RGB vector of each of the k = 4 classes has to be converted to the RGB vector, so an output image will be 32 ⇥ 32 ⇥ 3. Train at least for 5 epochs (30 epochs is strongly recommended). Plot training, (validation), and test errors in each epoch. Report the train and test errors and visually compare the artificially colored versions of the first 10 images in the test set with the original images.⁶

7. Extra Practice: Repeat the whole exercise with k = 16, 24, 32 colors if your computer can handle the computations.