Description

This homework asks you to fill in portions of code to classify an image dataset using three classifiers: the KNN, the SVM, and the MLP classifiers.

hw10 root folder
- data
  - train_data.npy
  - train_labels.npy
  - test_data.npy
  - test_labels.npy
- hw10.py
- README.md
- utils.py
- .gitignore
- mnist.jpg

Goals

In this assignment you will:

Get familiar with dealing with image datasets.
Learn the different stages of image classifications.
Learn how to use some classifiers available in Python’s sklearn package.

Background

Dataset

For this homework, we will be using the MNIST image dataset. Here is a brief description of this dataset from the official website:

The MNIST dataset consists of 70000 28×28 grayscale images of handwritten digits. There are 60000 training images and 10000 test images. Here are some sample images from the dataset.

To reduce your code runtime on this homework, we have provided you with the MNIST subset with 5000 images for training and 1000 images for testing. For each of the following tasks, you have to complete the corresponding function definition and return the classification accuracy and confusion matrix using the provided test set. While accuracy is a good way to quickly evaluate the performance of a classifier, it hides the detail you need to better understand the performance of your classification model i.e., its performance in individual classes. Therefore, we use the confusion matrix in addition to accuracy.
For more information see this link.

K-Nearest Neighbor (KNN)

k-nearest neighbor (kNN) is a classifier that requires no assumptions about the data (e.g., you don’t need to know the image labels during training). It looks at the classes of the k-nearest points and picks the most frequent one. The choice of the value of k determines how representative your algorithm is. Please refer to the class notes for a more elaborate definition. You may want to look into this documentation for the homework.

Support-Vector Machine (SVM)

SVM is a supervised machine learning algorithm which strives to increase the maximum margin hyperplane separating the data classes while training. Please refer to the class notes for more details. You may want to look into this documentation to learn more about its implementation.

Multi-layer Perceptron (MLP)

MLPs are supervised classification techniques which involve layering of multiple perceptron unit. This helps in solving non-linear classification tasks as well as linear classification. Please refer to your class notes for more details. You may want to look into this documentation for your code.

Instructions

Packages

If you don’t have the scikit-learn and scikit-image packages already installed in your environment, you can use the following commands to install them:

<code>python3 -mpip install --user scikit-learn
python3 -mpip install --user scikit-image
</code>

Note: if you’re on Windows, just use pip3 install scikit-learn and pip3 install scikit-image

Set up your repository for this homework.

The repository should contain the following files:

This README.md.
utils.py which contains a code for you to explore the provided dataset.
hw10.py which contains function definitions for KNN, SVM and MLP.
data folder which contains the MNIST subset.
.gitignore file will ignore the data folder in your commit(s). You don’t have to do change anything in this file, you can use your usual git commands and/or interface for pushing your code.

Homework Problem 0: Exploring your dataset

In this task, we ask you to run the utils.py file to visualize some sample images from the filtered MNIST dataset with 5000 images for training and 1000 for testing. dataset. By default, the code will display a 5×5 grid of random images. Explore your dataset in this way and see what do the images look like. You can increase/decrease the number of images displayed as you wish.

Note: you do not need to submit anything or fill in any code for this task. This is just for your own exploration.

Homework Problem 1: The K-nearest-neighbor classifier

In hw10.py, complete the skeleton function get_KNN_results for classifying the dataset using the k-nearest-neighbor where k=5. Your task is to classify the MNIST dataset using KNN for values of k For the given value of k, return the accuracy and the confusion matrix. For example, for k=5 the accuracy and confusion matrix are:

<code>Accuracy: 0.91
[[ 83   0   0   0   0   0   2   0   0   0]
 [  0 126   0   0   0   0   0   0   0   0]
 [  4   5  95   1   1   0   2   6   2   0]
 [  0   2   1  97   0   2   1   2   0   2]
 [  0   1   0   0 100   0   1   1   0   7]
 [  1   1   0   0   2  80   0   0   2   1]
 [  3   0   0   0   1   0  83   0   0   0]
 [  0   6   1   0   1   1   0  88   0   2]
 [  3   2   0   4   2   3   2   0  70   3]
 [  0   0   0   0   2   0   0   2   2  88]]

</code>

Homework Problem 2: The Support-Vector Machine classifier

In hw10.py, complete the skeleton function get_SVM_results for classifying the MNIST dataset using the SVM classifier. Your task is to fill in the missing parts of the code and return the accuracy and the confusion matrix for SVM.

Note: ignore any warning message you might encounter about convergence.

Homework Problem 3: The Multi-layer Perceptron classifier

In hw10.py, complete the skeleton function get_MLP_results for classifying the MNIST dataset using the MLP classifier. Your task is to fill in the missing parts of the code and return the accuracy and the confusion matrix for SVM.

Note: ignore any warning message you might encounter.

Steps to submit your work

Push your completed version of hw10.py

Python Programming Question