4 Linear classification

Page 1

4 Classification 4.1 Practical Exercises In this session, we take into account again the Iris dataset. This time we are more interested in the discrimination of the sample class, i.e., either Setosa, Versicolor or Virginica. We do not have any metric over the space of the classes, i.e., it is not possible to order them. In this case one could consider regression techniques. Instead, we consider clas- sification techniques. Let us start with discriminating between Setosa and not-Setosa by basing on the sepal length and width. 1 load iris_dataset.mat; 2 x = zscore(irisInputs([1 2],:)'); 3 t = irisTargets(1,:)'; 4 gplotmatrix(x,[],t); We will consider a set of methods which can be used when we want to discriminate between two classes, in the specific: • Single perceptron; • Logistic regression; • Naive Bayes; • K-nearest neighbour. 4.1.1 Perceptron At first, let us perform a classification with a perceptron classifier: • Hypothesis space: y(xn) = sgn(wT xn) = sgn(w0 +xn1w1 +xn2w2), • Loss measure: Distance of misclassified points LP (w) =− ∑ n∈M wT xnCn, • Optimization method: Online Gradient Descent, 1 4 Classification wheresgn(·) is the sign function. InMATLAB the perceptron is classified as a simple neural network with a single layer, thus can be found in the Neural Network T oolbox. 1 net = perceptron; 2 net = train(net,x',t'); The recap of the perceptron provides you information about the evolution of the ob- jective function reduction during the epochs and some performance measures. More specifically, to evaluate the performance of the chosen method we need to consider the confusion matrix which tells us the number of points which have been correctly classified and those which have been misclassified: Actual Class: 1 Actual Class: 0 Predicted Class: 1 tp fp Predicted Class: 0…