7 Support vector machines

Page 1

7 Support Vector Machines In this exercise session we will explore the possible option we have when we consider as classification tool the Support V ector Machines. 7.1 Practical Exercises Let us consider the problem of classifying the first two species of the Iris dataset (Setosa and Versicolor). At first, we load only the data corresponding to the two considered classes and we properly set the targets: 1 load iris_dataset; 2 3 irisInputs = zscore(irisInputs(1:2,1:100)'); 4 irisTargets = [ones(50,1); -ones(50,1)]; 5 gplotmatrix(irisInputs,[],irisTargets); As usual we normalize the data. Since we are going to use SVM we have to perform this operation which could considerably change the final solution and the influence the convergence rate (for instance, see Figure 7.2(b)). In this case we would like to resort to a linear SVM to classify the points, thus we have: • Hypothesis space: yn =f (xn,w ) = sign ( wT mxn +b ) ; • Loss measure: ||w||2 +C∑ iζi s.t.tn(wTxn +b) ≥ 1 −ζi ∀n; • Optimization method: Quadratic optimization. TheMATLAB function we use is fitcsvm, which considers a linear SVM (without ker- nel andC = 1 as default) if called with only two parameters: 1 svm_model = fitcsvm(irisInputs, irisTargets); With the optionverbose we are able to check the iterations performed during the op- timization procedure, for instance by inspecting the number of support vectors during the procedure or the total amount of violation of the constraints ∑ iζi. If we want to 1 7 Support V ector Machines visualize the classification boundary induced by the SVM, we should extract the model parametersw and the biasb stored in the model: 1 w = svm_model.Beta; 2 b = svm_model.Bias; We finally visualize the boundary between the two classes, as well as the margins: 1 figure(); 2 pos_class =…