16 01 2025 E T LAB

Page 1

Course: Numerical Analysis for Machine Learning Prof. E. Miglio - January 16th 2025 Duration of the exam: 2.5 hours. Exercise 1 Load the dataset contained in the file faces.mat using the following commands: import scipy.io as sio data = sio.loadmat('faces.mat') X = data['X'] The dataset contains a collection of 5000 32 × 32 grayscale face images. You can plot a single face as follows: import numpy as np import matplotlib.pyplot as plt x0 = np.transpose(np.reshape(X[0,:],(32,32))) plt.imshow(x0,cmap='gray') 1. Compute the normalized matrix ˜X. 2. Perform the PCA on ˜X and plot the first 25 eigenfaces. 3. Reduce the dimension of the sample from 1024 (32 by 32) to 100 by projecting the matrix ˜X onto U. 4. Plot the original images (pick the first 100 images) and the ones reconstructed from only the first 100 principal components. Plot also the error. Exercise 2 Consider the following simple linear network a(v, w) = w · v, (1) where w = [w, b]T is the parameter vector ( w is the weight and b the bias) and v = [x, 1]T is the input vector. Consider the samples ( x1, y1) = (2, 0.5) and ( x2, y2) = (−1, 0) and the cost function J(w) = (y1 − a(v1, w))2 + (y2 − a(v2, w))2, (2) where v1 = [x1, 1]T and v2 = [x2, 1]T . 1. Rewrite equation (2) as J(w) = 1 2 wT Aw + dT w + c (write explicitly A, d and c.). 2. Compute the value of the exact parameter vector w∗ that minimize J(w). 3. Plot the surface that represents J. 4. Implement the gradient descent method and use it to compute w∗: set the initial guess w(0) equal to [1, 1]T and the learning rate η equal to 0 .05. 5. What is the maximum value of the learning rate that can be used ? Exercise 3 Consider a logistic regression σ(βT x) = σ(β0 + β1x1 + β2x2) (3) where σ(c) = 1 1 + exp(−c). Let us consider the following sets of data: Set 1…