Complete course notes

Page 1

E d i t e d b y : M a r c o V a r r o n e S o f t C o m p u t i n g N e u r a l N e t w o r k s C o u r s e N o t e s These notes have been made thanks to the effort of Polimi Data Scientists staff. Are you interested in Data Science activities? Follow PoliMi Data Scientists on Facebook ! Polimi Data Scientist is a community of students and Alumni of Politecnico di Milano. We organize events and activities related to Artificial Intelligence and Machine Learning, our aim is to create a strong and passionate community about Data Science at Politecnico di Milano. Do you want to learn more? Visit our website and join our Telegram Group ! ! Credits The following notes have been written by the Polimi Data Scientists student association by combining Prof. Matteucci’s lectures, slides and multiple online resources, such as: • https://blog.paperspace.com/vanishing-gradients-activation-function • https://colah.github.io/posts/2015-08-Understanding-LSTMs/ • https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1 1 Contents 1 Introduction to Machine Learning 3 1.1 Learning Paradigms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.2 Choosing model complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Maximum Likelihood Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Maximum Likelihood Estimation (MLE) . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 The Perceptron 7 2.1 The biological neuron . . . . . . . . . . . . . . . . . . . . . . . . . .…