Artificial Neural Networks and Deep Learning

Synthetic program:

Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm. These topics will be described in the course according to the following detailed program:

-From the Perceptron to Neural Networks and the Feedforward architecture
-Backpropagation and Neural Networks training algorithms, e.g., Adagrad, adam, etc.
-Best practices in neural network training: overfitting and cross-validation, stopping criteria, weight decay, dropout, data resampling and augmentation.
-Image Classification problem and Neural Networks
-Recurrent Neural Networks and other relevant architectures, e.g., Radial Basis Functions, (Sparse) Neural Autoencoders
-Theoretical results: Neural Networks as universal approximation tools, vanishing and exploding gradients, etc.
-Introduction to the Deep Learning paradigm and its main differences with respect to classical Machine Learning
-Convolutional Neural Networks (CNN) architecture
-The breakthrough of CNN and their interpretation
-CNN training and data-augmentation
-Structural learning, Long-Short Term Memories, and their applications to text and speech
-Autoencoders and data embedding, word2vec, variational autoencoders
-Transfer Learning for pre-trained Deep models
-Extended models including Fully Convolutional CNN, networks for image segmentation (U-net) and object detection (e.g., R-CNN, YOLO )
-Generative Models (e.g., Generative Adversarial Networks)