Neural Networks – A.A. 2013 / 2014

Older editions:
Neural Networks – A.A. 2012/2013

Overview

This page contains the material for the lectures that I held in the Neural Networks course taught by Prof. Aurelio Uncini for the M.Sc. in Artificial Intelligence and Robotics. For general informations about the course please refer to the original web pages.

This section of the course will cover the ideas behind kernel methods in Machine Learning, especially Support Vector Machines. To this end, we will also explore the fundamental concepts of Statistical Learning Theory, such as the consistency of learning algorithms and their generalization capabilities, and of Tikhonov regularization theory. We will conclude by looking at other views on learning, such as the Bayesian or the Information-Theoretic frameworks.

For additional material on each topic, the slides are always concluded by a reference section with a selection of articles, books and other informations.

Syllabus

(click on the lecture title to download the slides):

Lecture Arguments Date
Lecture 0-A Elements of Unconstrained Optimization (definitions of unconstrained optimization; characterization of minima; first and second-order methods; conjugate gradient methods.)
Additional material: Example of Unconstrained Optimization in Matlab
03/10/2013
Lecture 0-B Elements of Constrained and Convex Optimization (definitions of constrained and convex optimization; characterization of minima of convex OP; duality theory; KKT conditions; penalty and barrier methods.)
Additional material: Example of Constrained Optimization in Matlab
10/10/2013
Lecture 1 Learning Problem and Loss Functions (definition of a learning algorithm; Empirical Risk Minimization; examples of loss functions) 17/10/2013
Lecture 2 Consistency of Learning Algorithms (key theorem of Statistical Learning Theory; VC Dimension; Structural Risk Minimization) 24/10/2013
Lecture 3 Geometrical Derivation of Support Vector Machines (geometric margin; maximal margin hyperplane; primal and dual optimization problems)
Additional material: Example of an SVM in Matlab
31/10/2013
Lecture 4 Kernels, Feature Spaces and Reproducing Kernel Hilbert Spaces (elements of functional analysis; definitions of a kernel; Reproducing Kernel Hilbert Spaces) 07/11/2013
Lecture 5 Regularized Learning Methods (regularization theory; kernel ridge regression; Bayesian learning; model selection) 21/11/2013

Projects

The course requires a small project to be performed by the students, eventually in groups of maximum 3 people. Several projects are available on the topics discussed in class. Please contact me if you are interested.