This class is a mandatory part of several curricula, e.g., Artificial Intelligence (Bachelor program); Electronics and Information Technology (Bachelor program); Computer Science (Master program, specialisations "Data Science" and "Computational Engineering"), and is also open to other interested students in search of a basic (and very practical) introduction to the fundamental concepts of supervised machine learning and classification.

The class is taught by Gerhard Widmer, opens an external URL in a new window and colleagues in the summer term. Language is English.

Information for the current semester (if available):

Goals of this class

The lecture offers a simple and practical introduction to the field of supervised machine learning and, in particular, classification. It covers some of the most fundamental concepts and methods in the field, and demonstrates the application of these methods in a variety of complex tasks, mainly from the area of audio perception and music information retrieval (which are the institute's special research focus). 
The lecture is accompanied by a practical track where the students carry out a pattern classification project of real-world complexity, on real-world data, in several stages, from feature definition and extraction to the training of various classifiers and systematic experimentation. 
After attending the class, the students should have a basic understanding of the central issues in pattern classification, should be able to read the scientific literature on learning and classification, and should have acquired the basic skills needed to realise and evaluate pattern classification systems. 

Contents

Basic notions and methods of machine learning, pattern classification, and statistical data modelling, including

• Bayes classification and Bayes error
• Parametric densitiy estimation, maximum likelihood, the Naive Bayes Classifier
• Non-parametric density estimation, nearest-neighbour methods, the k-NN Classifier
• Standard classifiers and ensemble methods in machine learning (decision trees, support vector machines, random forests, ...)
• Basic ideas of Neural Networks and Deep Learning
• Classifier evaluation and model selection
• Clustering, dimensionality reduction, mixture models
• Markov processes and Hidden Markov Models (HMMs)
   

Prerequisites

Basic knowledge of algorithms, probability, stastics and linear algebra. The practical project will require basic knowledge of the Python programming language, in order to be able to use machine learning toolboxes such as scikit-learn; a short introduction to fundamental concepts of Python and scikit-learn will be given as part of the class.

Course Material

PDF versions of the powerpoint slides used in the lecture will be made available electronically, via KUSSS.

Interested students may also want to consult the following books (but this is in no way required for the class):

• C. Bishop (2006). Pattern Recognition and Machine Learning. New York: Springer Verlag.
• C. Bishop (1995). Neural Networks for Pattern Recognition. Oxford University Press.
• R. Duda, P. Hart, & D. Stork (2001). Pattern Classification (2nd Edition), opens an external URL in a new window. NY: Wiley & Sons.
• T. Hastie, R. Tibshirani, & J. Friedman (2001). The Elements of Statistical Learning. New York: Springer Verlag.
• Murphy, K.P. (2012). Machine Learning: A Probabilistic Perspective. Cambridge, MA: MIT Press.
• S. Russell & P. Norvig (2002). Artificial Intelligence: A Modern Approach (2nd Edition). Prentice-Hall.