Computer vision and robotics

Artificial Intelligence and machine learning

(Multimedia Technology)

(Algorithms and Systems Analysis)

(Computational and Cognitive Vision and Robotics)

Introduction
Generative models (GMs) are widely used in many subfields of AI, Machine Learning and Data Science. Recent advances in parameterizing these models using deep neural networks, combined with progress in stochastic optimization methods, have enabled scalable modeling of complex, high-dimensional data. In this course, we will study the probabilistic foundations and learning algorithms for deep generative models, including generative adversarial networks (GANs), variational autoencoders (VAEs), normalizing flows (NFs), deep autoregressive (DAR) models, diffusion probabilistic models (DPMs), and energy-based models (EBMs). The course will also discuss application areas that have benefitted from deep generative models, including speech processing, computer vision, domain adaptation, and inverse problem solving. Demonstrations of deep generative models and code execution will be established. This course includes lectures, research papers, assignments, a project and a final exam.
 
Context

• (Probability basics): Random variables, conditional probability, chain rule, Bayes theorem, central limit theorem, multivariate Gaussian, PDF of a transformed random variable, inverse transform sampling
• (Shallow generative models): Gaussian mixture models, Expectation-Maximization algorithm, maximum log-likelihood estimation, basics on Monte Carlo, Metropolis algorithm & Monte Carlo Markov chains
• (Deep Autoregressive (DAR) models): Definition, time-series generation as the benchmark example, capturing long-range correlations, NN architectures (dilated CNNs, RNNs & transformers), MLE optimization, discrete vs continuous state space, conditional DAR models and presentation of WaveNet, WaveRNN, PixelRNN, presentation of the architectures, present the training algorithm
• Variational Autoencoders (VAEs)): Definition of an autoencoder, definition of VAE, MLE approximation, training algorithm, Denoising VAEs, beta-VAE, applications in scientific discovery
• (Normalizing Flows (NFs)): Definition, invertible transformations, tractable exact MLE, derive general formula, present the training algorithm, how it can extend the VAE model
• (Information theory basics): what is information?, Shannon entropy, divergences (KLD, f, alpha, Renyi), variational representation/duality formulas (scale well with dimension while density ratio type of methods collapse when applied to high dimensional datasets), probability distances (Wasserstein, MMD), examples with Gaussians and/or the exponential family of distributions
• (Generative Adversarial Nets (GANs)): Define it as a minimization problem, intractable due to unknown PDFs, use variational formulas, minimax optimization, vanilla GAN, training algorithm (stochastic gradient descend/ascend), basic properties, Wasserstein GAN, Conditional GAN, DCGAN, BigGAN, MelGAN, InfoGAN, CycleGAN (as a more general adversarial type of learning)
• (Energy-based models (EBMs)): Definition, architectures, training algorithms (score matching algorithm, contrastive estimation), product of experts

• (Diffusion probabilistic models (DPMs)): Definition, forward/reverse process, MLE approximation, denoising DPMs, applications, DALL-E2 & Imagen models for text-to-image generation

LEARNING OUTCOMES
1. Knowledge: Having attended and succeeded in the course, the student is able to describe the probabilistic foundations of deep generative models and gain knowledge about various model architectures, training algorithms and their underlying principles.
2. Understanding: Having attended and succeeded in the course, the student is able to comprehend the application areas of deep generative models in fields like computer vision, language and speech processing.
3. Application: Having attended and succeeded in the course, the student is capable of applying learned concepts to implement and train deep generative models and utilize these models for tasks such as synthetic tabular data generation, time-series synthesis and generative image processing.
4. Analysis: Having attended and succeeded in the course, the student is able to analyze and compare different deep generative models, understanding their strengths and limitations and critically assess the performance of these models in various scenarios.
5. Synthesis: Having attended and succeeded in the course, the student is able to develop new approaches in deep generative modeling and distill information from research papers and practical demonstrations to create innovative solutions.
6. Evaluation: Having attended and succeeded in the course, the student is able to critically evaluate the effectiveness of deep generative models in real-world applications and assess the impact of these models in advancing the field of generative AI.

STUDENT PERFORMANCE EVALUATION
Class attendance/participation
Exercises/projects
Oral/written exams