Module 3: data analysis and machine learning

[Back to main page]

Lectures

• day 1 - Machine Learning Trends – Generative Adversarial Networks (GANs)
  • Supervised and unsupervised learning - Density estimation - Prescribed models - Latent variable models - Generative Adversarial Networks (GANs)
• day 1 - GANs - Implicit Generative Modeling for Password Estimation
  • PassGANs
• day 1 - Machine learning and deep learning
  • Machine learning review: linear regression - overfitting and underfitting - loss function - risks - estimation vs. approximation - bias and variance - regularization - training, validation and test - supervised learning - unsupervised learning
• day 2 - Deep feedforward networks - introduction
  • Basic building blocks of neural networks (definition, design, training): cost function and output units - network design - optimization
• day 3 - Deep feedforward networks - regularization and optimization / Convolutional Neural Networks (CNNs)
  • Regularization in feedforward neural networks: parameters - optimization - dataset augmentation - I/O noise - semi-supervised learning - early stopping
  • Optimization in feedforward neural networks: batch and mini-batch algorithms - stochastic gradient descent - weight initialization
  • CNNs: convolutions (standard, unshared, tiled) - stride - padding - pooling
• day 4 - Deep unsupervised learning / Generative Adversarial Networks (GANs) / Self-supervised learning / Deep Recurrent Neural etworks (RNNs)
  • Deep unsupervised learning: autoencoders - generative models (variational autoencoders)
  • GANs: generative modeling - principles of adversarial Learning - issues: vanishing gradient and mode collapse
  • Self-supervised learning: principles - overview of the literature - recent developments - transfer learning
  • Deep RNNs and sequence Modeling

Tutorials (juypter notebooks)

• day 1 - tutorial 1: (jupyter notebook, pdf)
  • Introduction to TensorFlow
• day 1 - tutorial 2: (jupyter notebook, pdf)
  • Optimization in TensorFlow - Introduction to neural networks in TensorFlow
• day 2 - tutorial 3: (jupyter notebook, pdf)
  • Handwritten digit recognition
• day 2 - tutorial 4: (jupyter notebook, pdf)
  • Convolutions - Image recognition with Inception
• day 3 - tutorial 5: (jupyter notebook - v1, jupyter notebook - v2, pdf - v1, pdf - v2)
  • Deep models - Image classifier using pretrained Inception model
• day 4 - tutorial 6: (jupyter notebook, pdf)
  • Recurrent neural networks to predict/generate text sequence

Other links

• LeCun’s lectures about deep learning at Collège de France
• Gdrive repository with pdf of LeCun’s talks
• Pattern Recognition and Machine Learning course from PACE University
• Deep learning course from Stanford University
• personal reading notes: unordered information retrieved from various web sites (mostly about CNN, RNN, layer output size calculation, Tensorflow implementation and batches)
• personal project: final notebooks for my project (CNN)

Some books of interest

• The Elements of Statistical Learning by Hastie et al. (available online and as pdf)
• Neural networks and deep learning by Michael Nielsen (available online or as a compiled version)
• Neural networks and deep Learning by Aggarwal (pdf available online; slides for each chapter also available here)
• Deep learning - a practicioner’s approach by Patterson and Gibson (can be found online here or here)
• Deep learning by Goodfellow et al. (available online or as a compiled version)
• Pattern recognition and machine learning by Bishop (can be found online)
• Dive into deep learning by Zhang et al. (available online with pdf and notebooks)
• Tensorflow RIP Tutorial (available as ebook)

Contact

Questions, comments or suggestions ? Don’t hesitate to contact me !