Syllabus - Deep Learning (CD 601)

CSE-Data Science/Data Science

Deep Learning (CD 601)

VI

I

Introduction History of Deep Learning, McCulloch Pitts Neuron, MultilayerPerceptions (MLPs), Representation Power of MLPs, Sigmoid Neurons, Feed ForwardNeural Networks, Back propagation, weight initialization methods, Batch Normalization,Representation Learning, GPU implementation, Decomposition – PCA and SVD.

II

Deep Feedforward Neural Networks, Gradient Descent (GD), Momentum Based GD,Nesterov Accelerated GD, Stochastic GD, AdaGrad, Adam, RMSProp, Auto-encoder,Regularization in auto-encoders, Denoising auto-encoders, Sparse auto-encoders, Contractiveauto-encoders,Variational auto-encoder, Auto-encoders relationship with PCA and SVD,Dataset augmentation.Denoising auto encoders,

III

Introduction to Convolutional neural Networks (CNN) and its architectures, CCNterminologies: ReLu activation function, Stride, padding, pooling, convolutions operations,Convolutional kernels, types of layers: Convolutional, pooling, fully connected, VisualizingCNN, CNN examples: LeNet, AlexNet, ZF-Net, VGGNet, GoogLeNet, ResNet, RCNNetc.Deep Dream, Deep Art. Regularization: Dropout, drop Connect, unit pruning, stochasticpooling, artificial data, injecting noise in input, early stopping, Limit Number of parameters,Weight decay etc.

IV

Introduction to Deep Recurrent Neural Networks architectures,Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, TruncatedBPTT, Gated Recurrent Units (GRUs), Long Short Term Memory (LSTM), Solving thevanishing gradient problem with LSTMs, Encoding and decoding in RNN network, AttentionMechanism, Attention over images, Hierarchical Attention, Directed Graphical Models.Applications of Deep RNN in Image Processing, Natural Language Processing, Speechrecognition, Video Analytics.

V

Introduction to Deep Generative Models, Restricted Boltzmann Machines (RBMs),Gibbs Sampling for training RBMs, Deep belief networks, Markov Networks, MarkovChains, Auto-regressive Models: NADE, MADE, PixelRNN, Generative AdversarialNetworks (GANs), Applications of Deep Learning in Object detection, speech/imagerecognition, video analysis, NLP, medical science etc.

Course Objective

Introduce deep learning fundamentals and major algorithms, the problem settings, and their applications to solve real world problems.

Course Outcome

["Describe in-depth about theories, fundamentals, and techniques in Deep learning.", "Identify the on-going research in computer vision and multimedia field.", "Evaluate various deep networks using performance parameters.", "Design and validate deep neural network as per requirements."]

Practicals

  • Image Classification with CNN

  • Face Detection system with OpenCV library

  • Digit Recognition System with CNN

  • Music Genre Classification system (FMA: Free Music ArchiveDataset)

  • Image Compression and De-compression using Encoders and Decoders

  • Predicting Airline Passengers count based on LSTM and RNN

  • Diabetes detection in patients with functional and sequential implementation of Keras

  • Detecting customer churn on banking dataset with Deep Neural Network

  • Multiclass wine classification using Neural Networks

  • Breast Cancer Detection using Neural Network Architecture

Reference Books

  • Ian Goodfellow, YoshuaBengio and Aaron Courville; Deep Learning, MIT Press.

  • Charu C. Aggarwal "Neural Networks and Deep Learning: A Textbook", Springer.

  • Francois Chollet, "Deep Learning with Python", Manning Publications.

  • Aurelien Geon, and Tensorflow:Concepts, Tools, and Techniques to Build Intelligent Systems", O'Reilly.

  • Andreas Muller, "Introduction to Machine Learning with Python: A Guide for "Hands-On Machine Learning with Scikit-Learn DataScientists", O'Reilly.

  • Adam Gibson, Josh Patterson, "Deep Learning: A Practitioner's Approach", O'Reilly