Syllabus - INFORMATION THEORY AND CODING (EC-702 (B))

Information Theory

Introduction to uncertainty, entropy and its properties, entropy of binary memoryless source and its extension to discrete memory-less source, Measure of information, Information content of message, Average Information content of symbols. Self information, Mutual information and its properties,

Coding theorem

Source coding theorem, prefix coding, Shannon’s Encoding Algorithm, Shannon Fano Encoding Algorithm, Huffman coding, Extended Huffman coding, Arithmetic Coding, Lempel-Ziv Coding, Run Length Encoding.

Information Channels

Communication Channels, Channel Models, Channel Matrix, Joint probability Matrix, Discrete memory less channels, Binary symmetric channel and its channel capacity, channel coding theorem, and its application to Binary Erasure Channel, Shannon’s theorem on channel capacity, capacity of channel of infinite bandwidth, Continuous Channels.

Error Control Coding

Introduction, Examples of Error control coding, methods of Controlling Errors, Types of Errors, types of Codes, Linear Block Codes: matrix description of Linear Block Codes, Error Detection and Error Correction Capabilities of Linear Block Codes, Probability of undetected error for linear block code in BSC, hamming Codes and their applications, Cyclic Codes: Cyclic codes and its basic properties, Encoding using an (n-k) Bit Shift register, Generator & parity check matrix of cyclic codes, encoding & decoding circuits, syndrome computation, error detection and correction,

Introduction to BCH codes, its encoding & decoding, error location & correction.

Convolution Codes: Introduction to convolution codes, its construction, Convolution Encoder, Time domain approach, Transform domain approach, Code Tree, Trellis and State Diagram, Viterbi algorithm: Introduction of theorem for maximum likelihood decoding.