Syllabus - Data Engineering (CY-703 (C))

CSE-Cyber Security /Cyber Security

Data Engineering (CY-703 (C))

VII

UNIT 1

Introduction to Data Engineering

Definition, Evolution, Life Cycle, Data Engineering skills and activities, Data Maturity, Data Lifecycle Versus the Data Engineering Lifecycle, Security, Data Management, DataOps.

UNIT 2

Source Systems and Data Ingestion

Types of Data Architecture, Data Lake,Data Lakehouses,Modern Data Stack, Lambda Architecture, Kappa Architecture.

UNIT 3

Data Platforms, Stream-to-Batch Storage Architecture, Data Catalog, Data Sharing,Data Modeling,Dimensional Modeling,Creating Tables,Schema Migration,Building the data warehouse.

UNIT 4

Data Ingestion, SFTP and SCP, Webhooks, Web Interface, Web Scraping Business Intelligence Tools,Introduction to Superset,Creating visualizations,Data Quality, Data Catalogs, Data Lineage, and Data Governance.

UNIT 5

ETL, Reverse ETL, Security, Privacy, and the Future of Data Engineering, Patch and Update Systems, Logging, Monitoring, and Alerting.

Practicals

Reference Books

  • Fundamentals of Data Engineering by Joe Reis, Matt Housley Released June 2022, and Publisher: O'Reilly Media, Inc. NISBN: 9781098108304.

  • Data Engineering with Python: Work with Massive Datasets to Design Data Models and Automate Data Pipelines Using Python by Paul Crickard.