Syllabus - Data Engineering (CY-703 (C))
CSE-Cyber Security /Cyber Security
Data Engineering (CY-703 (C))
VII
UNIT 1
Introduction to Data Engineering
Definition, Evolution, Life Cycle, Data Engineering skills and activities, Data Maturity, Data Lifecycle Versus the Data Engineering Lifecycle, Security, Data Management, DataOps.
UNIT 2
Source Systems and Data Ingestion
Types of Data Architecture, Data Lake,Data Lakehouses,Modern Data Stack, Lambda Architecture, Kappa Architecture.
UNIT 3
Data Platforms, Stream-to-Batch Storage Architecture, Data Catalog, Data Sharing,Data Modeling,Dimensional Modeling,Creating Tables,Schema Migration,Building the data warehouse.
UNIT 4
Data Ingestion, SFTP and SCP, Webhooks, Web Interface, Web Scraping Business Intelligence Tools,Introduction to Superset,Creating visualizations,Data Quality, Data Catalogs, Data Lineage, and Data Governance.
UNIT 5
ETL, Reverse ETL, Security, Privacy, and the Future of Data Engineering, Patch and Update Systems, Logging, Monitoring, and Alerting.
Practicals
Reference Books
-
Fundamentals of Data Engineering by Joe Reis, Matt Housley Released June 2022, and Publisher: O'Reilly Media, Inc. NISBN: 9781098108304.
-
Data Engineering with Python: Work with Massive Datasets to Design Data Models and Automate Data Pipelines Using Python by Paul Crickard.