Learning Objectives

• Design and build scalable data lakes and data warehouses on AWS.
• Construct, optimize, and secure batch data processing pipelines.
• Build and manage streaming data solutions.
• Apply data governance and security best practices.
• Automate data engineering workflows using AWS services.
• Configure access control and security measures for data solutions.

Course Outline

Module Module 1: Data Engineering Fundamentals and AWS Context

[‘Responsibilities of a data engineer’, ‘Data discovery within a data analytics system’, ‘AWS services used in data workflows’, ‘Continuous integration and continuous delivery’, ‘Networking considerations’]

Module Module 2: Building Data Lakes on AWS

[‘Data lake concepts and use cases’, ‘Storage options for data lakes’, ‘Ingesting data into a data lake’, ‘Cataloging data assets’, ‘Transforming data’, ‘Making data available for consumption’]

Module Module 3: Data Lake Optimization and Security

[‘Performance optimization techniques’, ‘Applying security with Lake Formation’, ‘Configuring permissions in Lake Formation’, ‘Security and governance practices’, ‘Troubleshooting common issues’]

Module Module 4: Data Warehouse Concepts and Design

[‘Data warehouse fundamentals’, ‘Overview of Amazon Redshift’, ‘Loading data into Amazon Redshift’, ‘Processing data in Amazon Redshift’, ‘Exposing data for downstream consumption’]

Module Module 5: Data Warehouse Performance Tuning

[‘Monitoring and optimization approaches’, ‘Data-level optimization in Amazon Redshift’, ‘Query optimization in Amazon Redshift’, ‘Orchestrating data workflows’]

Module Module 6: Data Warehouse Security and Access Management

[‘Authentication and access control in Amazon Redshift’, ‘Data security practices in Amazon Redshift’]

Module Module 7: Batch Data Pipeline Design

[‘Batch pipeline concepts and patterns’, ‘Designing a batch data pipeline’, ‘Ingesting batch data’]

Module Module 8: Building and Integrating Batch Data Pipelines

[‘Processing and transforming data’, ‘Converting between data formats’, ‘Integrating data from multiple sources’, ‘Cataloging processed data’, ‘Delivering data for consumption’]

Module Module 9: Batch Pipeline Optimization, Orchestration, and Security

[‘Optimizing batch pipeline performance’, ‘Orchestrating batch pipeline execution’, ‘Applying security controls to batch pipelines’]

Module Module 10: Streaming Data Pipeline Architecture

[‘Streaming pipeline concepts and use cases’, ‘Ingesting data from streaming sources’, ‘Storing streaming data’, ‘Processing streaming data’, ‘Analyzing streaming data’]

Module Module 11: Streaming Solution Optimization and Security

[‘Optimizing streaming data solutions’, ‘Securing streaming data pipelines’]

Module Module 12: Compliance and Cost Management

[‘Compliance requirements and considerations’, ‘Tools for managing and reducing costs’]

Module Module 13: Course Summary

[]