This course is designed to build your foundational skills in data engineering on Microsoft Fabric, focusing on the Lakehouse concept. This course will explore the powerful capabilities of Apache Spark for distributed data processing and the essential techniques for efficient data management, versioning, and reliability by working with Delta Lake tables. This course will also explore data ingestion and orchestration using Dataflows Gen2 and Data Factory pipelines. This course includes a combination of lectures and hands-on exercises that will prepare you to work with Lakehouses in Microsoft Fabric.
Who Should Attend
The primary audience for this course is data professionals who are familiar with data modeling, extraction, and analytics. It is designed for professionals who are interested in gaining knowledge about Lakehouse architecture, the Microsoft Fabric platform, and how to enable end-to-end analytics using these technologies.
Course Outline
Introduction to end-to-end analytics using Microsoft Fabric
- Introduction
- Explore end-to-end analytics with Microsoft Fabric
- Explore data teams and Microsoft Fabric
- Enable and use Microsoft Fabric
- Module assessment
- Summary
Get started with lakehouses in Microsoft Fabric
- Introduction
- Describe lakehouse features and capabilities
- Ingest and transform data in a lakehouse
- Query and analyze lakehouse data
- Exercise – Create a Microsoft Fabric lakehouse
- Module assessment
- Summary
Use Apache Spark in Microsoft Fabric
- Introduction
- Prepare to use Apache Spark
- Run Spark code
- Work with data in a Spark dataframe
- Work with data using Spark SQL
- Visualize data in a Spark notebook
- Exercise – Analyze data with Apache Spark
- Module assessment
- Summary
Work with Delta Lake tables in Microsoft Fabric
- Introduction
- Understand Delta Lake
- Create delta tables
- Optimize delta tables
- Work with delta tables in Spark
- Use delta tables with streaming data
- Exercise – Use delta tables in Apache Spark
- Module assessment
- Summary
Ingest Data with Dataflows Gen2 in Microsoft Fabric
- Introduction
- Understand Dataflows Gen2 in Microsoft Fabric
- Explore Dataflows Gen2 in Microsoft Fabric
- Integrate Dataflows Gen2 and Pipelines in Microsoft Fabric
- Exercise – Create and use a Dataflow Gen2 in Microsoft Fabric
- Module assessment
- Summary
Orchestrate processes and data movement with Microsoft Fabric
- Introduction
- Understand pipelines
- Use the Copy Data activity
- Use pipeline templates
- Run and monitor pipelines
- Exercise – Ingest data with a pipeline
- Module assessment
- Summary
Organize a Fabric lakehouse using medallion architecture design
- Introduction
- Describe medallion architecture
- Plan a medallion architecture in Fabric
- Query and report on data in your Fabric lakehouse
- Secure and govern your medallion lakehouse
- Exercise – Organize your Fabric lakehouse using a medallion architecture
- Module assessment
- Summary
