Lesson 1 – Introduction to Microsoft Fabric

Microsoft Fabric is an end-to-end, cloud-based SaaS analytics platform where data experts and business teams can work together on data projects in one shared space.
Microsoft Fabric represents an innovative AI-driven analytics platform that consolidates data and services sourced from various outlets like Power BI, Azure Synapse, and Azure Data Factory.
Fabric consolidates various experiences like Data Factory, Real-Time Intelligence, Databases, Fabric IQ , Analytics (Data Engineering, Data warehousing, Data science), Industry solutions and Power BI onto a unified Software as a Service (SaaS) platform.

Source: Microsoft Learn

Why Microsoft Fabric – An umbrella?

Microsoft Fabric is a combination of three existing products Power BI, Azure Data Factory and Azure synapse. Its acts as an umbrella for all data analytics services from Microsoft that integrates various data technologies and tools into a single solution.

Fabric offers a single data repository, or data lake, accessible to data engineers, analysts, scientists, and business users without data duplication. This unified data lake is called OneLake-a single, multicloud data lake built on Azure Data Lake Storage. Integrated within the Fabric service, it establishes a centralized space to store all organizational data, facilitating the operation of various experiences within a unified location.

Components of Microsoft Fabric

Microsoft fabric offers tailored experiences for diverse roles and tasks, encompassing areas like Power BI, Data factory, Data engineering, Data science, Datawarehouse, Real-time Intelligence, Industry solutions ,Fabric IQ and Databases. Moreover, it features a unified data lake, enabling data retention in its original locations while supporting the use of preferred analytics tools.

Source: Microsoft Learn

Power BI

Power BI is a dynamic business intelligence tool that empowers users to gather, analyze, and visualize data from various sources, allowing for informed decision-making through interactive reports and dashboards. This tool is user-friendly, enabling both technical and non-technical users to explore data, discover insights, and share information across an organization.

Data Factory

• Azure Data Factory is a cloud-based data integration service provided by Microsoft Azure. It allows you to create, schedule, and manage data pipelines that move and transform data from various sources. Data Factory enables users to extract, transform, and load (ETL) data for analytics, business intelligence, or data warehousing purposes. This tool helps in orchestrating and automating the movement and transformation of data, both on-premises and in the cloud
• Data Factory is built directly on OneLake, making data integration faster and easier. It allows you to create and automate data pipelines, copy data from different sources, and schedule workflows without complex setup. Fabric Data Factory also supports modern tools like dbt and Apache Airflow and includes AI-powered Copilot to help build pipelines more quickly. These features make it easier to prepare and move data for analytics, reporting, and business intelligence.

Data Engineering

• Data engineering using Spark typically involves leveraging Apache Spark, an open-source distributed computing system, for processing and handling large volumes of data. Spark is known for its speed and versatility, making it a popular choice for data engineers working on big data projects.
• Microsoft Fabric Spark’s fusion with Data Factory allows for scheduling and orchestrating notebooks and Spark jobs.
• Fabric notebooks include automatic retry, allowing jobs to recover from temporary system failures without manual intervention. It also provides a VS Code extension to easily manage workspaces, edit notebooks, and work with environments directly from Visual Studio Code.

Data Science

• The Data Science experience facilitates the smooth construction, deployment, and operationalization of machine learning models within your Fabric environment.
• Data Science integrates with Microsoft Foundry, giving users access to prebuilt AI models and tools for building intelligent solutions. It also supports importing existing machine learning assets from Azure Databricks, Azure Machine Learning, and other MLflow-compatible platforms, making it easy to reuse existing models and workflows.

Data Warehouse

A data warehouse is a central repository for storing and managing large amounts of structured and often historical data from various sources within an organization. It’s designed to facilitate reporting, analysis, and decision-making by consolidating data from different departments and systems.

Real-Time Intelligence

Real-Time Intelligence is a dynamic service that allows users across your organization to gain insights and visualize data as it flows like only with few mins of data availability. It delivers a complete solution for handling event-driven scenarios, streaming data, and log files. Using simple, no-code connectors, it effortlessly integrates time-sensitive data from various sources, delivering real-time visual insights.

Data Activator

• Data Activator in Microsoft Fabric offers no-code solution to automatically trigger action when it notices changes in data. It keeps an eye on data in reports and checks if it reaches certain levels or follows specific patterns. When this happens, it can send alerts or start tasks without needing any complicated code.
• Fabric brings all kinds of different experiences together in one place, creating a complete solution for analysing big data. It helps both companies and people to turn huge and complex data collections into useful tools and insights.

Databases

• The SQL database in Microsoft Fabric is a newly added component in fabric which is developer-friendly, transactional database built on the Azure SQL Database platform, enabling you to easily create operational databases within Fabric. It utilizes the same SQL Database Engine as Azure SQL Database.
• Fabric brings all kinds of different experiences together in one place, creating a complete solution for analysing big data. It helps both companies and people to turn huge and complex data collections into useful tools and insights.

Industry Solutions

Microsoft Fabric provides tailored data solutions for various industries, offering a powerful platform for data management, analytics, and decision-making. These solutions are designed to tackle the unique challenges of each industry, helping businesses optimize operations, integrate data from diverse sources, and leverage advanced analytics.

Fabric IQ

Fabric IQ (preview) is a new workload for unifying business semantics across data, models, and systems. It includes items such as ontology, plan, Fabric Graph, data agent, operations agent, and semantic models, and it powers consistent decisions, reusable metrics, and context-aware automation across the Fabric platform.

Fabric Terminologies

The following are the fabric terminologies need to know before getting into Microsoft Fabric.

• Tenant – A “tenant” in Fabric refers to one unique setup of the platform specifically designed for an organization. It’s closely connected to an Azure Active Directory, which helps manage user access and permissions.
• Workspace – Workspace is a place to create collection of items such as report, data warehouses, lakehouses. It is a single environment that designed to collaborate and provide controls for who can access in it.
• Capacity – A capacity exists within a tenant. Each capacity linked to a particular tenant represents a separate set of resources dedicated to Microsoft Fabric. The capacity’s size dictates the level of computational power accessible.
• Experience – A collection of capabilities targeted to a specific functionality. 
• Item – An “item” comprises a collection of abilities within an experience. Users have the ability to generate, modify, and remove these items. Each type of item offers distinct capabilities.Examples of items include a lakehouse, notebook, warehouse, semantic model, and eventhouse.
• OneLake – OneLake is Fabric’s unified, multicloud data lake and single storage layer for all workloads, built on Azure Data Lake Storage. It lets every workload share the same data without copying or moving it, and now supports storage tiers and lifecycle management to help control long-term storage costs.
• Semantic Model-A semantic model is a metadata layer in Fabric that defines tables, relationships, measures, and data connections. Semantic models are the data source for Power BI reports, dashboards, and other analytics experiences in Fabric.
• Real Time Hub-The centralized location in every Fabric tenant for discovering and managing real-time data streams across your organization.
• Data Stream-A continuously flowing data source that appears automatically in the Real-Time hub, including stream outputs from eventstreams and tables from KQL databases.
• Fabric Event-An event generated by a Fabric or Azure resource such as a workspace item change, a job completion, or an Azure Blob Storage update that you can subscribe to in order to trigger downstream actions like pipeline runs or notifications.
• Ontology-An item, part of Fabric IQ, where you define entity types, relationships, and properties to organize data according to your organization’s business vocabulary.
• Mirroring-The process of continuously copying data from an external source into Fabric to create a live mirrored database or catalog, without a manual ETL process.
• Workload Development Kit (WDK) / Extensibility Toolkit-The WDK lets partners and developers build, validate, and publish custom workloads that extend the Fabric platform. The Extensibility Toolkit is its modern successor, providing an SDK and starter kit for the same purpose.
• Workload Hub-The central place inside Fabric where users discover, add, and manage custom workloads at the tenant, capacity, or workspace level.
• Shortcut-A OneLake feature that lets you mount an existing external storage account (Azure Data Lake Storage, Amazon S3, Google Cloud Storage, or another Fabric location) in place, giving zero-copy access to that data without ETL or migration.
• Governance / Microsoft Purview-Fabric’s centralized administration and data governance layer covering permissions, sensitivity labels, and auditing  is powered by Microsoft Purview, which is built into Fabric and applies automatically across items, including data shared across tenants via OneLake data sharing.