Lesson 27 – Using Notebooks with Microsoft Fabric

Microsoft Fabric’s notebooks offer a web-based interactive environment for coding, experimenting, and visualizing data. Supporting various languages such as Python, Scala, SQL, and R, these notebooks enable the development of Apache Spark jobs and machine learning solutions. They also provide connectivity to lakehouses, combining data lakes and data warehouses.

This blog guides you through the process of creating, importing, exporting, saving, and connecting notebooks to lakehouses.

Key features of Fabric Notebooks

• Interactive Coding – Write code, run queries, and visualize data seamlessly.
• Language Diversity – Support for Python, Scala, SQL, R, and HTML, catering to diverse tasks.
• Machine Learning Integration – Leverage Apache Spark for creating machine learning models.
• Collaborative Environment – Collaborate with others, add comments, and track changes using version control.
• User-Friendly Features – The notebooks provide IDE-style features such as IntelliSense, code snippets, and magic commands, enhancing the overall coding experience.
• Flexibility Modes – Switch between code-first and low-code modes based on preferences.
• Data Compatibility – Excel in managing different data types and tasks, offering robust support for a wide range of data sources and formats.
• Copilot Integration – Copilot is context-aware of the workspace, attached lakehouse schemas/tables/files, and notebook structure; it supports multi-step code generation, refactoring, and validation, plus performance guidance (joins, shuffles) and a “Fix with Copilot” option that diagnoses and proposes approval-based fixes for failed cells.

How to create notebooks?

You can either create new notebooks or import an existing notebook.

Create a new notebook

• Navigate to workspace you want to create notebook. Click New item–> Notebook.

Import existing notebook

• Go to My workspace homepage.
• Select and import existing notebooks from your local computer.
• Fabric notebooks recognize standard Jupyter Notebook (.ipynb) files and source files like .py, .scala, and .sql.
• Effortlessly create new notebook items based on the imported files.

How to export a notebook?

Notebooks can be exported in various standard formats, including. ipynb (Jupyter notebooks), .html, .py, and .tex.

How to save a notebook?

• Notebooks in Fabric automatically save by default when opened or edited.
• Manual save options are available, and users can create copies in the current or another workspace.

Save a copy feature

Utilize “Save as” to duplicate the notebook either within the current workspace or to another workspace.

Switch to manual save

• Click on “Edit” in the menu.
• Navigate to “Auto Save” and select “Off”.
• A prompt will appear for confirmation. Click “Confirm” to activate the manual save option.

• Save changes manually with a click on “Save” or by pressing CTRL+S.

Connect lakehouses and notebooks

• Fabric notebooks support interactions with lakehouses for data storage.

Note:  You cannot remove the existing default lakehouse by clicking the X without setting another lakehouse as default. You can set a lakehouse as default by pinning it.

• Click “Add Lakehouse” to add new or existing lakehouse.
• Choosing “Remove all lakehouse” will eliminate all lakehouses added to this notebook.

Notebook resources

• Notebook resources are files that you can store and access in a Fabric notebook, such as code modules, data files, images, and semantic models.
• Facilitates reuse of common functions.
• Enhances notebooks with dynamic visualizations and interactivity.
• Create, upload, download, and delete resources effortlessly through the notebook resource explorer.
• Integration into notebook cells using simple commands like %run for code, %fs for data, and %md for images.
• Seamless work with notebook resources in Visual Studio Code using the Fabric extension for editing, running, and debugging.
• Built-in resources folder a system-defined folder unique to each notebook (max 500 MB, single file up to 100 MB, up to 100 items). Supports create/delete, upload/download, drag/drop, rename, duplicate, search, and a one-click “Write to lakehouse” option.
• Environment resources folder a shared Resources folder inside an attached environment, letting multiple notebooks reuse the same files via a second root folder in the notebook, mounted at the relative path /env.
• File editor view and edit CSV, TXT, HTML, YML, PY, SQL, and similar files directly in the notebook’s resource folder (1 MB size limit; .xlsx and .parquet aren’t supported), with manual save via Ctrl+S.
• Environment library publishing now has two modes Quick mode (~5 second publish, installs at session start) and Full mode (3–6 minute publish, stable reproducible snapshot; pairs with a custom live pool for ~5-second starts).

• Fabric notebooks support collaborative editing with multiple users simultaneously.
• You can see their profile, run output, cursor, selection, and editing trace.
• This feature facilitates seamless pair programming, remote debugging, and tutoring scenarios.

Sharing and Managing Permissions

Sharing a notebook simplifies collaboration with team members, granting default viewing or editing/run permissions for authorized workspace roles. You have the flexibility to share a notebook with specific permissions tailored to your preferences.

• Click “Share” option in the toolbar to share the notebooks.

• You are allowed to choose the relevant category and grant specific permissions as needed.

• Sharing options include sending the notebook directly or copying a shareable link.

• Permissions can be managed through the Workspace item list.

Commenting and Switching Modes

Fabric introduces a valuable feature for collaboration – cell-level comments. This commenting feature encourages teamwork, allowing discussions and feedback within the notebook.

• Click the Comments button on the notebook toolbar or the cell comment indicator to access the Comments pane.
• Select code, choose “New” in the Comments pane, add comments, and then save by selecting “Post comment” or by pressing ctrl + Enter.
• Manage your comments effortlessly with options like Edit, Resolve, and Delete available under the More menu.

Fabric notebooks now support four modes, each tied to a permission level:

• Develop mode — requires read, execute, and write permissions.
• Run only mode — requires read and execute permissions.
• Edit mode — requires read and write permissions (cannot run).
• View mode — requires read permissions only.

Develop — Make changes, run, and comment. (requires read, execute, and write permissions)

Run only — Run cells, but make no changes. (requires read and execute permissions)

Edit — Make changes and comment, but run no cells. (requires read and write permissions)

View — View only. (requires read permissions only)

Version History (Preview)

Notebooks now support version history with manual checkpoints (“Create new version”) and automatic system checkpoints every 5 minutes, plus a diff view to compare a checkpoint against the live version and options to Restore or Save as copy. Versions originating from Git syncs, deployment pipelines, or VS Code publishing are labeled by source. System checkpoints expire after 1 year.