DP-700 Which one when · Module 6

Dataflow Gen2, Pipeline or Notebook? One Job Done 3 Ways

Three Microsoft Fabric tools to bring data in and shape it, one small job done three ways, plus query folding, staging and the append trap.

8 November 2026 · 5 min read

Which tool for which job: pipeline, Dataflow Gen2, notebook — Watch the lesson: Module 06 Theory: Dataflow Gen2, and which tool for which job | Microsoft Fabric DP-700

Fabric gives you three tools to bring data in and shape it: a pipeline, a Dataflow Gen2 and a notebook. The exam likes to describe a situation and ask which one fits. In Module 6 of the DP-700 course we do one small job three ways, a table of public holidays, to feel the difference.

Pipeline moves and orchestrates; Dataflow Gen2 transforms visually; notebook transforms in code — The short answer.

Tool	Is for	Typical job
Pipeline	moving data and orchestrating: copy at scale, loops, conditions, retries; it runs the other two	copy forty monthly files and record each one
Dataflow Gen2	transforming visually with Power Query, hundreds of connectors, small to medium data	an analyst maintains a small mapping table
Notebook	transforming in code, any logic, any volume, PySpark and Spark SQL, tests and version control	de-duplicate and merge millions of rows

When two tools could do the job, ask one question: who will maintain it? Choose the tool that person can read.

The same job, three ways

Dataflow Gen2. Power Query in the browser. Paste the 22 holidays, use the first row as headers, and add a custom column, is_long_weekend, with one formula. Every click becomes a step.

Power Query: holidays table with an added custom column is_long_weekend and the applied steps — No code, apart from one formula, and the steps are visible to anyone.

Then a data destination: the silver lakehouse, a new table, with update method Replace for a list maintained by hand.

Destination settings: update method Replace and column mapping — The update method is set per destination.

Notebook. The same table in eight lines of PySpark, which would treat a billion rows exactly the same way.

Notebook cell creating the holidays table with an is_long_weekend column — Full control in code.

Pipeline. A Copy activity can land the two columns of the file in a table, but look for a place to compute is_long_weekend: Mapping connects and converts columns; it cannot compute.

Copy activity destination and mapping tabs in a pipeline — A pipeline moves data; to transform rows it calls a dataflow or a notebook.

Two ideas behind dataflow speed

Query folding: against a database, Power Query translates your steps into one query and the source does the work. Some steps cannot fold; from that step on, the dataflow engine does the work, which is slower. Put filters early, where they still fold.
Staging: a query first lands its result in a hidden staging lakehouse and later steps run on that copy with a faster engine. It helps large transformations; for a small query, like ours, Fabric switches it off.

The trap

The update method is set on each destination. If a dataflow reloads everything and its destination says append, the table doubles on every refresh, and nothing reports an error.

An exam-style question

A business analyst who knows Power Query must load a small reference table from a public web page into a lakehouse every week, with a few clean-up steps. Which tool fits best? Dataflow Gen2.

Do it yourself

Lab guides, practice and full mock exams come with a subscription. See the plans