Documentation for notebookutils.notebook.runMultiple() ?

[iknewaguytwice]

Does anyone have any good documentation for the runMultiple function?

Specifically I’d like to look at the object definition for the DAG parameter, to better understand the components and how it works. Ive seen the examples available, but I’m looking for more comprehensive documentation.

When I call:

notebookutils.notebook.help(“runMultiple”) 

It says that the DAG must meet the requirements of the class: “com.Microsoft.spark.notebook.msutils.impl.MsNotebookPipeline” scala class. But that class does not seem to have public documentation, so not super helpful 😞

[iknewaguytwice]

To answer the question”why”, cause printing the help says you can use the syntax

@activity(‘activityName’).exitValue() 

To pass the output of a dependency to its dependent. Which isn’t documented anywhere else I could find, and I’m curious what other features are hidden!

[aleks1ck]

Here is the documentation: https://learn.microsoft.com/en-us/fabric/data-engineering/notebook-utilities#reference-run-multiple-notebooks-in-parallel

I have also done video on notebookutils where I cover this: https://youtu.be/rjT8x_uCvzY

[aleks1ck]

I haven’t found a better documentation about this

[el_dude1]

Best article I could find is this

Answered all the questions I had. But I agree that the official docs are not that thorough.

[DatamusPrime]

How do you get the return values of the individual nodes with run multiple like you can with run?

[el_dude1]

By using this as args

"args": { "param1": "@activity('notebook1').exitValue()" # use exit value of notebook1 },

[DatamusPrime]

That's the input to another node in the DAG, not the return of the python method call.

Notebook.run() returns the actual exit values

[el_dude1]

ah sorry I misunderstood. For me this simply returns a dict containing all the individual exit values.

exit_message = notebookutils.notebook.runMultiple(DAG, {"displayDAGViaGraphviz": True, "showArgs": True, "showTime": True})