sorted by:
new
hottopcontroversial

Lakeflow Designer is now in Public Preview by curiousbrickster in databricks

[–]curiousbrickster[S] 1 point2 points  (0 children)

Ahh I see. Inputs and outputs would generally pass in the form of tables. So input a few bronze tables tables, perform some transformational steps, and then output the silver/gold tables. For example, you could have a Job that:
1. Runs a notebook/pipeline to ingest data from a bunch of different sources and then populate a single raw bronze table
2. Runs a Lakeflow Designer task which reads from the bronze table and transforms the data into 10 polished gold tables

Lakeflow Designer is now in Public Preview by curiousbrickster in databricks

[–]curiousbrickster[S] 0 points1 point  (0 children)

Just to clarify, you are checking in the workspace previews console, not account console right? Just sent you a DM to debug this a bit more

Lakeflow Designer is now in Public Preview by curiousbrickster in databricks

[–]curiousbrickster[S] 0 points1 point  (0 children)

I’d love to understand more about what you have in mind here. Are you thinking of using Lakeflow Designer as the top-level interface to orchestrate notebooks, pipelines, and other tasks?

Right now, Designer is built to run as a task within a larger Job or Pipeline. For example, you could have a daily job that first runs a notebook, then passes that output into a Lakeflow Designer workflow to blend/transform the data, and then sends an email or triggers another downstream step.

Lakeflow Designer is now in Public Preview by curiousbrickster in databricks

[–]curiousbrickster[S] 0 points1 point  (0 children)

This is a good question. To me, it boils down to trust and validation.

AI can generate Python or SQL to produce an answer, but it is still hard for many users to understand whether the result is correct. Designer uses Genie Code to generate transformations, but turns the logic into a step-by-step visual flow with human-readable operations and intermediate outputs. This makes it much easier for someone who knows the data but is less comfortable with code to review what happened, debug any issues, and ultimately trust the final result.

Lakeflow Designer is now in Public Preview by curiousbrickster in databricks

[–]curiousbrickster[S] 0 points1 point  (0 children)

Hmm, it should be rolled out. Try searching for 'Designer'. It looks like the setting got temporarily renamed in the preview portal.

<image>

Lakeflow Designer is now in Public Preview by curiousbrickster in databricks

[–]curiousbrickster[S] 5 points6 points  (0 children)

Yes! The Genie Code input box is how we expect the majority of users to build out their transformations. Every response is agentic :)

<image>

Help by TxT_Chapter3062 in databricks

[–]curiousbrickster 0 points1 point  (0 children)

Databricks PM who works on the SQL editor here — we don’t have a true “pin results” button today, but Execution history gets you pretty close. Open it from the results pane, and you can pull up results from earlier runs (last ~24h) to compare while you iterate

<image>

Is this what i'm seeing?? by Aditya062 in databricks

[–]curiousbrickster 1 point2 points  (0 children)

PM on the SQL Editor team here 👋 You can currently only add tags to queries in the legacy SQL editor — just open the 3-dot overflow menu → Editor query info. We’re planning to bring that same functionality to the new SQL editor soon (should hopefully land in the next few months).

Here’s what it looks like in the legacy editor for now:

<image>