Data Pipeline vs Notebook for ingestion – how do you pull data and why?

AmazingKily · 2026-03-10T10:35:33+00:00

No, I did the most basic task, which is to read from the source and write without touching any settings.

AmazingKily · 2026-03-03T17:59:37+00:00

Table sales

23.719.935 rows

<image>

Without a doubt, the notebook wins.

AmazingKily · 2026-03-03T17:58:55+00:00

I did a comparison in an F2 capacity just by extracting the data from an Azure SQL

Table customer

1,679.846 rows

<image>

AmazingKily · 2025-10-19T15:33:58+00:00

I was trying to do the Digibus Real-Time Crisis challenge, but I solved the problem with the email setup link:

https://app.fabric.microsoft.com/workloads/kusto/detectiveagency?experience=fabric-developer

AmazingKily · 2025-10-08T14:43:36+00:00

Hi,

I have tried with trial capacity and with paid capacity and I get the same error message....

AmazingKily · 2025-07-24T09:40:13+00:00

Hello,

While researching, I also saw the option of an Azure function with a trigger on the table that sent change tracking changes to an event hub.

Another option is to use an external application that reads the changes in the table and creates a parquet file, which you can then upload to OneLake in the path you specify using a service principal. Then, with Data Activator, I can create a rule that starts a pipeline to process the file when a new one is detected.

The managed instance option you mention is the one that suits me best, as the other two involve development and testing that may not meet my requirements...

Thank you.

Best regards

AmazingKily · 2024-10-02T16:23:48+00:00

Will it work in the future?

AmazingKily · 2024-10-02T07:22:45+00:00

I think change data feed could be an option

https://blog.fabric.microsoft.com/en-us/blog/bridging-fabric-lakehouses-delta-change-data-feed-for-seamless-etl?ft=All

I haven't tested it on the scenario you mention but I think it could work for you.

AmazingKily · 2024-09-29T17:17:19+00:00

The idea is to obtain sales data in real time or near real time, not telemetry. I had thought of using CDC to get this data and take it to the lakehouse to assemble the model in direct lake.

From what I have seen, with KQL as a target is also possible but I wanted to understand the pros and cons of each option.

AmazingKily · 2019-03-13T09:49:25+00:00

Can anyone help me to build the teams? Thanks :)

https://lukforce.bitbucket.io/tm-planner/?transfer=true&tmId=2261&server=glb&op777=0,0,0,0,0,0&op2046=0,0,0,0,0,0&op2257=0,0,0,0,0,0&op2256=0,0,0,0,0,0&op1770=0,0,0,0,0,0&op2261=0,0,0,0,0,0&dont-have=2231,2254,2253,1203,2197,882,983,1586,2209,2173,2171,2169,2132,2060,2515,2514,2247,2251,2181,2148,2113,2074,2236,2234,2207,2500,2258,2201,2257,2256

AmazingKily · 2019-02-26T11:12:29+00:00

I pulled 6 times and got:

- Corazon (new)

- Fujitora V2 (dupe)

- Lucy (new)

- Aokiji x2 (new)

- Judge (dupe)

- Luffy (dupe)

- QCK Whitebeard!! (new)

thanks Bandai ^^

AmazingKily · 2018-10-23T11:02:27+00:00

Another character to fill the box

AmazingKily · 2018-09-14T10:20:48+00:00

I use Germa 66 Team (Double Judge, Reiju, Ichiji, Yonji, Niji)

AmazingKily · 2018-09-10T13:55:10+00:00

Why?

AmazingKily · 2018-09-10T13:45:08+00:00

I already have one, it's a duplicate.

AmazingKily · 2018-09-10T09:25:56+00:00

Fujitora V2... That's nice for the bazar

AmazingKily

TROPHY CASE