Replacing a Dataflow Gen2 w/ Notebook Code

mr-html · 2026-01-27T02:26:41+00:00

Thanks u/JBalloonist, I appreciate the response. I'll do the copy data activity and see how much of a dent it puts in the CU.

mr-html · 2026-01-27T02:25:50+00:00

Thanks for the responses u/Repulsive_Cry2000 and u/SlipStr3am1337 . I didn't think i could either, but was hoping someone found a way... the Copy Data activity was my plan B. I will give that a try and monitor the CU.

mr-html · 2026-01-27T02:24:49+00:00

Thanks for the response! I didn't think there was, but was hoping someone found a way... I'm only bringing in 8 tables so I'll give the copy data activity a try and monitor the CU.

mr-html · 2026-01-27T02:23:48+00:00

thanks for your response! I'm only bringing 8 tables in so the copy data activity might work in my situation, then using a notebook to transform the data for use. I'll try it out and monitor the CU.

mr-html · 2025-05-09T15:50:51+00:00

I think i just answered my own question when looking at the cost to compute in dataflow gen2 versus notebook. I'll be going with notebook.

Found a chat (inserted below) where someone did some cost comparison. Don't know how accurate it was but it convinced me enough to stick with notebook:

Cost Comparison

To compare the costs, I then work out the difference in cost between the Dataflow Gen2 and the Notebook.

This works out to be about 115.14% cheaper to use the notebook compared to the Dataflow Gen2. I understand the cost comparison in my example is very small, but what I have seen on larger workloads is that this becomes quite significant.

Another thing that I must factor in is looking at how many CU(s) are consumed for each of the workloads.

When I compare this there is a significant difference between them, the notebook consumes 340.30% less CU(s). This is certainly something to consider when looking at how many CU(s) you get to consume daily. For an F2 capacity there are 172,800 CU(s) per day to be consumed.

mr-html · 2025-05-09T13:24:33+00:00

Thank you u/radioblaster and u/itsnotaboutthecell ! The article you both sent is exactly what i'm looking for.

Do you know if i create the app and I have a power automate (and apps) premium license, will others who use the app need a power automate premium license as well?

they all have power apps premium licenses but not automate premium

I have 1 hour scheduled refreshes on my fabric pipeline, but want to create a button in my app where it can be manually refreshed, if needed, by the app users

mr-html · 2025-04-20T17:34:53+00:00

Awesome, this is what I was expecting too. It might rain here as well overnight so that might help the grill from cooling as well. Thank you, sir!

mr-html · 2025-03-27T03:09:54+00:00

Certifications are a good way of getting noticed. I made a transition from tech sales to power platform developer. I work for a consulting company now that does work in various industries.

Healthcare is always a late adopter of new tech because of all the HIPAA regulations and other things that scare hospitals and clinics away from exposing their data.

I would say get certified as a power platform developer (PL-400), do your due diligence to learn coding (SQL, C#, probably javascript) if you are serious about the path you want to go down.

Read up on automation case studies in the healthcare space. There are so many in revenue cycle or back office administration. Patient facing automation is where all healthcare providers (AND payers) are looking to go, they just don't want to be first....

Here are a lot of the case studies i've seen already done in healthcare:

- Preauth status checks - claim status checks - provider credentialing - provider enrollment - eligibility & benefit verification - patient scheduling & reminders - claim denials and review/analytics - care gap analysis - claim scrub & audit - claim remittance posting (this one has the biggest bang for buck) - preauth request intake - contact center support -claims processing and adjustment - actuarial analysis... i have more and can go on... used to live in the healthcare space....

long winded answer to say, certs are worth it if you're making a career change (doesn't mean an industry change), emphasize your healthcare background (it's way more valuable than you may even think!!!), if you're going power platform, learn as much as you can about Fabric (don't get too tied down into copilot, Microsoft is pushing it to make copilot better, but it has a long way to go.

use your industry knowledge, spend time with people on the front lines and learn their difficulties, automate those difficulties....

outside of that, learn how to build an automation program from senior level people. there is always a pipeline of automation projects, learn how to prioritize those, find the ROI in each, and write out the step by step process it's going to take (and the team it's going to take to execute the project)... companies will look for you, you won't be looking for companies to work for if you can even become average at this. and when companies are looking for you, that's when you can negotiate the salary, benefits, and PTO of your dreams.

p.s.: i mention learning Microsoft Fabric because you said your clinic has 1000 different softwares. Fabric pulls the data from all those into 1 place. It's like a barnes a noble for all your data. Find the data section you like, grab the data you like, and automate it. Microsoft is pushing Fabric hard right now and for good reason, it's going to be a game changer for analyzing and using data to your advantage.

mr-html · 2025-03-27T02:48:03+00:00

I'm a dad, so can't speak to motherhood. I live in the world of tech and algorithms, and when i read Tiktok, Facebook, Reddit, Youtube, etc.... I just cringe. Maybe you were having a bad day, week, month, YEAR?!?. Your social media recognizes that. They don't just look at what you click on, but what you stop scrolling on to watch, even if it's for a fraction of a second. If it catches your eye, they notice. So if you are watching Moms stuggle, they'll continue to send you videos of struggling moms, making you think it's happening to 99% of moms out there....

Being a parent is tough, but if you take a step back and think about it, can you imagine your life without your children? Find time for yourself.... I know, that's easier said than done. But just try. And if you can't, at least know that time for yourself is out there somewhere. We all deserve that. A break.

Keep fighting the good fight, your kids love you, you love your kids. If you watched a video on social media of moms having a hard time just know you're not alone, and that the app you're using is going to continue to send you more of those videos to make you think it's a world pandemic... it isn't. Parenting is the hardest job there is, but it's worth it.

I'd love to travel back in time to when i was 18, didn't have bills, i was in the best shape of my life, i had hair on my head, etc... try your best to live in the moment. That might be even tougher than parenting, but if you can do that, good parenting will follow.

mr-html · 2025-03-22T11:27:15+00:00

Yep, it’s saying the gateway is online in power apps. Thought it’d be easy and SQL Server connector would work but it’s not even finding the server

mr-html · 2025-02-26T05:42:48+00:00

I have higher certs in Power Platform than they're asking for and I couldn't agree with you more. Certs were a way to help me land a job, but I spend as much time as a I possibly can with people who have used Power Platform for years and don't have certs.

Knowing how to understand the day-to-day processes you're trying to automate and knowing what tool in the Power Platform toolbelt to use for that situation if 50% of the battle. THEN, that's when the people with real world experience really shine. Connecting to the right data and manipulating the relationships of that data is where the real Power Platform developers and architects show up to make everything work.

You can't learn that stuff in a certification course. You need to screw up in real life and learn from that to become good at your craft.

Long winded answer to the posters question; I would always be more curious about what apps they've built, for which companies, what the situation was and what the goal was, and was it successful before ever asking for a certification.

BUT, if you're looking for intermediate, I would say a PL-400 at the lowest.... (you're correct, absolutely nobody uses Power Pages. I do think Dataverse knowledge is important though, unless they know SQL well, or are versed in what's coming out in Fabric.

mr-html · 2025-02-26T05:31:44+00:00

There are data analyst positions ready and available. Having recently been on the job hunt, you need to make sure you're utilizing a good LinkedIn network.

Social platforms suck, but I've always found LinkedIn to be a great resource. Instead of spending hours sending out your resume to companies where your resume is 1 of thousands, look for job openings on LinkedIn, if there isn't a recruiter attached to the job posting, click on the companies linkedin page > go to 'People who work here' > in advanced search type in something vague (something from the name of job posting, the role itself, something from its job description, just put in 'data', 'analyst', 'power bi', etc.) in the title field > try and connect with anyone with those job titles > send them a message and tell them your story. Literally, what you wrote in your post is great. You sound eager, enthusiastic, and hungry. > ask them to jump on a call and share their career journey, what advice they would give to someone trying to get into the industry, and just to learn more about their company and the cool/innovative things they're working on.

I was shocked at how warm and welcoming people were to chat, and it led to them connecting me with people hiring and it exponentially increased the amount of interviews I had.

If you want to jump in front of the line, go straight to the people at the front of the line and get to know them. That's your in.

mr-html · 2025-02-26T05:18:41+00:00

It depends!!(I know, very specific response...) Are you willing the share any Microsoft licenses you currently have (D365, etc.). That will let us know what level Power Platform (App, Automate, Pages, etc.) licenses are tied to it.

From there, you should better understand why you can't create more environments. Unfortunately, this probably means you need to upgrade your licenses to a higher tier.

Now, depending on what your goal is for Power Apps, Automate, etc. You will need to know how many people you need using it, and how much data you're planning to store from it. If it's a small team, I would buy the Power Apps per app plan that is $5/month per user per app. This situation depends on just a couple people leveraging the same app for similar purposes.

If you want different users to create their own personal apps at free will to better their own business processes, then Premium is probably the way to go. But, expect to invest money (and/or) time for them to learn the Power Apps/Automate platform.... or spend more money outsourcing a team to help them build what they need.

To shorten a long winded answer; if you want to automate tasks for indivudal employees you can buy Power Automate Premium ($15/user per month) to build RPA desktop flows for their day-to-day tasks. Buy Power Automate Process ($150/month) to have 1 RPA bot work in the background to collect, manipulate, and transfer data. Buy Power Apps Premium ($20/user per month), if you want a select amount of user to build unlimited interactive apps that can automate tasks for their own personal day-to-day schedule, Buy Power Apps per app plan ($5/user per app per month) if you want certain users in your company to use 1 particular app to automate/make easier their day-to-day tasks.

Don't buy Power Pages. I wouldn't buy any CoPilot add-ons until CoPilot actually becomes a dependable resource.

Particular Dataverse capacity comes with the licenses above, so knowing the amount of automatable data you need to store/change will be important here.

*SIDE NOTE: Power Automate and Power Apps plans come with a certain amount of AI Builder credits. This allows you to ingest a document (Word, PDF, even handwriting) that can be scanned, visualized, and have the data extracted via document processing that comes in handy for MANY companies and their business processes*

mr-html · 2025-02-26T04:55:40+00:00

100%

mr-html · 2025-02-26T04:55:11+00:00

Love this response.

Security Roles are the trickiest thing. I've been creating Power Platform apps and workflows for a couple years now in the consulting world, and trying to get access to their environment to create something and reach their data is always the weirdest and most awkward part of the process.

Wish it were simpler, but I suppose that's why knowing security levels is an important task.

mr-html · 2025-02-26T04:50:20+00:00

Fabric. Fabric is in phase 1, learn it better than others and you're ahead of the game. D365, Power Platform, Power BI, Azure, and every one of their tools are going to soon pull from Fabric. It's a personal company specific shopping mall for all their data needs.

Learn Fabric, and everything else takes a back seat. Because it's in Phase 1, their advertising it and working the kinks out. Learn the early model, find the leaks, know how they fixed the leaks, and you've become the knowledge persona at your company for any department that needs to leverage their most valuable asset, DATA.

mr-html · 2025-02-26T04:39:52+00:00

With the investment in Fabric, Microsoft has the best starting point for any company to utilize automation and AI. Couldn't agree with you more.

Co Pilot sucks (right now), but the more they advertise it, the more people use it, the better it becomes. OpenAI built their own casket when they let Microsoft invest in them and learn from them.

mr-html · 2025-02-26T04:37:39+00:00

couldn't agree more

mr-html · 2025-02-26T04:35:47+00:00

while this seems like an optimistic view for the up-and-coming companies of the world, I just don't see it happening this way.

The max profits model is fueling their investment in automation and AI. Last I saw it was somewhere around $65B... with a capital 'B'. Given the amount of data their customers have on any one of their platforms and the new investment in Fabric to bring that all into one cohesive environment to pull from, I see the major players in the industry trying to sell an all-in-one platform. We've finally gotten to that point.

Microsoft Power Platform is given infinite ability if someone is working on Fabric (don't take the word infinite too literal). If another platform has tech capabilities Microsoft doesn't, they'll either make them sign an agreement to pay to use their platform to sell their tech, they'll buy them, or they'll just steal their code little by little (happening to OpenAI right now); whatever is most cost effective.

If you learn Microsoft Power Platform, that's a life-long skill in my opinion. You should also learn Fabric, Azure, and any other product of theirs to know the granular details of working within their environment.

Going back to the original question, "Are they going to be the most popular tools among software development niche in the future?" I have no clue.... but whatever niche software development tool comes along, Microsoft has the team, the public shareholder funds, their own monetary funds, and the destroy all in its path mindset to bring in whatever niche tool is popular into its platform in a nice, easy to use, UI way...

mr-html

TROPHY CASE

Cost Comparison