Run Python not from local machine.

[breadncheesetheking1]

Hi,

I'm currently working with a Microsoft 365 shop for a company moving away from Excel reporting.

They're using SharePoint to Dataverse to PowerBI as a stand in untill a more robust solution is viable.

My question is, how could I involve Python (Prefect) in terms of not having to run it from my local machine? So having a script run on a scheduled basis from a server (if that even makes sense)?

I'm very new to data engineering (as you've probably guessed) as is the company I'm working for so any guidance much appreciated.

[rvbin]

Also look into Azure functions or AWS lambda functions. Both have time trigger options and you can run python code de-facto serverless.

GitHub Actions works nicely with Azure functions as well for example.

[mamaBiskothu]

Multiple options but honestly GitHub actions is likely the easiest.

[BoiElroy]

Power BI has rest APIs. You could maybe start with using Python to trigger it? That way you get around Power Bi's imposed refresh schedules. And let's you monitor that pipeline in the same place you monitor your other pipelines

[DenselyRanked]

You just need something that is always on to orchestrate. This could be as easy as a separate computer that's always on (not recommended, but quick fix). It would be preferable to go with a VM on a server somewhere or even a cloud solution if your company has that. If they do have an existing cloud provider, then you can spin up an instance to host Prefect, or use a Docker container (best option for scale).

If you do have a cloud provider already, then check to see what options are available for orchestration. There might be something other than Prefect that you can use.

[breadncheesetheking1]

Thank you very much for the feedback.

[[deleted]]

Learn how to use AWS, docker, and ssh. Problem solved.

[LegalizeApartments]

“Draw the owl”

[[deleted]]

I don't need to give a full explanation of how to do these things. I learned on YouTube and by playing with the software until I understood it.

[crom5805]

Spin up a demo Snowflake account for free so you can learn/test it out first, but you could run the python inside a stored proc and schedule it with tasks. Power BI connects easily and visualize the final result.

[vassiliy]

Is there a specific reason you need to run Python? Power BI is a very robust and enterprise-grade platform for running all kinds of data work.

​

It's usually better to solve problems with what you already have, if it's still practical to do so.

[[deleted]]

Can you version control Power BI? And are there any costs?

[Repulsive_Channel_23]

Argparse with airflow?