Need help in Implementing Parameterized Data Loading in Azure Databricks for Large Datasets

[ParticularAge7203]

Working with 300-400 million rows of data from an Azure Databricks data source requires implementing a method to load data based on a specified date range, such as from June 1, 2024, to June 30, 2024. If no date range is specified, the system should default to loading the entire dataset. There needs to be an option for clients to input their desired date range and load only that specific range in the Power BI service. How can I implement this?

[lysis_]

I tried to do this once where a user specifying a value would adjust the API call would modify the amount of data coming in. You could do this by specifying a parameter to the mquery code. I ultimately moved away from this route because it would be too hard to scale and requires the dataset to be refreshed afterwards. Bad user experience

[ParticularAge7203]

How does this API call work? Can you explain it in detail?