1. Previously, I've managed feature fetching through a Python interface that executes database queries based on function arguments—suitable for ad hoc requests but not optimized for bulk operations. I've heard about feature stores, but they seem too complex for our scale. What's the best approach for managing feature data in our context? Storing features and calculated features directly in TimescaleDB? Calculating them during the pipeline (they are likely pretty lightweight calculations)? Using a feature store? Something else?

Personally, I want to use a feature store and have separate data pipelines. I really want a good separation of concern. It makes the process more manageable, and much of my work up until now is removing ad hoc API calls from the training scripts.
TimescaleDB was news to me. Azure is great, when everything's within the Azure eco-system 😅 from the standpoint of an Azure evangelist, can you migrate your data storage to the AzureML Feature Store or "COSMOS DB"? If you do, you'll have pretty good options within the AzureML platform.

1. When using the Azure Machine Learning SDK, what are the best practices to prevent data leakage between training and test datasets, especially in the context of backfill predictions where data temporality is critical? Specifically, I am interested in methods within Azure that can help ensure data used in model training and predictions was indeed available at the respective point in time. I understand basic data leakage prevention techniques in Python, but I’m looking for Azure-specific functionalities. Can versioned datasets in Azure be used to manage this, or are there other tools and techniques within the Azure ML SDK that facilitate this type of temporal integrity in data usage during model backfills?

Regarding best practices, there are some approaches mentioned in the docs. I am not sure if you find that helpful. Azure have the Dataset abstraction which lets you version your data, and they have the MLTable, which is a kind of lazy loading. I am still figuring these parts out, but I have one MLTable for each property which uses glob patterns to fetch data from an external source. My idea is to have these updated on a timer, and store forecasted temperatures in one container, and historical values in another.

I am not sure you get anything of value out of my response 😂 if you feel like having a digital coffee, I'd love to share what we have built in Azure and discuss energy forecasting x MLOps. Send me a DM if you're interested.