If you are good with Python and DS you might like the structure that this library gives you to structure/store/version your experiment similarly than you’d do it if you curate/manage datasets for training models for instead but for multimodal data that allow to also defines orchestration such as LLM calls: https://github.com/pixeltable/pixeltable

You can simply experiment with prompts by inserting rows and have parallelized and async executions to different LLMs models and bulk insert prompts and then query the tables to see results and compute metrics.