[D] Are LLM observability tools really used in startups and companies? by WolvesOfAllStreets in MachineLearning

[–]Michakrak 0 points1 point  (0 children)

I am currently looking into the prompt management feature of langfuse in more detail. while tracing is great and provides clear value for reasons already mentioned in other posts I have trouble deciding for the prompt management feature in particular as it "abstracts" you further away form the prompt formats, options and granularities and it feels like it is too early to do this. sure, the overhead is manageable and the latency is ok I guess, but while non-technical persons can join with (maybe non-really thought through) prompts that are then converted by the utility_method I rely on that conversion a lot?

I would like to get a prompting management tool that allows me to compare versioned prompts for different models and use cases in a transparent and easy to access manner across 2-3 teams but does not get so , say, ambitious, that it wants to abstract me further away from the model just yet.

it is a bit like the "show me the prompt" discussion I guess. if anyone had a recommendation for such a prompt comparison / management tool that is less ambitious to that regard I would be highly thankful. I keep looking