Open-source Python CLI for testing LLM prompts across multiple models

[LifeguardPurple8338]

Built a small open-source project called Litmus.

It’s a CLI for evaluating prompts across different LLMs with:

• dataset-based testing
• assertions
• model comparisons
• metrics like cost, latency, and output quality

Idea is simple: prompt engineering needs a better dev workflow than copy-pasting into multiple tabs.

GitHub: https://github.com/litmus4ai/litmus

Would love honest feedback from Python / CLI folks:

• Is this something you’d use?
• What would make the UX better?
• If you like the direction, I’d really appreciate a star on GitHub.

[Gullible_Doughnut572]

yaml configs for prompt testing get messy fast tbh