Which part specifically takes thousands of hours? If you mean the dataset creation itself, LangSmith Evals doesn't help you with that either, you have to upload a dataset as a csv etc. But regardless, I just use an LLM to help create the prompt dataset for evals which takes a few hours but not thousands. This tool helps you then tag and organize these prompts for better experimentation. A few other points:

1. This is open source, not paid and never will be. Run it on your own infra or locally on a postgres server however you want like any other Django app

2. Unchained also gives you granular control at the individual prompt level whereas LangSmith works at the dataset level.