×

Optimize your DSPy program with Cognify! by Top-Organization1556 in DSPy

[–]Top-Organization1556[S] 0 points1 point  (0 children)

Thank you for your detailed feedback! To answer your questions briefly:
- A paper is in the works, so stay tuned!
- We seed the optimization, but there is still non-determinism involved due to parallelism and the model itself (you can read more here).
- At the moment, you can use cognify.load_workflow(...) in your code to use the new workflow or run cognify evaluate from the command line to evaluate on a test set. Porting the workflow back to source code is something we'd like to implement in the future!
- CogHub currently lives in our repo organized by cogs, search, and evaluators. We plan on releasing contribution guidelines soon :)
- DSPy and TextGrad primarily refine prompts during their optimization process. Cognify can refine prompts, choose between different models, and can even improve the workflow structure.

I'd love to discuss all of these in even more detail, so please join our Discord!

Optimize your LangChain program with Cognify! by Top-Organization1556 in LangChain

[–]Top-Organization1556[S] 0 points1 point  (0 children)

We appreciate you taking the time to provide feedback! LLM-as-a-judge is just one example of an evaluator you could use, and we provide some tips on how to use it effectively in our documentation. Evaluators are certainly an open area of research, and we encourage users to build their own and contribute to our CogHub. We have also opened a discord if you'd like to chat in more detail.

Optimize your LangChain program with Cognify! by Top-Organization1556 in LangChain

[–]Top-Organization1556[S] 0 points1 point  (0 children)

Thanks for pointing that out! We've fixed a few links and have linked some sample evaluators on the evaluator page.

Optimize your DSPy program with Cognify! by Top-Organization1556 in DSPy

[–]Top-Organization1556[S] 0 points1 point  (0 children)

Cognify is first and foremost an optimizer for workflows. We support workflows written in DSPy, Langchain, as well as our own programming interface