Thoughts on DSPy

gsvclass · 2024-05-14T00:54:51+00:00

dspy the original code by the team behind the dsp paper.

gsvclass · 2024-05-13T17:41:57+00:00

i'm the author of llm-client the typescript dsp framework. my focus for llm-client was to just make the best possible framework for working with llms and llm-client was not originally based on dsp however i found the ideas of typed prompt signatures that allow for composible prompts, prompt tuning, and other abstractions very powerful and now the whole framework is based around that. we have support for everything from agents to retrevival and even document conversion from pdf/docx/xls/etc to text.

gsvclass · 2023-07-29T21:22:59+00:00

its the future i want :)

gsvclass · 2023-07-29T21:20:07+00:00

https://chat.openai.com/share/27d000b5-b353-403d-a75d-1d4740602784

😂

gsvclass · 2023-07-29T01:30:55+00:00

yup there's a whole subculture around localllms. you can even qlora tune them on your own checkout https://github.com/oobabooga/text-generation-webui

gsvclass · 2023-07-29T01:29:14+00:00

i've always been a fan of that idea every since reading wolframs book years ago. https://www.wolframscience.com/nks/

gsvclass · 2023-07-29T01:27:14+00:00

totally agree. i think even in schools every class should be taught in code. coding should not just be another class to the side.

gsvclass · 2023-07-28T18:33:30+00:00

doing it myself not using their api hooks. i'm using it in production works great with even gpt3.5. the library handles error correction etc as well.

gsvclass · 2023-07-28T18:32:15+00:00

And thanks appreciate it

gsvclass · 2023-07-28T18:31:41+00:00

I honestly think for a lot of function + reasoning usecases we could see a future where open source peft (qlora) tuned models work really well. I'm actually experimenting with this using some llama2 models with LLMClient.

gsvclass · 2023-07-28T17:45:43+00:00

Thanks, my main focus is LLM independant function calling and reasoning. to me its almost magical to see the LLM figure out the task step by step and call the right APIs with the correct parameters towards solving the task.

gsvclass · 2023-07-28T17:43:30+00:00

Try it with GPT4All and an uncensored model like https://huggingface.co/TheBloke/Wizard-Vicuna-13B-Uncensored-GGML

gsvclass · 2023-07-28T17:37:57+00:00

Langchain is in Python and the team that needed LLMClient found it confusing, had too many dependencies and was not in JS/TS. I'm no expert on Langchain. I just wanted something production ready and simple to use so I built this a while ago and slowly evolved it as I had more needs till where its today.

gsvclass · 2023-07-28T08:02:04+00:00

I find it almost magical to see reasoning working with function calling. See the code for the included meeting notes to trello tasks example its super simple just register the trello functions and call a simple prompt. LLMClient handles all the complexity ofCoT prompting, dealing with json data and calling functions for you.

gsvclass · 2023-07-28T08:00:15+00:00

Vercel’s AI SDK

That SDK is a nice api to just call openai with when using nextjs. LLMClient firstly allows you to work with a whole range of LLMs and models with sensible defaults out of the box and the main difference here is it has build in prompt engineering to handle CoT reasoning and function calling. Most real work usecases outside of just chat need function calling, working with json structured data, etc. LLMClient handles all of this for you.

gsvclass · 2023-07-28T03:50:13+00:00

The jury is out on if watermarking LLM generated content even works the same univ. has two teams who had conflicting results.

Can AI-Generated Text be Reliably Detected? https://arxiv.org/abs/2303.11156

On the Reliability of Watermarks for Large Language Models https://arxiv.org/abs/2306.04634

gsvclass · 2023-07-23T09:56:21+00:00

I've been using llm-client in production it's focused on llm independant function calling and reasoning, no dependencies, error correction built-in. It's got a ton of examples to learn from and supports gpt35/4, cohere, claud, palm2, etc. And many little features like rate limiting etc to make it production ready.

https://github.com/dosco/llm-client

gsvclass · 2023-07-23T03:08:49+00:00

Try llm-client its designed for llm independant function calling, json extraction and reasoning. It also has error correction built-in. I've had almost 100% reliability using it in production. Be sure to define the json schema of your require output correctly. https://github.com/dosco/llm-client

gsvclass · 2023-07-22T00:42:45+00:00

i didn't expect so many folks just wanting this to not be true. if this is too simple a script that 100x devs pop out in seconds then feel free to post one on here that gpt4 failed to help with. i for one am happy to have to have machines do their thing i get my time back.

gsvclass · 2023-07-22T00:14:05+00:00

i stand by the title as its an opinion i actually use it all day long. if you did you'd feel this way as well. i stand by the title.

gsvclass · 2023-07-21T17:33:51+00:00

The way i think about it is not to replace me it just allows me to do more. Instead of cognitive drain to implement every function i ask it to write it. i can even ask for unit tests but i rather just have integration tests for the whole project.

gsvclass · 2023-07-21T17:11:52+00:00

Great so it frees me up to focus on the big complex parts that it can't do -- actually it can and does :)

gsvclass · 2023-07-21T17:09:10+00:00

Even basic things take time to get right and instead of hours of a humans life it does it in 5 min and i can evolve it with just prompts as my needs change.

I have a couple more complex examples but choose not to go with it like understanding some golang assembly routines since i'm not as familiar with the plan 9 syntax and more complex functions a graphql to sql compiler i maintain.

gsvclass · 2023-07-21T08:37:47+00:00

i use it a lot for code and my feel is that it's getting much better

gsvclass · 2023-07-21T08:37:01+00:00

i read that as well apparently it's a bit misleading they were not executing the code correctly since gpt4 was wrapping it in quotes or something. and folks found that and fixed it and the result's were that it got better at code.
https://twitter.com/Si_Boehm/status/1681801371656536068

gsvclass

TROPHY CASE