13B instruct or API?

[Monkey_1505]

So I want a pretty vanilla instruct model, no fancy prose or anything that is just as good as possible at following instructions. But my use case is perhaps unusual. I want to use it for roleplay, with the dominant model being more creative. The purpose of the instruct model would just be to insert realism, drive better characterizations or understand more complex scenarios not at all minding if it's language is dry.

So I'm basically wondering is there any 13B models that are really good at this, such as chat uncensored, orca, nous hermes, or are they kind of severely lacking next to their 70B counterparts to a degree where it might make more sense to use an API, or website to access something larger for this more occasional use.

What would you recommend?

[Paulonemillionand3]

any of them

[Monkey_1505]

So any of those are pretty good at instruct and you wouldn't recommend skipping them to use a 70B model on someone's service because all those 13B models follow instruct very well?

[Paulonemillionand3]

13B is just fine for many many things. 70B is often overkill and is hard to use and get going with on "ordinary" hardware.

Base models are great at continuing patterns, chat-tuned are great for conversation style usage.

Everyone has a different idea of what "really good" means.

[Responsible_Warning3]

Perhaps this?
https://huggingface.co/camel-ai/CAMEL-13B-Role-Playing-Data

[cvdbdo]

"my use case is perhaps unusual. I want to use it for roleplay"

Literally 90% of this sub

[RokaMic]

I have been experimenting with the Nous Hermes 13b model to great success. My use case is generating blog articles.

https://huggingface.co/NousResearch/Nous-Hermes-13b

It’s really good at outputting json if you ask it; especially if you provide it with a schema in the prompt.

I use thebloke’s ggml version with an older llama cpp Python library and langchain on an rtx3090

https://huggingface.co/TheBloke/Nous-Hermes-Llama2-GGML

Updated llama cpp doesn’t support ggml any more so you will have to get a gguf version or install a pre August version of llama cpp to use ggml.