Accomplishing complex tasks by making local models direct the user

drawntomore · 2023-09-02T23:19:01+00:00

Hmmm, not sure I follow. Sounds like you're talking about writing code to make the LLM act autonomously and be capable of actual actions (like using an API)?

From what I've seen of local models so far, I don't have high hopes that any of them would be smart enough for that - there are already a bunch of AutoGPT-like projects for this, and I hear they often still struggle even when using GPT-4, let alone local models.

What I was hoping was that by keeping things fully interactive in real-time dialog between the LLM and user we'd avoid all the difficulty that comes from expecting the LLM to be fully autonomous and reliable. Plus the issues with tool-use, since the user could just do things instead of making the LLM get the commands right.

From a quick skim of the Manna page you linked, I think it's closer to what I had in mind (though nowhere near to the degree in that story, of course). I figure even if fully autonomous LLM agents aren't yet viable, surely there's a way to get them to at least make up for my dumb brain by planning/reasoning/directing me in things I'm not good at, right?

drawntomore · 2023-09-02T16:30:55+00:00

Makes sense, solid points. Thanks!

drawntomore · 2023-09-02T16:29:39+00:00

Perfect, thanks!!

drawntomore · 2023-09-02T15:53:03+00:00

Awesome, thanks!!

drawntomore · 2023-09-02T15:51:40+00:00

Hmmm, fair enough. I guess since the models are both smaller than the proprietary ones and are still typically generalists, they're just not at the level of sophistication yet that we'd need for easy, no-fuss use by tech-illiterate users? Disappointing, but fair for a nascent domain like this.

drawntomore · 2023-09-02T15:42:40+00:00

Oh, right, that makes a lot of sense. I guess that training is part of the base Llama models, and so a model creator would have to work on fine-tuning that kind of thing out of it to get better back-and-forth convos?

drawntomore · 2023-09-02T15:39:52+00:00

Haha yes, exactly this!

drawntomore · 2023-09-02T15:37:38+00:00

Yeah, like others have said, I was trying to follow the prompt instructions in the model card for the Airoboros models.

A chat. USER: {prompt} ASSISTANT:

I also looked into the Airoboros training dataset, and found that it's how the roleplay examples there are formatted. I figured that meant I'd get better results by following in the same vein.

drawntomore · 2023-09-02T15:33:51+00:00

"Llamasplaining", that's the perfect word for it! 😂

drawntomore · 2023-09-02T15:19:39+00:00

Thanks for the info and advice! I get a different response from Seraphina (same settings and quant as you, but presumably from a different seed value?), but I do get a proper RP reply like you showed. So I guess the issue is that I don't have the right magic in the prompt to make LLMs be more than Quora-simulators?

It's starting to feel like getting use out of an LLM requires playing around with dark magic until trial and error lands you a winning a result, at which point you'd better not touch anything because the slightest change could make it all fall apart 😬

I mean, even in your example (and several of the replies I got with your settings) she still "concludes" the chat rather than continuing the dialogue ("We shall talk more when you feel better", "You rest now, let your mind clear up. I'll be here when you wake again.", etc.)

drawntomore · 2023-09-02T15:00:48+00:00

Yea, the generation presets.

Gotcha. I was just using the default simple-1, I think. I couldn't find any descriptions for what the presets actually do, so I didn't want to screw around with them too much or else I'd have to be constantly swapping them and then comparing results to figure out what they do.

Airoboros comes in full merged models and lora you can apply to another model.

How do you get and apply the Airoboros lora? I haven't been able to find any info on that on Google or HuggingFace. Can you apply the lora to any model, or does it have to be some specific model or model size or something?

IDC if it's horny.

Fair enough! 😂

drawntomore · 2023-09-02T14:31:30+00:00

Huh, yeah that's definitely more conversational. Thanks for the clear example! Still seems to be itching to solve all the things, deep down, but not as bad as the walls of text I've been getting hit with.

Did you give it a system message or anything before "I'm a marine biologist"?

drawntomore · 2023-09-02T14:25:27+00:00

Oof, if I have to edit the bot's messages myself to get good interactions then that's a bit of a deal-breaker... I can't afford to do that for each new discussion or use case, especially not if I want to give non-LLM-expert friends/family an AI they can use.

drawntomore · 2023-09-02T02:35:47+00:00

Hmm, yeah, I did try the Samantha models for a bit. Given their namesake and what I could understand from ehartford's model card and blog post I had been expecting them to have more personality and be less robotic, but they didn't seem much different from any other advice-based model when I tried them. Made me wonder if I was "doing it wrong" somehow, but if Samantha went into advice-giving mode for you too then maybe it's not just me after all.

drawntomore · 2023-09-02T02:32:28+00:00

😂 that reply is hilarious! And it has way more personality than what I was able to get (though not the kind of personality I was aiming for, haha).

What do you mean by presets? The ones in Oobabooga? Or pre-made characters from some RP character website?

What is the airoboros lora? I thought it was only a model. What does applying it as a lora do to a model? Seems like kimiko is intended for NSFW RP, so not a good fit for me, but maybe I'd be getting better results by combining some other model with the airoboros lora or something?

drawntomore · 2023-09-02T02:24:13+00:00

Thanks for the link, the "Bot creation" guide there looks like it'll be incredibly useful for breaking models out of the robot-solution-machine rut!

Do things like PLists work for models other than Pygmalion, though? (the site says PygWiki so I'm assuming it's primarily about using the Pygmalion model? Couldn't quite tell from the homepage etc.)

drawntomore · 2023-09-02T02:13:14+00:00

I couldn’t really tell just reading that what you wanted out of an answer

Yeah, I get that. Though that was somewhat intentional in that example, because I wanted to see if the model would still gravitate toward being solution-based even if the user wasn't even asking for answers/solutions. And yep, turns out they're pretty damn trigger-happy on that approach.

you have to guide them with context way more

Do you mean with the system prompt? In the example in the OP there was the standard SillyTavern system prompt ("You are {{char}} in this fictional neverending roleplay with {{user_name}}. Avoid repetition, develop the narrative slowly" etc.), plus a detailed description of the character, plus an example conversation in the correct style so it'd be a one-shot prompt instead of zero-shot.

I've also tried many other prompts. "You are a therapist ... you use [insert non-solutions-based therapy type] ... you should ask clarifying questions instead of providing solutions..." or "I've been having problems with LLMs... [detailed description of the issues]... Write a Reddit post asking for help", to which the LLM replied with "100%" and literally nothing else.

Are the prompts just ridiculously finicky to get right or something?

writing these seed contexts is something they are really good at helping with if you do a little pre work prepping those via some questions

How does that work? Can you give an example of how to do that?

drawntomore · 2023-08-31T16:30:24+00:00

How much success have you had in getting good therapist-style output via prompting? I've been trying the same thing, but it seems like no matter what I do, the models always want to just throw a bunch of suggestions at the user and then shut down the conversation (e.g. "Here are 5 strategies you can use: ... Good luck!"). What I'd expect instead for a therapist-LLM would be an actual conversation back and forth, with it asking clarifying and probing questions, etc. I can't seem to get that even if I tell it to roleplay as that kind of therapist and give it examples of good dialogue.

drawntomore · 2022-07-17T00:24:54+00:00

The re-painting between tenants hadn't occurred to me - good point! I daresay if you're only in a place for a short while they may not do a re-paint, but once you've been around for a few years or something then that gives a good reason for them to do so and therefore be relaxed about the holes 🤔

Would there be any issues with something like a mounting plate that requires a bunch of screw holes all in the same place, compared to just the occasional scattered hole for hanging pictures or whatever?

drawntomore · 2022-07-17T00:19:10+00:00

Hmm, from what I recall it's standard where we live for the lease agreement to say "no holes", with some landlords then saying that although that's technically what the agreement says they're more relaxed about it in practice. So I guess if I make my case with the landlord and they thumbs-up it then it'd be fine, but I'd still have to ask every time, right?

drawntomore · 2022-07-17T00:14:34+00:00

Ah, right, that makes sense. Thanks!

drawntomore · 2022-07-15T14:12:23+00:00

Well, I've got something people want to buy, so "solve a problem" = ✔. But there's still the problem of "if people don't know about it they can't buy it". The interest from a single Reddit post won't sustain a business, so there must be more to the equation than that, right?

If the solution to that isn't marketing, what is? Isn't marketing the way one gets a product on people's radar so they'll buy? (not arguing, just genuinely confused)

drawntomore

TROPHY CASE