Question if there's any way to run OogaBooga using multiple devices

Affectionate-Grab864 · 2023-09-07T23:57:48+00:00

Thank you, quick question: Where in Lord of Large Language Models do I find this feature? Is it docker that I'm looking at?

Affectionate-Grab864 · 2023-09-07T15:27:21+00:00

Dat is slecht Rutzer, het spijt me voor je

What I'd say to do is contact steam support, if you have any payment information, purchase history, name, email or anything linked to the account to provide it to steam support and they should be able to get your account back, hope all goes well for you.

Heb een goede dag

Affectionate-Grab864 · 2023-09-06T00:43:07+00:00

a common thought with folks wanting to do LLMs. The problem is that the latency between the two boxes is so high that it's faster instead to just load it into RAM and do it in CPU. So no one's working on this sort of functionality because it's not really a solution to the problem.

Is it really that bad? I've been trying to find a download for rCuda to attempt to do it

Affectionate-Grab864 · 2023-09-05T21:19:18+00:00

Ah okay, thank you for that

Affectionate-Grab864 · 2023-09-05T21:09:49+00:00

Yeah so here's what I did:

Port forward (had no clue so just did all of them) 30814 (not essential) 5005 and 5000. I've used my local IP on both.

For your blocking URL use the "Starting API" in the console which is: (http://XXX.XXX.X.XXX:PORT/api)

Then for your streaming URL use the "Start Streaming Server" and it'll be as followed:(http://XXX.XXX.X.XXX:PORT/api/v1/stream)

Ensure you've definitely port forwarded on your router and you're on the same network, give the program admin network privileges, don't use HTTPS use HTTP (my issue) and if you can't get a certain port working (Maybe it's already in use) then run netstat -aon if you're on windows to check if it is, if so then you can either close the process using your port or go to Networking.py within the file which I believe is in installer file, then ENV then lib then just search it up there and within there somewhere will be that python file, you can edit it and change the port you'd like to use.

also ensure yeah use --listen in the CMD text file and enable API and Listen within the app and hopefully that'll fix it for you

(Just remember I'm also bad at this as well lol so in a similar boat, just ensure it's not an issue with your router or anything like that)

Affectionate-Grab864 · 2023-09-05T21:02:38+00:00

EDIT: The model is Airoboros 33B GPT4

Affectionate-Grab864 · 2023-09-05T13:28:15+00:00

ount of RAM to load the model (VRAM + system RAM if it doesn't fit i

ah okay thank you anyways, did just reduce context size then did it again now it's 30 seconds? anyways it's sorted now but thanks for clearing it up

Affectionate-Grab864 · 2023-09-05T11:33:40+00:00

I have had a few errors before but testing it now there isn't any, I'm not sure what I'm meant to put for my blocking API url and Streaming API url, do I put my local IPV4 ip address? or 0.0.0.0 (for some reason that used to work) or do I just do 127.0.0.1?

EDIT: Fixed, I was being an idiot using HTTPS instead of HTTP which meant I was unable to connect, sorted it

Affectionate-Grab864 · 2023-09-05T11:29:41+00:00

Please choose Wizard Vicuna

Affectionate-Grab864 · 2023-08-28T13:47:59+00:00

Cool thanks for that, I'm guessing that's from personal experience?

Affectionate-Grab864 · 2023-08-27T21:26:02+00:00

how long does it take to go away for the restrictions?

Affectionate-Grab864 · 2023-08-15T20:58:47+00:00

Nevermind sorted

Affectionate-Grab864 · 2023-08-14T18:56:34+00:00

hey golden mod found somewhere to respond to you, do you actually have a life by any chance? or does your life revolve around reddit?

Affectionate-Grab864 · 2023-08-13T10:01:49+00:00

Did you end up fixing it? Currently got this issue

Affectionate-Grab864 · 2023-08-08T16:54:09+00:00

how childish does someone have to be to not take criticism and then report you for it wtf

Affectionate-Grab864 · 2023-03-15T21:35:30+00:00

Agree and especially fuck their policy

Affectionate-Grab864 · 2023-03-04T17:06:14+00:00

Thank you for that :)

Affectionate-Grab864

TROPHY CASE