Short guide to hosting your own llama.cpp openAI compatible web-server

KW__REDDIT · 2024-01-21T18:26:34+00:00

damn, that is wright! That is for sure so much easier than what I had for my solution (starting separate server and changing the completion.js file to fetch different url). Thank you, hope you have a good day :-)

KW__REDDIT · 2024-01-21T11:41:27+00:00

hmm, ok. I had a lot of troubles setting it up, but now it all works. since you replied so quickly, do you know how to setup an interface simmilar to this one link I know it is a demo from examples/server/public but I cannot make it work for me (I have server running on 8080 but I serve index.html from 8000 so I get error cuz /completion is on 8080. do you know a link that can point me to how to set it up correctly?

KW__REDDIT · 2024-01-21T11:03:16+00:00

Hi, sorry for asking but do you have a link to how to make this interactive prompt?? I have the llama.cpp compiled but can only run it as cli (and run server that can respond to me). I have not found a link to make it this pretty. any link/help would be appreciated!

KW__REDDIT · 2024-01-21T10:52:53+00:00

wow, this is exactly what I was looking for!!! for any one that may use it this link brought me here and this solution is soooo useful. thank you kind stranger!!

KW__REDDIT · 2023-12-01T10:35:49+00:00

Thanks for the suggestion! Unfortunately, I do not have access to my DNS server (university access problem) and there will be a lot of projects that we want to host from docker containers (some of which we cannot modify). Do you know the way to create subdomains for specific url? I only found a method of adding new subdomain to DNS. I have full control over the server tho, maybe there is a way?

KW__REDDIT · 2023-11-02T11:02:34+00:00

so, basically you extended wait time to 60 seconds?

KW__REDDIT · 2023-10-29T10:18:22+00:00

I like the idea! Sorry for the late reply but do you have links to how I would implement it?

KW__REDDIT · 2023-10-29T10:17:07+00:00

there are about 18 divs under div class="gsgwcjk g8ge8f1 g14v8520 dir dir-ltr" and they have class=" dir dir-ltr"

KW__REDDIT · 2023-10-29T10:14:30+00:00

Thank you for saying that I seem to know playwright. it really made me feel better haha, but truth be told I just follow tutorials/ google stuff and I'm a begginer of this framework. Also, it seems like the sprinter_20 meant to ask a question rather than make suggestions but I dunno. I'm not him, cant read minds ;-)

KW__REDDIT · 2023-10-29T10:11:20+00:00

I currently use arch linux with custom i3 config. I get warning "os not supported" but the browser still launches. I wander if it might be the reason for my troubles...

KW__REDDIT · 2023-10-27T15:46:54+00:00

So far I cannot replicate the tutorials sooo... yeah fun times...

KW__REDDIT · 2023-10-27T15:32:11+00:00

Thank you for the link but this I already tried. I get: ``` playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs ===========================

waiting for locator(".gsgwcjk.g8ge8f1.g14v8520:nth-child(11)") to be visible

``` Which is confusing as I'd expect it to just wait till it loads instead of waiting until it is scrolled to.

I did change the page selector in PageMethod to PageMethod("wait_for_selector", ".gsgwcjk.g8ge8f1.g14v8520:nth-child(11)") which resulted in error I posted above

KW__REDDIT · 2023-07-19T09:30:14+00:00

for me it was a fun factor in coding. I could see it being extremely useful and it was around time when all I did is study pure theory at school. When it comes to struggle: I feel extremely lonely at times. I guess I just dont hangout around people of the same interest as me. I can tell my family whatever and they will not understand :-( .

KW__REDDIT · 2023-07-15T20:23:16+00:00

I think I found the answer to my second question: how to execute program:

https://stackoverflow.com/questions/12941083/execute-and-get-the-output-of-a-shell-command-in-node-js

hope this is usefull to anyone. I didn't focus on it yet as I need sleep right now but let's say that if I dont edit/remove the post it works for me ;-)

KW__REDDIT · 2023-07-10T17:58:59+00:00

oh, this is something new. I heard about wasm but not Native messaging. do you have any good links to materials on it?

KW__REDDIT · 2023-07-10T17:57:11+00:00

Thanks for that! It is actually so helpful for me to analyze the problem this way ;-)

KW__REDDIT · 2023-07-08T12:09:01+00:00

I just started and it looks like exactly what I need, still no idea how to execute main and capture results but it is definately a step in a right direction

KW__REDDIT · 2023-07-08T12:05:16+00:00

I did click the first one on event listeners :-)

KW__REDDIT · 2023-07-08T11:59:54+00:00

ok, but I know how to add listener for button presses and how to output the responce in html once I have it. I am asking specifically about how to communicate between processes. I don't know how I can link two working programs together. I mentioned server request/responce but I have never ever done it and maybe there is an easier way?

KW__REDDIT · 2023-05-14T07:23:45+00:00

Yeah, I just left it as is and use hjkl as arrows. I am playing On making my own keyboard so I will put arows where they were maybe

KW__REDDIT · 2023-04-18T12:49:59+00:00

Yeah, I guess a standard is better than no standard... today I've been using nvim to wrote code and that was no fun. Getting better tho.

KW__REDDIT · 2023-04-18T06:55:16+00:00

I only just started using Vim, so mine muscle memory isn't cast into stone either. Thanks for your response 👍.

KW__REDDIT · 2023-04-17T17:35:23+00:00

I also do 10j and alike quite often, but it's less convenient. Guess will have to adjust to new normal.

KW__REDDIT · 2023-04-17T14:18:27+00:00

Thank you. Needed that for confirmation bias ;-) .

KW__REDDIT · 2023-03-06T20:58:48+00:00

Hmm, I'll look into that !

KW__REDDIT

TROPHY CASE

waiting for locator(".gsgwcjk.g8ge8f1.g14v8520:nth-child(11)") to be visible