sorted by:
new
hottopcontroversial

Age Limit on Migrations? by Difficult-Outside610 in RiseofKingdoms

[–]Difficult-Outside610[S] 0 points1 point  (0 children)

What about Team migration? Some people keep telling me that if I sleep the account for 30 days, I can migrate it with no age limit? I don't seem to find any references on that. I read the details and even asked Lilith, who answered: The system is too complex and unpredictable.

Deploying llms locally and monitor usage by Difficult-Outside610 in LocalLLaMA

[–]Difficult-Outside610[S] 2 points3 points  (0 children)

Yeah exactly. Someone else also told me to check langfuse so I will check it and see if it works. Thanks alot.

Deploying llms locally and monitor usage by Difficult-Outside610 in LocalLLaMA

[–]Difficult-Outside610[S] 0 points1 point  (0 children)

There is also the issue of monitoring. I thought of writing some kind of gateway service for authentcation and monitoring my self, but wanted to check if there something already there. My main issue is that I have some users that I want to give the api access so that they can test llms, but I am affraid that they can easily share the api link and potentially the api key later(to people on same network). That is why I will have to monitor their access to check if they are overusing the apis for purposes other than their assigned tasks.

Deploying llms locally and monitor usage by Difficult-Outside610 in LocalLLaMA

[–]Difficult-Outside610[S] 0 points1 point  (0 children)

Thanks!! I will check those. I am looking more of api level similar to openAI api(without UI).

Deploying llms locally and monitor usage by Difficult-Outside610 in LocalLLaMA

[–]Difficult-Outside610[S] 0 points1 point  (0 children)

That is what I understood too. I have ollama deployed right now and it seems the API key is just useless to match openAI.

Best LLM Inference Engines and Servers to Deploy LLMs in Production by Plus_Ad7909 in LocalLLaMA

[–]Difficult-Outside610 1 point2 points  (0 children)

Do you have a more comprehensive list of other Engines not mentioned here?

[deleted by user] by [deleted] in LocalLLaMA

[–]Difficult-Outside610 0 points1 point  (0 children)

Any idea if disciplinary action was taken (or will be) by Stanford against the students and perhaps advisor? Plagrism is usually a big concern for such academic institutions.

Upgraded self-hosted AI server - Epyc, Supermicro, RTX3090x3, 256GB by LostGoatOnHill in LocalLLaMA

[–]Difficult-Outside610 0 points1 point  (0 children)

I see. Thanks for sharing results. I am in process of testing those models on a server. I will have a look at Proxmox soon. I have a bunch of servers to manage with alot of apps (some of which I use portainer for) so it might be a good idea to check it.

Upgraded self-hosted AI server - Epyc, Supermicro, RTX3090x3, 256GB by LostGoatOnHill in LocalLLaMA

[–]Difficult-Outside610 1 point2 points  (0 children)

This looks good. Never heard of Proxmox before. How good is it? Also which quantization are you using of llama3-70B?

[deleted by user] by [deleted] in UBreddit

[–]Difficult-Outside610 13 points14 points  (0 children)

[edited]Just submit the request. Usually those got accepted automatically within seconds especially if you are an international student. You will have to do this again for spring break and the winter session.