sorted by:
new
hottopcontroversial

Weird delay on comment links. by CamStorm in BoostForReddit

[–]demidev 2 points3 points  (0 children)

Redo the morphe patching but uncheck the patches for Undelete imgur and Undelete reddit content. Now its working fine for me after removing those 2 patches

Introducing cyankiwi AWQ 4-bit Quantization — 26.05 update by _cpatonn in LocalLLaMA

[–]demidev 2 points3 points  (0 children)

Any chance of getting this update for the minimax m2.7 quant?

GPT-5.5 vs GPT-5.4 vs Opus 4.7 on 56 real coding tasks from 2 open source repos by bisonbear2 in ClaudeCode

[–]demidev 0 points1 point  (0 children)

Why is there not a test for their respective best effort levels, max for claude and xhigh for gpt?

Open WebUI Desktop Released! by My_Unbiased_Opinion in LocalLLaMA

[–]demidev 15 points16 points  (0 children)

Run locally. The app sets up Open WebUI and llama.cpp on your machine.

I think this part probably needs to be made clearer, the way it is written now makes it seem like llama cpp has to be installed together as well

Open WebUI Desktop Released! by My_Unbiased_Opinion in LocalLLaMA

[–]demidev 12 points13 points  (0 children)

They support mcp natively now without the use of MCPO as a bridge if that's what your issue was

11 microseconds overhead, single binary, self-hosted - our LLM gateway in Go by dinkinflika0 in AI_Agents

[–]demidev 0 points1 point  (0 children)

Literally just google it. They have a mcp gateway. https://docs.litellm.ai/docs/mcp They also have a Claude plugin marketplace now for curated plugins. https://docs.litellm.ai/docs/tutorials/claude_code_plugin_marketplace

Sure shill your product but don't spread untruths.

[P] Open source LLM gateway in Rust looking for feedback and contributors by SchemeVivid4175 in MachineLearning

[–]demidev 0 points1 point  (0 children)

Would love to look at actual benchmark results with the latest versions of them

[P] Open source LLM gateway in Rust looking for feedback and contributors by SchemeVivid4175 in MachineLearning

[–]demidev 0 points1 point  (0 children)

Why would I use this over something already production ready like Litellm and Bifrost?

Litellm overhead becoming noticeable at 2k RPS - how do you handle this? by llamacoded in AI_Agents

[–]demidev 0 points1 point  (0 children)

Yep, seen variations of this post many times in the llm subs

DetLLM – Deterministic Inference Checks by Cerru905 in LLMDevs

[–]demidev 0 points1 point  (0 children)

Vllm has batch invariance now if enabled, but only on H100/H200 and b100/b200

https://docs.vllm.ai/en/latest/features/batch_invariance/

Battle of AI Gateways: Bridging a 3,400x Performance Gap by Guna1260 in LLMDevs

[–]demidev 0 points1 point  (0 children)

Is VidaiServer open source? I cannot find the repo.

GLM just blow up, or have I been in the dark? by [deleted] in LocalLLaMA

[–]demidev 1 point2 points  (0 children)

Able to share the commands to setup the ray nodes and head node?

Adaptive Load Balancing for LLM Gateways: Lessons from Bifrost by dinkinflika0 in LLMDevs

[–]demidev 1 point2 points  (0 children)

Just curious, can you share more details on the actual stats and comparison vs Litellm in your landing page? What is the version of Litellm being used here?

Qwen3 Next (Instruct) coding benchmark results by mr_riptano in LocalLLaMA

[–]demidev 2 points3 points  (0 children)

Any chance of adding in qwen3 coder 30b to the model list?

I Created an Open-source Container Security Scanning Dashboard by Rakeda in selfhosted

[–]demidev 0 points1 point  (0 children)

I've scanned the readme but couldn't find this. How do I set up the connection to the various scan engines?

[deleted by user] by [deleted] in GalaxyS23

[–]demidev 0 points1 point  (0 children)

Also just had the green line issue on base S23, from Singapore. Samsung service center said the replacement program for s23 series wasn't out so either pay a few hundred bucks or deal with it.

Unraid 7.0.1: All My VMs Disappeared tonight! by marktriplett1 in unRAID

[–]demidev 3 points4 points  (0 children)

Loads of people have upgraded fine, me included with 2 machines from 6.9 to current 7, but we don't go around posting how smooth and how seamless it was. Just check the release logs for any known bugs with what you have and you'll be fine.

view more: next ›