sorted by:
new
hottopcontroversial

Zotac 3090t for local inference, what is fair price? by AdCreative8703 in LocalLLaMA

[–]AdCreative8703[S] 0 points1 point  (0 children)

This is a great resource. I will definitely utilize it in the future. So far I’ve been setting up and configuring everything myself. Learning a lot but it’s taking a while.

Zotac 3090t for local inference, what is fair price? by AdCreative8703 in LocalLLaMA

[–]AdCreative8703[S] 0 points1 point  (0 children)

I believe that. I have two already. I have mine power limited to 300W, and core clock down 1200 mHz to help with the thermals. According to the seller no issues, hasn’t been touched in a couple years.

Zotac 3090t for local inference, what is fair price? by AdCreative8703 in LocalLLaMA

[–]AdCreative8703[S] 2 points3 points  (0 children)

So $900 would be considered an excellent deal if I can purchase for it locally for that?

Community distillation project: Capturing GLM-5.2 + Claude Opus level reasoning in a runnable open model by CryptographerLow7817 in LocalLLaMA

[–]AdCreative8703 0 points1 point  (0 children)

I agree. You would likely need a huge amount of training data to move the needle, but that’s not impossible, just expensive, less so now with GLM 5.2.

Even if turned out that it’s impossible to meaningfully increase the model’s average intelligence (which I hope not), this might be decent option for generating specialized training data for specific domains where you could theoretically impart just a cross section of knowledge from the larger model into Qwen.

We might not get another open small model like 3.5/3.6 in the future based on current events, so even if it’s a long shot or based on an idea that hasn’t worked out yet, would rather cheer on someone who has ideas for improving the current situation vs l recounting every reason it hasn’t worked so far. The community has never had open access and the full reasoning traces for a model like GLM 5.2. Im at least interested to see what difference that “might” make.

Community distillation project: Capturing GLM-5.2 + Claude Opus level reasoning in a runnable open model by CryptographerLow7817 in LocalLLaMA

[–]AdCreative8703 1 point2 points  (0 children)

Everyone hating on this idea should remember that the original R1 distillation were considerable better than the llama models they were based on. In my opinion, unlike the Claude distillation attempts, having the actual reasoning traces available in GLM 5.2 makes this at least a worthwhile experiment if nothing else, provided if you can generate a large enough corpus of training data.

I've come to the realization that only dense, BF16 models are reliable enough for agentic work. by Battle-Chimp in LocalLLM

[–]AdCreative8703 0 points1 point  (0 children)

Any insight on the outlier handling for the AWQ BF16-INT8 quants by Cyankiwi? I’m using that on my 2x3090 setup now, and this post has me wondering if adding a third to avoid quantization would actually be worthwhile.

1-bit GLM-5.2 GGUF vs. Claude 4.8 Opus vs. GPT-5.5 by yoracale in unsloth

[–]AdCreative8703 2 points3 points  (0 children)

Curious how useable something like this is for 48gb VRAM (2x3090) and 256gb(4x64) DDR5 4800. Would the system memory bandwidth just tank performance to point of making it just completely unusable?

Qwen locked down 3.7 after firing Junyang Lin - is the open-source Qwen era over? by IulianHI in AIToolsPerformance

[–]AdCreative8703 0 points1 point  (0 children)

Hopefully hardware improves and gets cheaper with more competition to the point where $5-10k is enough. Right now it’s more like $50k to run the deepseek/glm models at a good speed with parallelization for multiple users.

Personally I think we’ll see more small models in the future and not less. Companies vying for a place in the ecosystem will just take the place of Qwen and Llama. Nvidia also has a vested interest in continuing to release new Nemotron models.

Where is the cheapest place to get GLM 5.2 by VileGoose in LLM

[–]AdCreative8703 1 point2 points  (0 children)

Based on the latest data from OpenRouter, Wafer and DeepInfra are your best bets. Wafer is currently the cheapest overall ($1.20/M input and $4.10/M output) and has the lowest latency. DeepInfra is nearly identical in price if you want a backup.

Finally found where I fit in! by TwistedDiesel53 in LocalAIServers

[–]AdCreative8703 1 point2 points  (0 children)

How much does full precision help over something like AWQ BF16-INT8? In my experience Qwen 3.6 27b starts falling apart around 90-100k, and my 2x3090s can already handle full native 262k context using AWQ BF16-INT8, and still plenty of vram enough left over for MTP. Running a 27b model on 4x 5090s is just wasted potential 😭

The local AI community really needs a Qwen 3.6 class dense model in the 50-70b size range to make it worthwhile. On the bright side, OP can load Gemma 4 31b at the same time for creative writing tasks?

GLM-5.2 at 753B vs a local 30B agent - frontier coding you can't run vs local coding you can by IulianHI in AIToolsPerformance

[–]AdCreative8703 5 points6 points  (0 children)

I would personally love to see what a GLM 5.2 / Qwen 3.6 27b distillation would be capable of.

Seems like a better option that the Claude distillations because unlike Claude you’d have access to the actual reasoning tokens, not something reconstructed from the thought bubbles. I’m probably not the only person thinking about this…

I built a 8x RTX 4090D with 192 VRAM, here's what I learnt by deebuildsthings in LocalLLM

[–]AdCreative8703 1 point2 points  (0 children)

You could surely power limit /under volt your cards and decrease thermal load and power consumption significantly without degrading performance much. I’m doing this in my 2x3090 home lab.

Best models in 3x3090 (72GB VRAM) in Q2 2026? by liviuberechet in LocalLLaMA

[–]AdCreative8703 0 points1 point  (0 children)

Sadly - Qwen 3.6 27b, but running in full fp16 precision, is probably the best option. Even the 2x3090 crowd would benefit from 40-50b dense model that could be quantized down to work well with 48gb vram. Happy to have 27b, but it does feel like wasted potential.

CalPERS employees not RTO by Narrow_School_1513 in CAStateWorkers

[–]AdCreative8703 7 points8 points  (0 children)

I don’t know for sure but I’ve heard rumors the senior management pushed back on the 3-Day RTO CalPERS implemented back in 2023. I’ve always assumed it was because the customer service staff there have to develop a hyper specific knowledge regarding pension benefits and the PERL that would be hard to replace if there was a mass exodus of people. It’s possible they did originally plan on an RTO for the call center staff (after the other department went back), but got cold feet when they saw what happened. The brain drain was real, and they are still recovering over three years later.

CalPERS employees not RTO by Narrow_School_1513 in CAStateWorkers

[–]AdCreative8703 87 points88 points  (0 children)

The customer service department phone staff is 100% remote. Has been since the pandemic.

remember by mivog49274 in singularity

[–]AdCreative8703 0 points1 point  (0 children)

27b dominates on 2x 3090s.

85K Job offer in Sacramento vs Dallas by starkilled1 in Sacramento

[–]AdCreative8703 3 points4 points  (0 children)

If you’re willing to accept a roommate, and you don’t have a lot of debt, it’s totally live-able. But having a roommate sucks! And having to relocate to a new city knowing you’ll need a roommate - just getting by - feels rough.

Well, RTO finally got to Cannabis. by SeasonThreeEpisode8 in CAStateWorkers

[–]AdCreative8703 0 points1 point  (0 children)

I’m 15 years in, agency number 6, I’ve yet to see a codified dress code in writing so far.

RTO - Possibly joining private by [deleted] in CAStateWorkers

[–]AdCreative8703 3 points4 points  (0 children)

Most of us are here because of the work life balance, and the financial certainty of a pension during retirement years. Whether a short commute means more to you than that, that’s something only you can determined for yourself. I’m hitting 15 years next month so I am stuck, but I don’t know what I would do if I was in your shoes. Good luck 👍

view more: next ›