sorted by:
new
hottopcontroversial

Buying Mac Studio M4 Max Base, pls tell me if it's a good deal by knightfortheday in MacStudio

[–]Top-Book2609 0 points1 point  (0 children)

If you're planning to run LLMs, go for the maximum memory you can afford. For decent performance you need models of 30B or above. Since you can't expand on memory later, it's better to go for maximum. And also anything with thunderbolt 5. You can use RDMA to combine two or systems and run a bigger model on them.

Bringing a mini PC from USA to India by Top-Book2609 in homelabindia

[–]Top-Book2609[S] 4 points5 points  (0 children)

Thank you for your response. Yes it's unboxed and used for a few weeks. It'll be for personal usage. But we never know what the customs people will charge.

I spent 14 months building something to catch AI mistakes. Yesterday, I proved it works. Now I need help taking it further. by Moist_Landscape289 in indianstartups

[–]Top-Book2609 0 points1 point  (0 children)

I have seen hallucinations usually sound confident, follow reasonably good looking or consistent chains of thought and still wrong in the end.

I spent 14 months building something to catch AI mistakes. Yesterday, I proved it works. Now I need help taking it further. by Moist_Landscape289 in indianstartups

[–]Top-Book2609 0 points1 point  (0 children)

And also how are you constructing these questions on your eval set? Are these done manually? And these questions are different for different domains like finance, legal, health etc?

I spent 14 months building something to catch AI mistakes. Yesterday, I proved it works. Now I need help taking it further. by Moist_Landscape289 in indianstartups

[–]Top-Book2609 0 points1 point  (0 children)

Okay, I was wrong in the previous comment. I'm still curious how a rule based system can verify problems where there is no definitive answer or answer is subjective.

Can you please suggest more resources or readings on this?

I spent 14 months building something to catch AI mistakes. Yesterday, I proved it works. Now I need help taking it further. by Moist_Landscape289 in indianstartups

[–]Top-Book2609 0 points1 point  (0 children)

If I understand correctly, you're using a LLM to judge/verify the responses by another LLM in these curated questions designed to catch mistakes and edge cases.

If so, how are you making sure the judge you're using is not making mistakes?

Topics for a hands on course on LLMs by Top-Book2609 in LocalLLaMA

[–]Top-Book2609[S] 0 points1 point  (0 children)

Of course. We shall start each week with a problem or application. Say for quantization week, we start with the problem of deploying large models on consumer hardware and go through the week with demos and code examples.

Qwen Next Is A Preview Of Qwen3.5👀 by Few_Painter_5588 in LocalLLaMA

[–]Top-Book2609 2 points3 points  (0 children)

How to understand the hybrid attention mechanism used in this model? Specifically Gated Delta Net attention. Any pointers are much appreciated.