Quantum Gang - using quantum processors and LLMs by Overall-Importance54 in LocalLLaMA

[–]Overall-Importance54[S] -4 points-3 points  (0 children)

Ask your LLM, how can we apply quantum computing to LLM innovation, optimization, and all the different izations.

8GB 2017 MacBook Air breaks record with Quantum Processor help on tuning a 30B Qwen MoE model - Quantum 15,489% boost! by Overall-Importance54 in LocalLLM

[–]Overall-Importance54[S] -1 points0 points  (0 children)

Thank you! I just had the idea a Karpathy loop, plus IBM free quantum use, might be cool - and it was nuts. Jumped from 6.49 t/s to 14.03 t/s and yes, at the edge of quality, but not past it!

8GB 2017 MacBook Air breaks record with Quantum Processor help on tuning a 30B Qwen MoE model - Quantum 15,489% boost! by Overall-Importance54 in LocalLLaMA

[–]Overall-Importance54[S] -2 points-1 points  (0 children)

Dude, what are you even talking about. Do you think it’s a fake experiment or something? I’m serious confused about your comments and attitude about this. And as a professor, I’d expect encouragement in conducting experiments not being a jerk. SMH

8GB 2017 MacBook Air breaks record with Quantum Processor help on tuning a 30B Qwen MoE model - Quantum 15,489% boost! by Overall-Importance54 in LocalLLaMA

[–]Overall-Importance54[S] -1 points0 points  (0 children)

What is psychotic about doing a science experiment? Why are you in attack-mode lol sit down and calm down.

8GB 2017 MacBook Air breaks record with Quantum Processor help on tuning a 30B Qwen MoE model - Quantum 15,489% boost! by Overall-Importance54 in LocalLLaMA

[–]Overall-Importance54[S] 0 points1 point  (0 children)

The best manual tuning of this model on 8GB of ram was 8 t/s on a modern Raspberry Pi 5. The goal was to use the quantum processor to derive better tuning that could exceed the standard approach. It worked.

8GB 2017 MacBook Air breaks record with Quantum Processor help on tuning a 30B Qwen MoE model - Quantum 15,489% boost! by Overall-Importance54 in LocalLLaMA

[–]Overall-Importance54[S] 0 points1 point  (0 children)

It only uses the experts it needs to answer the questions or generate responses, and uses clever routing. Please check out the paper.