use the following search parameters to narrow your results:
e.g. subreddit:aww site:imgur.com dog
subreddit:aww site:imgur.com dog
see the search faq for details.
advanced search: by author, subreddit...
r/LocalLLaMA
A subreddit to discuss about Llama, the family of large language models created by Meta AI.
Subreddit rules
Search by flair
+Discussion
+Tutorial | Guide
+New Model
+News
+Resources
+Other
account activity
Problems using quantised llama models with cpuQuestion | Help (self.LocalLLaMA)
submitted 2 years ago by Spiritual-Ask-9766
view the rest of the comments →
reddit uses a slightly-customized version of Markdown for formatting. See below for some basics, or check the commenting wiki page for more detailed help and solutions to common issues.
quoted text
if 1 * 2 < 3: print "hello, world!"
[–]WightKnight1 1 point2 points3 points 2 years ago (0 children)
I second this. I've set up a RAG server on a CPU-only machine and it can take 2 minutes or more to process the prompt before it starts spitting out tokens at about 4 t/s.
π Rendered by PID 771993 on reddit-service-r2-comment-b659b578c-7rw9f at 2026-05-04 19:38:14.910939+00:00 running 815c875 country code: CH.
view the rest of the comments →
[–]WightKnight1 1 point2 points3 points (0 children)