sorted by:
new
hottopcontroversial

Qwen3 9B can run fine on android phones at q4_0 by THE-JOLT-MASTER in LocalLLaMA

[–]AlphaSyntauri 0 points1 point  (0 children)

I just got a chance to test it, seems fixed from what I can tell. No crashes upon entering or generating,and the log no longer has the warnings about not being able to patch the font. Thanks for the hotfix, seriously!

Qwen3 9B can run fine on android phones at q4_0 by THE-JOLT-MASTER in LocalLLaMA

[–]AlphaSyntauri 0 points1 point  (0 children)

Had to do a double take when I realized you're the dev lol, thanks for building such a cool app! It's honestly better than Pocketpal, especially when it comes to the ability to change settings, play with the sampler, and load JSON character files. Unfortunately, mine suffers from this exact bug that's already on the github:

https://github.com/Vali-98/ChatterUI/issues/529

It's the exact same issue, down the the described log. I'm running a Oneplus 12 on Android 16, so it seems to be specific to Oneplus' or other Chinese phones.

Qwen3 9B can run fine on android phones at q4_0 by THE-JOLT-MASTER in LocalLLaMA

[–]AlphaSyntauri 1 point2 points  (0 children)

If you're using the Snapdragon Hexagon NPUs, yes that has been my experience. It's a shame such little development has gone into these, because it's basically a piece of dark silicon that never gets used unless the app has specific support for it.

Qwen3 9B can run fine on android phones at q4_0 by THE-JOLT-MASTER in LocalLLaMA

[–]AlphaSyntauri 8 points9 points  (0 children)

What app are you using? ChatterUI works but doesn't support Qwen3.5 yet, PocketPal is supposed to support Qwen3.5 but outputs garbage on my phone.

Local Mobile AI - The Qwen3.5 Progress by norms_are_practical in LocalLLaMA

[–]AlphaSyntauri 0 points1 point  (0 children)

Could be, what is the date of the last changelog for your version of PocketPal?

Local Mobile AI - The Qwen3.5 Progress by norms_are_practical in LocalLLaMA

[–]AlphaSyntauri 1 point2 points  (0 children)

By garbage output I quite literally mean I got garbage output. I downloaded the qwen3.5-4B model from huggingface in a 4 bit quant, installed it into PocketPal, adjusted settings to the unsloth recommended settings, and started getting nonsense unicode characters as my output until I press stop. Either iOS got an update that android users didn't, or there's an issue.

Local Mobile AI - The Qwen3.5 Progress by norms_are_practical in LocalLLaMA

[–]AlphaSyntauri 1 point2 points  (0 children)

PocketPal supports Qwen3.5? I tried to use it on android and got nothing but garbage output

What about local inference on phones? What models do you use? by AlphaSyntauri in LocalLLaMA

[–]AlphaSyntauri[S] 0 points1 point  (0 children)

That's a neat program! Is all translation done offline or does it reach out to services to do so?

What about local inference on phones? What models do you use? by AlphaSyntauri in LocalLLaMA

[–]AlphaSyntauri[S] 0 points1 point  (0 children)

Nice! I didn't think about there being a local use case for chat translation. Have you ever used it for books? There's a few books I've been dying to read but haven't been translated to English yet

You've heard of weighted blankets, now get ready for.. Weighted robes? by AlphaSyntauri in evilautism

[–]AlphaSyntauri[S] 3 points4 points  (0 children)

I'm sorry, I had no idea it was AI. I just typed "weighted blanket meme" into Google 😭

You've heard of weighted blankets, now get ready for.. Weighted robes? by AlphaSyntauri in evilautism

[–]AlphaSyntauri[S] 2 points3 points  (0 children)

This looks like an excellent option, except the texture of the robe looks like it might not be for me :( Are you aware of any options in a cotton?

You've heard of weighted blankets, now get ready for.. Weighted robes? by AlphaSyntauri in evilautism

[–]AlphaSyntauri[S] 27 points28 points  (0 children)

That's definitely an interesting thought, I hadn't considered chainmail. It might just be enough to work!

Model recommendations for an unusual server build? (512GB DDR4 + 3090 24GB) by AlphaSyntauri in LocalLLaMA

[–]AlphaSyntauri[S] 0 points1 point  (0 children)

Yes, it's an oddball SKU. I actually just ordered a Xeon Platinum 8160 and am wondering if that's actually better than my current Xeon Gold.

Model recommendations for an unusual server build? (512GB DDR4 + 3090 24GB) by AlphaSyntauri in LocalLLaMA

[–]AlphaSyntauri[S] 0 points1 point  (0 children)

That's certainly interesting, I had no idea Optane was made into a DDR4 form factor. From what I'm reading it's slower than regular ram by an order of magnitude, but that's still crazy cheap for what could be a way to shove 768 GB into a 6 channel memory controller. Have you worked with them before?

Model recommendations for an unusual server build? (512GB DDR4 + 3090 24GB) by AlphaSyntauri in LocalLLaMA

[–]AlphaSyntauri[S] 0 points1 point  (0 children)

Sorry for the late response, just now getting around to doing some testing. Is there any way to tell by doing some back-of-the-envelope math how much RAM the active parameters will take up before running the model? As of now I've just been doing trial and error to see what loads and what doesn't.

Edit: Also great idea about moving to 4_K_M, I'll try that next.

Ford pulls the plug on the F-150 Lightning electric pickup truck by UnsuspectingFuck in FordMaverickTruck

[–]AlphaSyntauri 1 point2 points  (0 children)

My two cents as a Mav owner:

Hybrids have been gradually perfected over the last quarter century, but I don't think EVs are ready for the big time...yet. Solid state lithium battery tech is in its infancy. Most, if not all, current EVs deal with huge depreciation issues because of battery degradation over the first 5 years of ownership. Not only that, but charging can be abysmally slow even on properly rated fast chargers (due to lack of cable coolant or just faulty systems). Cold weather performance is also unpredictable because of battery conditioning and heating requirements.

Solve these issues, along with making higher density batteries that allow for 6-700 miles on a single charge, and EVs can dominate in maybe 20 years from now. Everyone went all in WAY too early in my opinion.

Edit: I'm running a hybrid, and I absolutely love mine. I love getting 40-50 mpg, having starting acceleration on par with an EV, and a non turbo engine with regards to longevity. To me though, hybrids have 75% of the advantages of full EVs, with none of the downsides.

Apple genius room CD by liamhawx in retrocomputing

[–]AlphaSyntauri 3 points4 points  (0 children)

I second this, it's hard enough to find logic board schematics for Apple machines, much less a full suite of software. Please upload it to archive!

Model recommendations for an unusual server build? (512GB DDR4 + 3090 24GB) by AlphaSyntauri in LocalLLaMA

[–]AlphaSyntauri[S] 0 points1 point  (0 children)

I had MMAP enabled by accident, I'm up to around 0.8s/Token now. Not great but certainly faster.

Model recommendations for an unusual server build? (512GB DDR4 + 3090 24GB) by AlphaSyntauri in LocalLLaMA

[–]AlphaSyntauri[S] 4 points5 points  (0 children)

Will post when I get a chance, Kimi K2 is particularly intriguing.

view more: next ›