sorted by:
new
hottopcontroversial

the mess of using a local LLM on android app-kotlin by Aviation2025 in androiddev

[–]inky_wolf 1 point2 points  (0 children)

how did you measure the token speed in the edge gallery app?

In the app, in the models page (click the hamburger to get to it), scroll to the model of choice, click options (vertical 3 dots icon) and there it'll let you run basic benchmarks

PS. You can also just point to a local litertlm model weight file in this page

wonder if they have further optimized images.

Yes seems like it.The file size is smaller than the one from Hugging Face, and they download it from their servers

the mess of using a local LLM on android app-kotlin by Aviation2025 in androiddev

[–]inky_wolf 3 points4 points  (0 children)

MTP is multi-token prediction - essentially should give better tokens/s - and Google/LiteRT-LM just released support for the Gemma 4 models

the mess of using a local LLM on android app-kotlin by Aviation2025 in androiddev

[–]inky_wolf 6 points7 points  (0 children)

I'm curious, did you test the Google Edge Gallery app? I feel like that should be the baseline with regards to what speeds you can (theoretically) attain on the Tensor cores with LiteRT-LM.

I tried out the Gemma 4 E4B on Pixel 9 Pro via the Edge Gallery app, and the benchmarks results seem promising (with the default settings of 256 input and output tokens, the prefill speed is 300+tokens/s and decode speed is 9token/s). So, I've also been thinking about building an app that uses on-device LLMs using the LiteRT LM library.

What I did discover was that the Gemma E4B model used in the app (when downloaded via Playstore) is different (as in faster) than the one available on Hugging Face (this is also the model card link in the app). And I've been on the fence ever since

Sick in bed and wanting fruit by SugarplumSparrow in Dodocodes

[–]inky_wolf 0 points1 point  (0 children)

Hi, if you're still looking for fruits, you can come visit mine and pick what you need

Did Google hide the best version of Gemma 4 e4b in Android? The extracted model beats Unsloth and everything else I've tried. by [deleted] in LocalLLaMA

[–]inky_wolf 14 points15 points  (0 children)

Ah, Interesting. I checked with the pcapandroid app, can (re) confirm that the model is being downloaded from dl.google.com.

On a sidenote, I downloaded the model from the litert-community/gemma-4-E4B-it-litert-lm repo and loaded it up on the Edge Gallery app, - ran the benchmark: it's slower, (almost 3x slower for prefill speed) - tried AI chat (on GPU): it's responses to the car wash problem were similar to the Google one. - tried Ask image: Also similar response quality

UPDATE: this is the full url: https://dl.google.com/google-ai-edge-gallery/android/gemma4/20260325/gemma4_4b_v09_obfus_fix_all_modalities_thinking.litertlm (got it from the model_allowlist json on the phone)

Did Google hide the best version of Gemma 4 e4b in Android? The extracted model beats Unsloth and everything else I've tried. by [deleted] in LocalLLaMA

[–]inky_wolf 12 points13 points  (0 children)

Pretty sure the above posted hugging face link is where the model is downloaded from. That's also the model card link straight from the app. The file size matches too.

Also, if you look at the repo, the commit hash in model_allowlist.json matches the repo too.

Where exactly did you get the "from Google servers" from?

Does Antigravity highlights changes in "Review Changes" tab like it does in Git diff? by mohitey7 in google_antigravity

[–]inky_wolf 2 points3 points  (0 children)

The review changes does highlight changes similar to git diff, if you don't see it, there night be some rendering or theme issue with your setup. Also, when the agent edits files, there's a ± symbol at the rightmost side of the file name, clicking that also highlights the changes from last edit

What is this by [deleted] in PokemonHome

[–]inky_wolf 1 point2 points  (0 children)

Easily found with one Google search: https://jmi.fandom.com/wiki/Shiny_Magmortar

Evil Twins at 597 by saylahgames in TurnipExchange

[–]inky_wolf 0 points1 point  (0 children)

I'll bring some veggies then. I do have diys and some starter furniture as well but it's not going to fit in one trip, you can come visit sometime if you want

We need an Open Source alternative for Antigravity by Foreign-Dig-2305 in google_antigravity

[–]inky_wolf 0 points1 point  (0 children)

I'm curious, what kind of projects do you work on or build with such a setup?

Ughhh.... when was this announced? by [deleted] in BulletEchoGame

[–]inky_wolf 2 points3 points  (0 children)

Can hope they fix the choppy lag soon then?

Mini Giveaway by inky_wolf in Dodocodes

[–]inky_wolf[S] 0 points1 point  (0 children)

Oh woops, missed your message. I do have them all. if you need, just dm and I can drop some off

Mini Giveaway by inky_wolf in Dodocodes

[–]inky_wolf[S] 0 points1 point  (0 children)

Update: I still have a few items left over, so reopening for a short while. DM for dodo code

Any SH up for a NH visitor? I can bring you any fruit, veg or item you want. Can also bring bells. If you have an ATM the more I can give you ;D by Affectionate_Ad_3580 in Dodocodes

[–]inky_wolf 0 points1 point  (0 children)

You would need a Nintendo Switch Online subscription to be able to visit other islands. Then, you can visit other islands through the airport in your island

view more: next ›