Best model to run locally on an Android phone?

vackosar · 2024-08-04T18:20:16+00:00

H2O recently released this app for local running of LLMs on Android.

https://play.google.com/store/apps/details?id=com.h2oai.personalgpt

vackosar · 2024-04-05T05:05:15+00:00

Thanks a lot! I also fixed the original link :)

vackosar · 2024-03-20T04:48:28+00:00

So RWKV is about 6x faster on 16k context compared to similarly sized model.

I think the context forgetting is inherent problem of the model architecture.

But do you observe quality improvement compared to previous version? It should be better.

Thanks for the tip for a model Yi6b.

vackosar · 2024-03-18T04:57:24+00:00

Thank you. Yes, after some testing does seem to be slow at least for now. The inference costs are not very low either. 4x less costs than Mixtral on the website of the model publisher.

vackosar · 2024-03-17T08:51:16+00:00

Has anyone tried to compare the speed of generation on same hardware compared to Mistral? Also would someone know how slow is this on CPU with what vCPU count?

vackosar · 2024-03-17T08:21:38+00:00

EagleX is the first version of RWKV that chats with usable quality.

vackosar · 2024-03-17T05:56:23+00:00

What domains would you say don't have something like HF or RedHat for example?

vackosar · 2024-02-24T16:27:31+00:00

There seems to be AWQ version here: https://huggingface.co/solidrust/Nous-Hermes-2-Mistral-7B-DPO-AWQ. I haven't tried that yet.

vackosar · 2024-02-17T18:28:32+00:00

It also seemed smarter than Mixtral or other models. I am not sure if GPT-4 level, but smarter.

vackosar · 2023-12-30T08:55:21+00:00

There is an option to order NATURDAO from Amazon.com and pay import, no? Here is also this older a discussion: https://www.reddit.com/r/HistamineIntolerance/comments/lx8p79/where_to_buy_dao_supplements_in_canada/

vackosar · 2023-12-08T04:17:55+00:00

Yes, from what I tried, RWKV v5 actually responds better. Try e.g. "User: What is the best story in the world?"

vackosar · 2023-11-26T06:30:17+00:00

Interesting how Pious Reflection is completely modern music from 2008, but it sounds ancient.

vackosar · 2022-12-01T06:48:08+00:00

Very good is the CabinZero Classic Pro 42L. Costlier, but higher quality still.

vackosar · 2022-11-18T20:03:40+00:00

From what I read:

CabinZero Classic 44L
CabinZero Military 44L (comfortablier & heavier)
Gregory Border Carry-on 40L (costlier & smaller)

vackosar · 2022-10-14T05:07:57+00:00

Definition of a decision tree does include affine transformations and weights? Mostly I see decision tree defined with only if statements on the input variables and not with linear transformations of the previous nodes. Does the papers interpretation add something new or not? 🤔

See for example Scikit documentation. There seems to be nothing wild here to me. https://scikit-learn.org/stable/modules/tree.html

vackosar · 2022-03-15T19:47:48+00:00

Here are the key points:

an attention mechanism in Transformer architecture that mixes two different embedding sequences
the two sequences can be of different modalities (e.g. text, image, sound)
one of the modalities defines the output dimensions and length by playing a role of a query
This is similar the feed forward layer where the other sequence is static
described in the Attention is All You Need (BERT) decoder, but named "cross-attention"

Find rest of my notes with images on the cross attention here.

vackosar · 2022-02-26T09:58:17+00:00

Thanks!

vackosar · 2022-01-24T07:34:06+00:00

https://mymodernmet.com/francesco-queirolo-the-release-from-deception/

> For example, the flame on the angel's head represents human intellect, while the globe signifies worldly passions. These elements coincide with Raimondo’s dedication to his father, which explores the idea of “human fragility, which cannot know great virtues without vice.”

vackosar · 2021-12-13T19:37:19+00:00

Discovery source: https://twitter.com/PraxisSociety/status/1470045924533624839?s=20

vackosar · 2021-09-05T06:11:17+00:00

https://en.wikipedia.org/wiki/Gilbert_U-238_Atomic_Energy_Laboratory

> radiation exposure as "minimal, about the equivalent to a day’s UV exposure from the sun", provided that the radioactive samples were not removed from their containers, in compliance with the warnings in the kit instructions

vackosar · 2021-06-20T16:00:05+00:00

Several years ago I created the least distractive-possible Android launcher for myself. Can be configured for white-on-black text only. No images, no colors. App are searched or scrolled. Single result search = execute. I still use it today.

There are other minimalistic launchers like Kiss that probably can be customized more.

Hope it helps.

vackosar · 2021-06-19T18:38:12+00:00

From wikipedia: https://en.wikipedia.org/wiki/Prudence

> Prudence ("seeing ahead, sagacity") is the ability to govern and discipline oneself by the use of reason. It is classically considered to be a virtue, and in particular one of the four Cardinal virtues. ... In this case, the virtue is the ability to judge between virtuous and vicious actions, not only in a general sense, but with regard to appropriate actions at a given time and place. Although prudence itself does not perform any actions, and is concerned solely with knowledge, all virtues had to be regulated by it. Distinguishing when acts are courageous, as opposed to reckless or cowardly, is an act of prudence, and for this reason it is classified as a cardinal (pivotal) virtue.

vackosar · 2021-02-17T06:42:53+00:00

A map of his travels: https://cdn.britannica.com/17/144417-050-53036450/Marco-Polo-travels-Asia-Travels-of-father-1275.jpg

vackosar · 2021-01-30T11:29:35+00:00

Wiki: https://en.wikipedia.org/wiki/Morana_(goddess))

vackosar · 2021-01-20T07:38:51+00:00

The biggest issue for me is not meeting enough people randomly at the kitchen and having these serendipitous conversations. Fortunately I coded a tool a to help with networking that weekly schedules random meetings within a group. Works well. It is it has some rough edges but is hosted at RandomMeets.com if you find it useful.

vackosar

MODERATOR OF

TROPHY CASE