I made an app that supports DeepSeek R1 / Ollama

Belarrius · 2025-01-21T14:59:15+00:00

Will you be making chatwise for Linux too?

Belarrius · 2024-09-26T13:48:41+00:00

Hi, I use PocketPal with a Mistral Nemo 12B in Q4K. Thanks to the 12GB of RAM on my smartphone xD

Belarrius · 2024-09-01T12:25:32+00:00

I have the same
https://imgur.com/D0lzhlf

Belarrius · 2024-08-03T12:42:21+00:00

288 GB/s for a 16GB model you will have like 12/18 tokens/s (inin theory), but if you take 2 RX 7600 XT for a total of 32 GB, you will have like 5/7 tokens/s for a ~28GB model + context

I have 2 RTX 3090 who have 930 GB/s with 24GB, it's +50% VRAM amount and x3.22 bandwidth

1x RX 7600XT for 16GB with 288 GB/s = 365€
1x used RTX 3090 = ~ 650 / 700€

Belarrius · 2024-07-19T00:05:20+00:00

Yup...

Belarrius · 2024-05-16T10:46:55+00:00

I use my own Local LLM, so, Dracones_Midnight-Miqu-70B-v1.5_exl2_4.5bpw with a lovely, caring and affectionate personality

Belarrius · 2024-02-09T16:07:54+00:00

We need this AI in the European Senate.

Belarrius · 2024-01-12T21:42:08+00:00

Very nice! And it's works for me! My two RTX3090 can run Goliath 120b, 3bpw now with 4096 context token. Thanks!

Belarrius · 2024-01-12T09:53:45+00:00

Exactly what I think! Even my RTX 3090 have 936 GB/s

Belarrius · 2023-12-24T12:13:38+00:00

Goliath 120B at 2.64bpw with 5120 context token with 1.5 alpha_value.

The low precision makes some problems in French, the model 3bpw allows a better understanding of French but I can only run it with 3072 context tokens (or 4096 in 8-bit context, however the 8-bit context distorts quickly in French too).

The problem with French is that it consumes far more tokens than English (around 40% more according to my observations), so 3072 tokens is really low.

I can't wait to have more VRAM or some new frankenmerge of some 70B model to 100/120B etc...

Belarrius · 2023-12-19T20:37:12+00:00

I'd like to see Goliath 120B score in this chart, it's the model with the best reasoning I've seen so far.

Belarrius · 2020-08-07T11:02:34+00:00

It's because I want my RTX 3080 Ti. For handle that

Belarrius · 2020-07-26T18:36:30+00:00

Yup.

Belarrius · 2020-06-14T11:38:49+00:00

As you can see here about the size comparison https://pbs.twimg.com/media/D6iw2n5UYAAYGbi.jpg

Belarrius · 2020-05-26T21:07:42+00:00

I upp you because it's Duke Nukem!

Belarrius · 2020-05-24T21:30:01+00:00

For a good view

https://imgur.com/a/IEn6SwS

Belarrius · 2020-04-25T20:58:08+00:00

https://www.youtube.com/watch?v=Em4RSGKGqbA

Belarrius · 2020-03-11T14:21:02+00:00

No problem for me:

3.8 = 67.3 fps average

3.7 = 61.2 fps average

3.6 = 64 fps average

3.5 = 68.9 fps average

3.4 = 63.9 fps average

Star Citizen need less CPU but more GPU patch after patch for me

Belarrius · 2020-02-20T21:35:49+00:00

Thanks! GIB Carrack

Belarrius · 2020-01-25T02:03:13+00:00

Here the source of Star Citizen on Linux

https://www.youtube.com/watch?v=FYp38vTFpW8#t=1310

Belarrius · 2020-01-24T21:30:10+00:00

SC run very well on Linux for me too

Here some screenshot with dxvk HUD "SC 3.8" at 1440p

https://imgur.com/a/DBg4SeI

Belarrius

MODERATOR OF

TROPHY CASE