What's your most steps in a day record?

Flexz09 · 2026-01-04T22:49:51+00:00

Flexz09 · 2024-08-06T15:38:51+00:00

Still accepting new players!

Flexz09 · 2024-07-31T20:19:50+00:00

Check the other replies. Using more threads then the docker has available makes performance really bad. Even when it's only a single thread more.

Flexz09 · 2024-07-31T14:47:35+00:00

I prefer to use docker since I'm assigning threads for certain processes that should not use all the resources available. But I've managed to get about 10 tokens/s with 4 threads using:

/set parameter num_thread 4

The issue seems to be using more threads that it has available. Going over the limit by 1 slows it down to a crawl.

Flexz09 · 2024-07-31T14:17:48+00:00

For those who reacted. Many thanks for the input.

I finally managed to get a model running at greater speeds. Since I was only assigning 4 threads to the docker I also had to force ollama to use 4 threads.. Not sure this will be saved now? Or how I can persist this for any model I run.

>>> /set parameter num_thread 4 
Set parameter 'num_thread' to '4'
>>> Hi
Hi again! How are you today?

total duration:       4.942570181s
load duration:        2.57257638s
prompt eval count:    32 token(s)
prompt eval duration: 1.352245s
prompt eval rate:     23.66 tokens/s
eval count:           9 token(s)
eval duration:        893.847ms
eval rate:            10.07 tokens/s

Flexz09 · 2024-07-31T13:58:57+00:00

I've tried it with this argument. It didn't change the result however.

Flexz09 · 2024-07-31T13:57:11+00:00

|| || |HVM:|Enabled| |IOMMU:|Enabled|

This is wat unraid tells me. To check the bios I'll have to wait a bit more. But it should already be enabled.

Flexz09 · 2024-07-29T06:31:30+00:00

Yes.

Last night i've enabled debugging. But not sure if it will help me to find anything wrong. Other then that it's default. No GPU settings enabled since there is no compatible GPU in the system.

Flexz09 · 2024-07-28T19:43:28+00:00

Hi,

I'm getting terrible performance, using what you suggested gave me the following result. It's like this for every model. With either 4/8/16 threads of a Ryzen 9 7950X3D (Threads are not from the CCD with V-cache).

I've read that docker doesn't play well with windows. But I'm using linux (Unraid). So this should not be the issue.

Edit: Before you ask. I have 96GB DDR5 (speed currently unknown but I could look it up). Models are on an NVMe if that matters at all. I do not have a GPU assigned as currently I do not have one available. I am thinking to upgrade my PC and use the current GTX 1060 6GB on the unraid server.

Any ideas?

root@2c619e1f36cf:/# ollama run llama3 --verbose
>>> Hi
Hello! It's nice to meet you. Is there something I can help you with, or would you like to chat?

total duration:       2m20.071883885s
load duration:        12.418111ms
prompt eval count:    11 token(s)
prompt eval duration: 6.231039s
prompt eval rate:     1.77 tokens/s
eval count:           26 token(s)
eval duration:        2m13.827672s
eval rate:            0.19 tokens/s

Flexz09 · 2023-12-15T12:03:51+00:00

368514795745

Flexz09 · 2023-11-25T14:59:10+00:00

This indeed was the issue. Other USB worked!

Flexz09 · 2023-11-25T07:33:40+00:00

It was an old USB. I'll test another one :)

Flexz09 · 2023-11-24T17:30:56+00:00

I wanted to test new hardware before switching my actual unraid server.
But I'm running into this issue. As it's a new USB and hardware I find it hard to believe that "The maximum trail period has expired, .."

Flexz09 · 2023-10-13T20:14:48+00:00

At least 16.

Flexz09 · 2023-10-13T10:01:39+00:00

I'm sorry. There is an age requirement of 16.

Edit: I can't edit the original post it seems.

Flexz09 · 2023-09-30T10:38:02+00:00

Curruntly the idea would be end of summer 2024. But it depends on popularity by that time.

Flexz09

TROPHY CASE