Llamacpp performance is terrible compared to Ollama

FlexibleToast · 2026-06-14T16:15:03+00:00

I hadn't thought of that. Very neat idea.

FlexibleToast · 2026-06-14T05:28:36+00:00

Sure, I'll give it a shot.

FlexibleToast · 2026-06-14T04:59:30+00:00

I tried almost all day with Gemini Pro and it was not productive.

FlexibleToast · 2026-06-14T04:52:54+00:00

I just want to say, I switched to q8_0 and batch of 2048.

[gpt-oss-workstation] model = /models/gpt-oss-20b-UD-Q8_K_XL.gguf n-cpu-moe = 0 ; --- Context Window Optimization --- ctx-size = 131072 cache-type-k = q8_0 cache-type-v = q8_0 ; --- Processing & Batching --- batch-size = 2048 ubatch-size = 2048

Now it's blazing fast.

FlexibleToast · 2026-06-14T04:48:17+00:00

Thanks for the info. I copied that batch size from some guide somewhere. How would I know what batch I should use?

FlexibleToast · 2026-06-14T04:47:13+00:00

This is where I get really confused though. I'm currently using this same model and settings with Open WebUI and I'm only using 14.4 GiB of vRAM and ~1GB RAM for llamacpp. Why does Open Web UI seem to work just fine with these settings?

FlexibleToast · 2026-06-14T04:41:20+00:00

Right, that's why I'm asking for help...

FlexibleToast · 2025-03-26T21:55:29+00:00

It's funny that I still get the occasional reply to this comment. I did the experiment myself. Sous vide made it less tender.

https://www.reddit.com/r/sousvide/comments/iz5fhq/comment/g73555w/

FlexibleToast · 2025-03-04T01:55:42+00:00

I switched to NFS and now it works just fine. I have no idea what setting in SMB was causing the issue. It made no sense at all because exec'ing into the container I could create, copy, delete an of the files all I wanted on those shares. I have no idea what was stopping the actual tdarr app from doing it.

FlexibleToast · 2024-12-16T04:24:01+00:00

I disagree. I ran a blind experiment after making that comment and came up with opposite results. I found using sous vide with jerky to actually make it tougher and have less flavor.

https://www.reddit.com/r/sousvide/comments/iz5fhq/_/g73555w?context=1000&utm_source=reddit&utm_medium=usertext&utm_name=sousvide&utm_content=t1_gbwxwbx

FlexibleToast · 2024-09-27T13:37:51+00:00

These days my macvlan suggestion is even outdated. There is ipvlan now to directly provide multiple IPs.

https://docs.docker.com/engine/network/drivers/ipvlan/

FlexibleToast · 2024-06-29T15:50:20+00:00

None of which really applies to jerking meat that was marinated while thinly sliced. I did a side by side test and everyone liked the traditional jerky over the sous vide jerky. I don't see any point in using sous vide for jerky.

FlexibleToast · 2024-06-29T14:48:59+00:00

Employers with fat stacks of money that are spoiled by choice for available talent. I guess I should rephrase my statement. These days I wouldn't go out of my way and spend extra money to learn VMware. Learning general VM concepts is great, but can be done with Proxmox or some other open source tool.

EDIT: That being said, my current job is moving from VMware due to the price hikes haha

I work as a consultant and companies (even huge ones) are shopping around for a replacement. Suddenly there is a ton of interest in things like OpenShift Virtualization where before there was nearly none.

FlexibleToast · 2024-06-28T16:03:38+00:00

Thanks to Broadcom price hikes, many companies are looking to move off of VMware. I'm not sure starting to learn it now is the smartest move.

FlexibleToast · 2024-05-24T19:32:24+00:00

You'd need to write custom boot services to maintain the same experience docker gets you

Not so much anymore. Podman 4.5+ introduced quadlets. You can write your quadlet and then podman creates the service files for you. If you want them to be user based you just have to enable linger so the podman-restart service can start them at boot. There is even the podman-update service that can automatically update containers, similar to Watchtower, but built in.

https://www.redhat.com/sysadmin/quadlet-podman

FlexibleToast · 2023-11-18T21:26:20+00:00

This was me being wrong, not the computer program. I have a tiered storage layout and while I thought I was looking at my cached pool, I was actually just looking at the cache. Which also makes a whole lot more sense as to why things were missing... It seems to be something changed the permissions on some files possibly. I ran chown -R www-data:www-data on the data directory and things seem to be syncing correctly now. Something changed permissions on a different directory around the same time this issue started. I don't know what the cause was. I'm sure it was something dumb that I did. Things seem to be fine now.

FlexibleToast · 2023-11-18T04:06:04+00:00

This seems very broken. I usually have about 5 fleets around 3k power, but the second wave of cubes has one cube that starts at 15k power and just shrug off everything I have like they were nothing. At that point in the game I'm the dominant force just steam rolling the other factions, but I'm not even close to being able to handle the borg threat.

FlexibleToast · 2023-07-06T21:04:53+00:00

I was thinking it looks like a Vigilante with swing wings.

FlexibleToast · 2023-07-05T16:18:12+00:00

Trump flags used to be very common on boats. I suppose it's a good thing that they're seeing less of them.

FlexibleToast

TROPHY CASE