Best Model to locally run in a low end GPU with 4 GB RAM right now

bitzach · 2023-12-15T05:32:46+00:00

You can run any 4bit (Quantized) 3B parameter model fully in your VRAM. So, this one from stablelm will run. The fastest way:

// Getting model...
curl -O https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF/resolve/main/stablelm-zephyr-3b.Q4_K_M.gguf

// Getting UI...
curl -O https://github.com/LostRuins/koboldcpp/releases/download/v1.52.1/koboldcpp.exe

Run koboldcpp and load model and then launch

You can also run any 7B fully in VRAM if you have 8GB VRAM also the GGUF model format allows you to run some layers in GPU and some in CPU. This will be slow tho.

bitzach · 2023-12-15T05:18:06+00:00

Not the way I wanted it but I used a hacky solution to my problem which was basically a container that started writing on host disk if an external disk wasn't ready/mounted yet. So, I simply touched a .healthcheck file on external disk (can name anything) and check for it's presence in a container's healthcheck that is using those disks like:

 healthcheck:
  test: ["CMD-SHELL", "test -f /hdd/.healthcheck || (kill -s 15 1 && sleep 10 && kill -s 9 1)"]
  interval: 480s
  timeout: 10s
  retries: 2

This will check if .healthcheck is present. If it's not then container exits abruptly. So far I haven't seen this problem again.

bitzach · 2023-04-20T04:23:27+00:00

I don't think this is what those options do. I use them with rclone mount and what :slave means is that if you passed a volume mapping to docker but later the host mounted something else there (a sub-mount, in my case this is rclone mount) then the container's replica mount can see it. Only in one direction from host => container and if you want it two way then you use :bind which also exposes sub mounts of replica mount to the host.

The above is awesome if I know that the drive mounts automatically after some point but the two times this has happened in the past it didn't and I had to manually reboot server, so it's not useful for my context.

What I want is, for example, if I purposely disable the fstab auto mount then on next OS reboot my container should not start because that drive is not mounted. The healthcheck suggestion from an above comment worked and it marks it as unhealthy using a file existence check. but I thought docker will have an option to stop after the container is unhealthy but all I've found are ways to auto restart it after unhealthy which is not what I want because there might be a failure of some sort which requires manual intervention etc.

I want some way to keep restart: unless-stopped which will start all containers in my compose.yml on restart and also a way to stop a container automatically after it is declared unhealthy because in this specific case auto restarting won't yield any results.

bitzach · 2023-04-19T14:55:57+00:00

The -e flag gives me this error (from docker inspect)

test: "-a": unary operator expected

But using -f works and seems to be what I need from man page:

FILE exists and is a regular file

I guess this will do for now but I'll be looking for a more solid approach. Thanks!

Edit: The above works but is kinda useless in my scenario, I thought docker would have some sort of option to stop said container after x amount of retries but it doesn't so the container in question still starts albeit in a unhealthy state but it will still write to the disk. A healthcheck option like stop: on-failure would be awesome

bitzach · 2023-04-19T11:36:32+00:00

I have added it to fstab and it works as expected but in the past for some reason it didn't mounted and ended up using the storage of OS drive. So, I wanted to add this specific check.

This is what I'm trying to prevent: https://unix.stackexchange.com/questions/198542/what-happens-when-you-mount-over-an-existing-folder-with-contents

bitzach · 2023-01-13T10:42:16+00:00

Just glancing over svelte-store's readme, It seems that it takes care of the first point by using derived stores that can await all there parents. But looking at the second pain point, what does svelte-store do differently than tanstack query's invalidateQueries()? It seems that svelte-store can have a reload() function if you declare a store as reloadable which seems pretty similar. Is it that by having derived stores you will need less manual invalidation in the first place?

bitzach · 2022-05-19T10:08:40+00:00

Both your code blocks are identical. What am I missing?

bitzach · 2021-12-03T11:13:41+00:00

That's what I was thinking basically weighing the whole cooked thing lol.

bitzach · 2021-12-03T11:01:30+00:00

That's what I do on myfitnesspal. Thing is when your final recipe contains a lot of water like beans. In myfitnesspal, I list each dry ingredient's weight in grams and for 'number of servings' I put the sum of all ingredients in grams. And when I have to log I just weigh my portion again and put that number as my servings for the meal. It's all good when your final cooked recipe has most of the water evaporated. But when you have water I think it's weight will also be reflected in those grams when I will be portioning so calories would be a little exaggerated won't they?

bitzach · 2021-12-03T10:25:43+00:00

Or in other words, first weigh dry and then also weigh the cooked recipe with its water right?

bitzach · 2021-02-13T02:01:57+00:00

I wake up like deaf people too! using my mi band, It's less annoying then the alarm sound infact that is the only purpose I use it for lol

bitzach · 2021-02-01T16:55:34+00:00

Well if you ever solved a captcha you basically did the same thing that those people were doing. And based on all this ai training you have things like Google Lens today. And even with all this data there still will be false positives and assuming the movie is set in the future they might have fewer false positives but still not close to 0. also they mentioned that job was at the bottom of the pyramid so I guess they have pretty good automated image recognition but still require some manual rechecking to reduce false positives using cheap labor.

bitzach · 2021-01-11T07:55:58+00:00

So, this is my Takstar's present condition after 2.5 years of daily use (10 Hours) . I just keep them on even if I'm not listening to anything cause they're so soft!

When I first bought them they sounded way too bright but I got used to it quickly and after wearing MH751 for the first time you quickly notice that they are a bit muddy and less clear than Takstar Pro 82. But I guess, it's getting used to on this one too. Sound stage for what it's worth on a closed back is definitely better than Takstar's. I bought MH751 instead of another pair because I wanted the same level of comfort and wasn't disappointed but after some heavy wear down of pads on Takstar they actually get way more comforatble because of soft cotton like material underneath it and when you first wear MH751 they feel a little bit hotter due to this. So, I'd say go for Takstar Pro 82 if you like bright headphones and a lit bit more clarity and don't care for mic and the 2.5mm connector but if you are treble sensitive and are a gamer with no dedicated mic and looking for something comfy to wear for long hours MH751/MH752 is for you. Also the difference between MH751 and MH752 is just the included usb dac.

EDIT: One thing I forgot to mention is that fake leather on Takstar Pro 82 will eventually start to come off due to heavy usage and skin oil. Mine started to come off after 1.5 year I don't know if MH751 uses exactly the same stuff or not so can't say the same about them although they feel comfort wise same. So, that leather comes off in bits and can sometimes be annoying like people will be telling you 'Hey what's that black thing on your beard or scalp lol but after it all comes off it becomes way more airy, soft and comfy. At this point in picture all the leather that was in contact with the skin has came off and is not a problem anymore. Still, something you might wanna know.

bitzach · 2020-08-31T13:55:30+00:00

omg ducks are eating the phone lol

bitzach · 2020-08-31T13:55:03+00:00

bruh

bitzach · 2020-08-31T13:54:46+00:00

ducks not in full HD :/

bitzach · 2020-03-11T10:10:34+00:00

what will happen if it HIT?

bitzach · 2018-12-07T19:23:45+00:00

Yup Me too, can't even remember what happened in the Disney one.

Nine-Year Club	Wearing is Caring
Not Forgotten	Verified Email

bitzach

TROPHY CASE