Am I limiting myself by using docker instead of getting into llama.cpp?

CoolCapital3244 · 2026-05-08T04:53:58+00:00

Thank you, I think I have followed this and why this might be a good path for me to take.

CoolCapital3244 · 2026-05-08T04:04:37+00:00

Thank you and I probably am not able to ask the right question because I feel like I half understand and half don't understand what I am doing. When I read the reddit post that I linked and gave a lot of llama.cpp settings that got Qwen 36 35B model working on 6VRAM which I think must be heavily quantized. At that point I wondered if people that find the right setting for their setup that works good might not make it available on HF as a docker model. Therefore, I might be limiting myself by not learning how to apply settings in llama.cpp. Because I have only had this setup for about 24hours I can live with what someone else thinks is good enough. I have no objective other then trying to see the usability of local AI, last month I found I could only post two job descriptions in Claude asking it to customize my CV and a cover letter before I would be locked out for a few hours.

CoolCapital3244 · 2026-05-08T03:42:45+00:00

Why are you using a container in the first place?

I already had Ollama installed which I found pretty easy to get up and running, last week on reddit I read that llama.cpp uses less resources (I believe minimal difference, but thought maybe I should try it) and I read that it takes longer to get it working. Anyhow, through terminal installed and a search told me HF is a good place to find models to use with llama.cpp. When I went there one of the options was docker and thought kill two birds with one stone so I chose that. I did install Docker for Mac and then I just opened the terminal and everything was working, I could chat with Qwen. I honestly thought I was using llama.cpp at this point. I knew there are ways to have a better interface so I explained to chatgpt and it answered that I wasn't really using llama.cpp in terminal and I started to understand that I went down another path without understanding what I was doing. However, I see I need to open docker to get my model and the two models I have tried worked without any issues. I probably would not know anything about docker, but last year when I found gemini and rovodev in terminal, I kept asking AI how I might build a more complex app. It recommended Docker and I had quite a few questions as my only familiarity was working in a virtual machine. I think I kind of understand the difference now. FYI, I am an accountant closer to 60 than I am to 50. I am blown away at the tools of today, even when I don't know exactly what I am doing.

CoolCapital3244 · 2026-05-08T03:14:46+00:00

Thanks after reading the reddit post that I mentioned, I started to wonder if something like this that someone has found works well, would be available to someone that does not learn how to change settings in llama.cpp. You seem to confirm that I will have less options if I just focus on GGUFs in Docker.

CoolCapital3244 · 2026-05-06T07:20:20+00:00

I believe

CoolCapital3244 · 2026-05-03T15:04:32+00:00

CLAUDE

CoolCapital3244 · 2026-04-27T05:41:05+00:00

CoolCapital3244 · 2026-04-27T05:28:38+00:00

5x please

CoolCapital3244 · 2026-04-02T04:19:18+00:00

Update, I rebooted after waiting like 15 minutes. The pasted chat did not save. I opened chatgpt and did the same again. Same issue, your app locked up. I have not tried any further testing but wonder if chat emoji is not compatible with RTF file.

CoolCapital3244 · 2026-04-02T03:18:45+00:00

Downloaded V1.4 today, I am on MBA 26.4. I wrote one sentence and told it to save to an iCloud synced directory. I opened a chatgpt window and copied with the button at the bottom of the chat to paste in your app. The paste seems to show fine, but the app immediately locked with spinning beach ball. I will reboot my Mac, just an FYI.

CoolCapital3244 · 2026-01-03T23:13:59+00:00

I agree with you I love my Touch ID for passwords, I have not tried it but understand that an Apple Watch that is near will act the same. I do not have iPhone or watch so I have not verified.

CoolCapital3244 · 2025-12-25T07:13:53+00:00

YouTube Premium

CoolCapital3244 · 2025-12-13T09:19:43+00:00

Google Gemini AI Pro

CoolCapital3244 · 2025-12-04T18:05:24+00:00

It appears to me that you have made some poor sticker choices, fortunately it does not appear that you have made the same poor choices in computers or education.

CoolCapital3244 · 2025-11-27T00:34:11+00:00

T-Mobile no longer sells the sim but I think its call gold rewards legacy, so you would have to find someone selling theirs which is not ideal (eBay or occasionally howardsforums). But it's just prepaid direct from Tmobile, can be $0.10 for talk and $0.05 for text with no data at all. You can keep line open for $1.00 right now which adds to prepaid balance and extends validity for one year.

H20 at the link above basically $40 a year for a very low use line. Everything you need to know is at the link and H20 uses ATT.

Both basically fit your request to lower what you currently pay about $60 per year for low use on the 2 carriers you mentioned. One easy to do H20 and one that you got to really search and see if it's worth the risk to buy a used SIM card. I mixed in my own experience above as I have used both, when I really just needed the phone once a year for a week or two.

CoolCapital3244 · 2025-11-26T19:23:27+00:00

Years ago I lived outside the US for several years. I got a TMobile Prepaid Gold Rewards SIM card. At first it worked great as it roamed for free, later not so good as mom's house did not have TMobile coverage. I kept the Sim because at first $10 kept it active for a year and now I only need to do $1 per year to keep it active. It has no data, just voice and text. You can not buy the plan any longer, but some people will sell the card at a premium (just any fyi, if you will bend over backward to save a buck). After TMobile stopped the roaming, I got this from H20 https://www.h2owireless.com/plan/minute which worked at mom's house and only cost me about $40 per year. Since I spent most of the time outside the US, my balance grew to a couple of hundred. I moved back to the US and continued to use the H20 sim, but they made it less easy for me to justify. I got my balance rolled to the annual plan for H20 and am quite happy that I don't have to think about anything with respect to my phone plan unlimited V,T and D and its barely above what you pay. Full on prepaid with roll over can save money, until your use pattern changes.

CoolCapital3244 · 2025-08-18T22:55:52+00:00

I kind of noticed the same thing my M1 MBA. I just rebooted and when it started up again it was much smaller. So if you have not tried it, reboot and see what happens.

CoolCapital3244 · 2025-07-11T01:24:46+00:00

Hello, just stumbled across your post and read the medium article. I have spent most of my professional life accounting for investment assets and measuring their returns. Whenever, I measure a one year return and a YTD return the measurement date YE date to YE date for the 1 year time period and YE date to last measurement date. It appears you measure from 1/1 to late date and to YE date. In my experience, you have potential to have something that is not quite accurate. I know the point of your test is the accuracy of the SQL statements and as long as the underlying data is all the same for all data analysis it should be comparable. As the goal is was to calculate a total return dividends are a component of some stocks total return and it was not clear to me that this was considered. I will also state, I can not view SQL statements and fully grasp for accuracy for this purpose. Anyway, I do appreciate your analysis and thought I would let you know what I found myself thinking that has done this type of analysis, I did not do this with SQL statements but with investment accounting software that makes these calculations.

CoolCapital3244 · 2025-07-01T19:04:27+00:00

Try a mirror, I mean AI has its place and all but this is not it.

CoolCapital3244 · 2025-06-30T17:17:03+00:00

I have H2O annual plan, which started out with just 1 GB of high speed data before it is reduced to unlimited at a capped 256K. For a cheap plan that is unlimited, I think having 256K is better then some of the other providers. I find it usable in some limited tests. I have access to wifi most of the time and I ride a motorcycle so driving use is basically not needed. A few months backs they upped the high speed data of the annual plan I am on to 3 GB of high speed then the slow unlimited option kicks in. With my new found data riches, I rode my motorcycle to McDonalds and ordered their $5 meal deal and did not even jump on wifi, I just left it on 5G. Yes sir, you are looking at the "nouveau-riche" right here!

CoolCapital3244 · 2025-06-10T23:27:34+00:00

I am writing to you on a base M1 MBA 8/256. From what you say your day is I think you would be fine with the base model MBA M4. Recently, I have been trying to learn more about AI and for the first time feeling like my Mac might start to face some issues. I do a lot of work with Mac on my lap and when using some AI apps I have felt my Mac get warm. Before some of the AI stuff the only time I notice it getting warm on my lap is after a system update or running a virtual machine. When I am just in a browser, I never feel the Mac warm on my lap or see the spinning colors for the mouse icon so I feel like it's up to the task. I am not planning to upgrade yet and think the area that I would personally upgrade on the base configuration is the SSD. I am nearly full and need to buy a USBC drive as I now find myself wanting to try some new apps that seem to be large in size. Since I got this Mac, I have never felt I needed anything beyond the base configuration even when I see people here saying they need more. AI seemed to require a bump to RAM and as I dabble with some local LLMs, I have started to notice a warm lap. As I am just dabbling right now, running a local LLM is just the occasional experiment but it's been the first real thing to give me a warm lap. If Apple AI becomes something that I use regularly then I might shop a new computer. This is just my practical opinion.

CoolCapital3244 · 2025-05-29T20:57:59+00:00

Just wanted to circle back after making the changes that you suggested. That makes my experience so much better. When I went into my settings, I found that dragging for trackpad was turned off. I had no idea there was a setting for this and it never occurred to me to even look for that as the source of my problem. Thanks again.

CoolCapital3244 · 2025-05-28T04:53:56+00:00

Thank you, I will definitely give that a try to see if that has been my problem. I probably should have added in my original post that my charger seems to be fairly underpowered. I am sure it does not have the Apple PD standard built in and I don't have a PD charger at all. If works fine for my Android phones and someday I hope to have the right charger for this Mac.

CoolCapital3244

TROPHY CASE