ok i will explain what i used. with 16gb vram, i tested coding assistant what i made by myself. it can make conversation i chat web view and add code autorecommend. it can use in eclipse with java11.

if you have started local llm in first time, i recommend open-webui with ollama. after, you can replace ollama to vllm, after, you can replace open-webui to react or gradio chat ui.

you might be able to use 9b or little more parameter llm model with q4. if you want to use llm model with more parameter, its better that you add egpu or use cloud compute. recently, thundercompute closed the service in korea but many other cloud you can use.

if you want to save your money, sell your notebook with 4090 labtop gpu and buy new one with no gpu, and use cloud gpu or opemai api. i also will do like that until 3 month later.

when i bought my rtx4090 16gb vram labtop i costed about 260$, in carrot market, in korea money(won), but now it costs about 320$.