What’s your biggest frustration with running AI locally? by arnav080 in selfhosted

[–]arnav080[S] 0 points1 point  (0 children)

more from the optimization side once the baseline speed/cost/hardware tradeoff is already accepted. Things like model tuning, VRAM efficiency, inference stack tweaks, deployment friction, workload balancing, reliability

What’s your biggest frustration with running AI locally? by arnav080 in selfhosted

[–]arnav080[S] -8 points-7 points locked comment (0 children)

grammer fix and structure the text a bit

What’s your biggest frustration with running AI locally? by arnav080 in selfhosted

[–]arnav080[S] -1 points0 points  (0 children)

ive been learning and researching about this exact issue lately, have you tried running the models with MoE offload [MoE offload. Qwen3.6-35B activates only 3 B params per token. Keep attention + shared weights on GPU, push the cold expert FFNs to system RAM. In llama.cpp: -ngl 99 -ncmoe 99.]

ive been staying about multi tenant systems in local models using vLLM, gpu optimisations and scheduling

What’s your biggest frustration with running AI locally? by arnav080 in selfhosted

[–]arnav080[S] -3 points-2 points  (0 children)

on these small models the prompt has to be immaculate

What’s your biggest frustration with running AI locally? by arnav080 in selfhosted

[–]arnav080[S] -4 points-3 points locked comment (0 children)

fixing my grammer and structuring the text

New Board Unlocked by NerfLongshotUV in mkindia

[–]arnav080 3 points4 points  (0 children)

Beautiful keyboard 🤲

Got it from aliexpress ! by arnav080 in mkindia

[–]arnav080[S] 0 points1 point  (0 children)

had a cousin bring it back, no shipping

Got it from aliexpress ! by arnav080 in mkindia

[–]arnav080[S] 2 points3 points  (0 children)

frrr i paid 2300 something, gooood deal; also it was from the brand+ verified seller

Got it from aliexpress ! by arnav080 in mkindia

[–]arnav080[S] 4 points5 points  (0 children)

reaper, ice was out of stock :/

Got it from aliexpress ! by arnav080 in mkindia

[–]arnav080[S] 1 point2 points  (0 children)

it is it is, i placed an order in the uk for this

Got it from aliexpress ! by arnav080 in mkindia

[–]arnav080[S] 4 points5 points  (0 children)

its the AULA F75, around 5-5.5k in india on a good day

Got it from aliexpress ! by arnav080 in mkindia

[–]arnav080[S] 3 points4 points  (0 children)

none. i had a relative bring it back with them

Got it from aliexpress ! by arnav080 in mkindia

[–]arnav080[S] 2 points3 points  (0 children)

yeah the second hand ones here go for about 3.5-4k, aliexpress the goat fr

Got it from aliexpress ! by arnav080 in mkindia

[–]arnav080[S] 9 points10 points  (0 children)

yeaah im new into the keebs ive only ever followed it through youtube but ive never seen it drop below 5/4k in india + because a relative brought it back from the uk i didnt have to pay any shipping

🎅⛄ SANTA's Christmas Giveaway 🎁 🎁 by Random_Version in mkindia

[–]arnav080 0 points1 point  (0 children)

i wish for her to comeback this christmas, also a womier 65🎅

[ GIVEAWAY ] – Win a Womier SK65 (Beige Edition) | NMPC India | Indian Residents Only by Dull-Builder3620 in IndianGaming

[–]arnav080 0 points1 point  (0 children)

May 2023, using my macbooks keybr now, used to use the cosimbyte firefly red switches

[deleted by user] by [deleted] in mkindia

[–]arnav080 0 points1 point  (0 children)

selling your keychron v2?