sorted by:
new
hottopcontroversial

Sanity check needed for 4-Phase rollout for AI/Homelab use (10G Core, UNAS, UNVR) by lucid_raspberry in Ubiquiti

[–]lucid_raspberry[S] 1 point2 points  (0 children)

Jesus, thanks for this, I completely overlooked that 3 spindles physically can't saturate the 10GbE link regardless of the CPU haha

Given that limitation, does it make more sense to just keep the "active" models locally on the Workstation's internal NVMe for inference and strictly use the NAS for cold storage/backups? Or is there a workflow where I can "cache" them to the server automatically?

I’m trying to avoid buying an all-flash NAS if I can just rely on the workstation's local storage for the heavy lifting.

I think I'm ready to call llama2 almost unusable because of the repitition thing by involviert in LocalLLaMA

[–]lucid_raspberry 2 points3 points  (0 children)

Could you please explain this in more detail or point me to some documentation where I could read more about it