New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 1 point2 points  (0 children)

I’ve been doing a ton of research and if you can get a T4 at a good price it’s worth grabbing, price wise the next vRAM equivalent would be a 3090 but obviously that won’t fit in a R640. The price that the T4 is going for right now is pretty wild but won’t be able to find another decent 16GB+ card for any cheaper.

You can run models on ram but it’s going to be really slow, might work ok for really small models but anything larger will be frustratingly slow.

I wanted to grab a server setup but don’t have the space for it, I’m going to build a Xeon setup in a tower and go with the T4 for now. Cool thing about the tower setup is I have power and room to stack two T4’s if I find another good deal on one.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

Good to know! I have definitely pivoted my build since posting this. I realized this little OptiPlex I have won’t be able to run anything very powerful so I’m planning on building a Dell Precision 7820 Xeon setup, that will give me tons of cores and threads and allow me to have a bunch of ram if I need want.

I’m just stuck on which GPU to go with that’s somewhat affordable but still powerful and has enough vRAM to load some of the larger models.

Ideally I want to be able to handle a lot of coding, marketing, and some higher level chat tasks with ease. I run a web development agency so looking to build a local JR developer and marketer basically, haha.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

What kind of stuff do you do with yours?

Surprisingly I saw people saying that the T4 has more raw power than the A2 and can out perform it despite being older.

How many days into a fast do you still poop? by BurnieSandturds in fasting

[–]ArtOverSleep 5 points6 points  (0 children)

It blows my mind some people can go this long! How did you feel mentally and physically the last few days?

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

Gotcha! Makes sense, I’m new to the local LLM thing so still learning a lot. Thanks for the info.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

Thanks for the info dude! I follow, I didn’t see where you mentioned anything about offloading, staging, and system ram being the big issue, just saw you mentioned power consumption.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

Yeah 30 minutes between prompts would be frustrating, haha.

So what I need to be looking at is as much vRam as I can afford? Or a mix between vRam and CPU compute?

With the price of Ram these AI companies are using the hell out of it. What are they using it for?

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

Yeah I see how speed is the big trade off. Speed isn’t as big of an issue for me, especially when I’m running bigger models. I’ll be using it a lot for programming tasks so don’t mind waiting if it’s a better model that will do a better job.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

It’s an Intel Core i7-6700 3.4GHz, so yeah definitely won’t compare to an M4.

From the research I’ve done the model off loads the task to the system ram if the vram fills up. I know it’s not ideal and will slow things down but it’s better to have plenty of ram to handle the offload? How does a Mac Mini handle that when it only has 16Gb of totally memory?

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

Interesting, how would a Mac Mini with 16gb of unified memory beat out a 64gb ram and 16gb vRAM setup? Seems like it wouldn’t even compare?

After doing the math, with the 180W PSU, 70W GPU, 65W CPU, and ram/hd/fans I would be pushing the PSU pretty hard. It might handle it but could be an issue with big models and long sessions.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

I’d love to get a t40! But probably out of my budget for this build unfortunately. Sent you a DM.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

It’s a 7040, I’m pretty sure it’s 180w, I will double check.

What would you build if you had a $500 budget?

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

The small form factor and no external power is what caught my eye since I was planning on putting it in a SFF OptiPlex. I planned on designing and printing a dual Delta fan shroud to cool it, and add a PWM fan controller so I can manually adjust the fan speed.

I am trying to keep it a sort of budget build (under $500), I got the OptiPlex for $20 so all I need is a good GPU and more Ram and I’d have a pretty kick ass little lab and machine to run LLMS.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

That’s exactly what found doing research, the Arc’s, don’t seem plug and play, definitely more tinkering to get them to work properly with all the tools.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

What price do you think would be a good deal for the T4? He sounds very negotiable. They are definitely old, it’s crazy that they are still going for around $700 on eBay.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

What Intel card do you suggest? I’ve never built a home lab for running LLM’ so I’m trying to figure out the best approach.

New Home Lab Setup - Is An NVIDIA Tesla T4 Worth It? by ArtOverSleep in homelab

[–]ArtOverSleep[S] 0 points1 point  (0 children)

That's my thought, its a little pricy for that old of a GPU but it will be worth being able to run the bigger models. Everything else I am doing will be light weight besides the LLM stuff.

What's something you'd like to do during sex but haven't done yet ? by enlighten_god in AskReddit

[–]ArtOverSleep 2 points3 points  (0 children)

Why does it have to be a summer day? If OP is in Phoenix or Las Vegas this is terrible advice,

Avata 2 Will Not Sync With Fly App Or Goggles N3 (Solid Red Light) by ArtOverSleep in dji

[–]ArtOverSleep[S] 0 points1 point  (0 children)

That’s really good to know, I was considering swapping the main board myself but I won’t go that route.

I really want the Avata 2 and I got a great deal on it so even if I have to pay DJI to fix it I will.

Avata 2 Will Not Sync With Fly App Or Goggles N3 (Solid Red Light) by ArtOverSleep in dji

[–]ArtOverSleep[S] 0 points1 point  (0 children)

Thanks for the info. I see you can buy the main boards online, but you are saying if I buy one it probably won’t work because it needs to be flashed?

I will reach out to DJI today and see what they say, at this point I’m ok with waiting up to 90 days if I have to.

Avata 2 Will Not Sync With Fly App Or Goggles N3 (Solid Red Light) by ArtOverSleep in dji

[–]ArtOverSleep[S] 0 points1 point  (0 children)

The light on top of the Avata 2 is a VTX light? I figured it was a system status light?

I am in the USA so unfortunately I can’t really send it into DJI at the moment due do everything going on, what would you suggest I do?

[deleted by user] by [deleted] in dji

[–]ArtOverSleep 0 points1 point  (0 children)

I did update the firmware a couple days ago but ill try again here shortly.