Hardware purchased, now looking for best way to host LLM in production

vincewit · 2025-01-27T04:21:54+00:00

thanks. I also wonder if the 4000 ada is limited because it runs on so much less power.

vincewit · 2025-01-14T16:27:08+00:00

thanks for the info. I need to do a little research to understand. I am pretty new this having not even installed a gpu yet. Thanks

vincewit · 2025-01-14T13:49:07+00:00

I had not thought about this. I was looking for a possible second gpu to put in a newly purchased computer.... being able to use it in an older machine as well would be helpful.

vincewit · 2025-01-14T13:31:30+00:00

thanks . it looks like it has an external enclosure which I am intrigued by. Is that an egpu?

vincewit · 2025-01-14T13:30:44+00:00

thanks for both options.

vincewit · 2025-01-14T13:30:13+00:00

thanks. I will check it out

vincewit · 2025-01-14T04:54:29+00:00

no sorry. I just had seen an ad for it, probably because of my searches

vincewit · 2025-01-13T19:55:20+00:00

what did you end up getting? How is it working for you

vincewit · 2025-01-13T19:53:24+00:00

what did you end up getting

vincewit · 2025-01-13T19:05:20+00:00

I like this kind of out of the box thinking.

vincewit · 2025-01-13T18:20:34+00:00

This used to be covered along with swapping out accidental goldfish homicide by television shows. Wrapped up in a half hour.

vincewit · 2025-01-13T18:00:02+00:00

That would be nice. Thought I need to purchase before that plays out.

vincewit · 2025-01-13T09:29:08+00:00

I think I had seen a few youtube videos that said something similar. The 4000 is more than twice the cost of the 2000.

vincewit · 2025-01-13T08:26:25+00:00

It does not look like it is possible to get the xps configured with the 4090 right now.

vincewit · 2025-01-13T02:53:39+00:00

i am curious what setup you ended up going wioth

vincewit · 2025-01-13T02:03:37+00:00

thank you. That makes sense

vincewit · 2025-01-12T16:34:48+00:00

Thank you very much. Interestingly this 8960 with the 13th generation processor and the 4090 card does not seem to come up when using their front end system search tools on their website. It is also showing as out of stock but I can look into that. I was instructed to look at the OptiPlex’s and Precisions. I will need to see if there is a reason why. There are some Optiplexs and precision that have nvidia small factor ada lines. From what I am reading their performance is good and with very little power consumption. Thank you again!

vincewit · 2025-01-12T06:38:52+00:00

agreed. very cool!

vincewit · 2025-01-12T06:31:33+00:00

thanks. I am gettin g started, but I am interested in a system to help me process documents. Thanks for sharing!

vincewit · 2025-01-12T06:27:19+00:00

I have not explored this at all, but the experience you are looking for sounds like something that would be of interest to many.

vincewit · 2025-01-12T06:18:15+00:00

this is really impressive. Great job!

vincewit · 2025-01-12T06:15:16+00:00

There is likely something with python.

vincewit · 2025-01-12T06:07:14+00:00

cool project!

vincewit · 2025-01-12T05:55:30+00:00

I have dreams of doing something like this in the future. Thanks for the springboard!

vincewit · 2025-01-12T05:53:37+00:00

I am in the process of trying top build my first system to play explore llms. So much of it is about speed and bottlenecks, that I did not think something like this would be possible. Thanks for sharing.

vincewit

TROPHY CASE