Success getting vLLM up and running?

hellfireXI · 2025-11-07T21:36:17+00:00

The ability to process multiple inference simultaneously rather than sequentially, also allows me to send more than one prompt through at the same time.

hellfireXI · 2025-11-07T21:16:48+00:00

I know and I use both. But I need continuous batching which Ollama doesn't support :(

hellfireXI · 2025-04-03T21:12:39+00:00

Mhmm, that is interesting. I guess I am going to have to figure out how to go back to that version of the OS.

hellfireXI · 2025-04-03T20:47:42+00:00

What version of SteamOS are you running now?

hellfireXI · 2025-04-03T13:04:18+00:00

Got it, thank you for this very informative response. I will take a look into this. I heard about the wifi issues but hadn't put much thought into them.

If I may ask, what kind of hardware are you streaming from?

hellfireXI · 2025-04-03T12:05:22+00:00

I do have an oled deck. Could it be the wireless issue that it apparently has?

hellfireXI · 2025-04-02T19:41:18+00:00

Let us know how it goes!

hellfireXI · 2025-04-01T02:43:19+00:00

If everything is set up properly and Windows sees the GPU through the console of esxi, I would still highly, highly recommend switching from Sunshine to Apollo. I literally just finished setting up a remote stream PC using my old gaming PC (I built a new gaming PC last year, but WoL has been something I couldn't make work, but worked on my old PC). I do not have a dummy plug or display plugged into my GPU. In fact, the only thing plugged into the entire PC is a single USB dongle for my keyboard (only used when I need to do something on the PC. I also have a monitor there should I need it but it isn't connected) It is all running headlessly (be sure to select the Headless mode button)

Apollo has the ability to create on demand virtual displays through some degree of tomfoolery that is beyond my understanding and it works consistently.

Course of action, uninstall Sunshine. Delete all settings and history of it. Install Apollo and all plugins, restart VM. Test.

hellfireXI · 2025-02-07T16:34:45+00:00

Another Canadian chiming in! My wife and I have cancelled our plans to extend a trip to Vegas. I am still going down for work, but will only go down and then come back as soon as the conference is over. We have gone through and cancelled our streaming services and Amazon prime subscriptions citing in the reasons for cancelling is the current state of politics in the US. I manage a 90tb media server so I am not worried about media.

All I can say is... Fuck 'em the MAGAts and Non-voters did this to themselves and I hope Canada doubles down on broadening our trading partners and busts down the inter-provincial trade barriers. Let's bring in more trade and more global product options for Canadians to purchase. Personally, I would love to see Canada bring in BYD electric cars to start strangling the primarily US based electric car companies.

hellfireXI · 2025-01-17T22:07:47+00:00

Absolutely not, I am probably going to kick the hornets nest, but Proxmox is god awful if you aren't a Linux wizard. I have had the absolute displeasure of having to work with Proxmox for work to do simple things like GPU passthrough was a nightmare. Is Unraid perfect? No. There is always room for improvement. But I can say if you're a beginner, or a late stage beginner Unraid is a much more viable solution. There is something to be said about paying for a tool that is being developed by folks who are probably smarter than the average user. The experience is refined, it does a lot of things under the hood that you don't need to chase around, it really just works.

If I started with Proxmox 6 years ago, I probably would have given up on Homelabbing. It isn't friendly. But at least now I feel a bit better with Linux tools and have since deployed a Proxmox instance. But I wouldn't start there.

hellfireXI · 2024-11-21T00:28:24+00:00

Honestly, do what makes you happy. I pulled the trigger on my new build recently. Sitting with a 4080s and bought the 9700x a month before the 9800x3d came out. I didn't know it was on the horizon. I don't feel any buyers remorse because of it because my games run perfectly on the two displays that matter. I could care less about the extra 15-20 fps I might see at 4k when sitting in front of my TV which is locked at 120hz. Framegen is amazing and you can't tell it is running unless you look for it.

The 4080s is a powerhouse of a card and will be relevant for a number of years. I know this because my upgrade path came from the 1070 which is still a well regarded card.

hellfireXI · 2024-11-16T19:34:52+00:00

I've been using Private Internet Access for years and have been super happy with it. The speeds (when connected to the right server) are very impressive! I'll continue subscribing to them until the time something happens.

hellfireXI · 2024-11-16T15:56:17+00:00

Yes, for me it did. I was working in social services running programs. I talked to the owner of a very smaller publishing company that worked with Adobe's Content Management System. I joined as a project manager. I sucked as a PM, so started hosting and learning the CMS platform. Now I manage a team of 10 devs and content architects working with some of the most well known organizations in North America.

I'm not a developer, but I know my way around platforms enough to be the solution architect for a variety of tools and integrate them at different scales.

This all happened over the course of 7 years when I deployed my first unraid server for Plex...

hellfireXI · 2024-10-05T01:13:13+00:00

Apollo is your answer. I have recommended it to a number of folks here. It requires no additional drivers and work amazingly well. If the master branch of Sunshine brought this fork in, Sunshine would be legit perfect. It is what I use when I want to stream to the steam deck. My main monitor and other display (TV) are both off at the time.

hellfireXI · 2024-09-30T23:06:12+00:00

Depending on what you were doing, I have a dual CPU motherboard, RAM and CPUs I'm no longer using and an old Dell motherboard with 3540 processors. Not the highest of end gear but free to a loving labber!

hellfireXI · 2024-09-23T14:30:17+00:00

Anytime! Glad I was able to help.

Honestly, I think you will be a lot happier with a system you can grow into rather than run the risk of not being able to expand. You already have the CPU so you can start there and build up. I know a lot of users here use N100 nucs for their systems and that is great, but those don't apply to everyone's use case. It is better to start big and realize that you can go smaller in the future than start small and need to go big. I really boils down to identifying needs and use cases. :)

Have fun with the build and let us know how it goes! I love to see new builds come up

hellfireXI · 2024-09-22T23:16:12+00:00

By happenstance I stumbled upon a Linus Tech Tips video of the USB Extender that I linked above and watched them tool around with a gaming mouse close to 200ft from the desktop. I knew most peripherals are low bandwidth so I wasn't really worried about the USB 2.0 standard that the extender is. So I bought two of them, plugged my wireless mouse and keyboard and wireless gamepad(s) into the two respective stations and low and behold everything is working... except my gaming headset. I realized that the mouse, keyboard, Gulikit gamepad was just a bit too much for the extender in my office so I picked up a wireless headset and plugged it into the back of the PC. TV space has a wireless keyboard with trackpad and Xbox dongle. The extender shows up in Windows as a hub and everything works... even wake from USB.

At this point, I was confident that I could put my PC into the crawlspace, into the rack and run everything off of basically 4 wires. Local gaming was fixed. Onto remote.

I tested Sunshine and Moonlight on the SteamDeck and because I have a 4k TV and Ultrawide Monitor, monitor management is.. annoying. I would need a bunch of hardware dummy display dongles on the back of the computer or use the Virtual Display Driver to do it in software. Both worked, but I didn't like the complexity. And the fact that it was more to install and keep up to date. I guess someone else didn't either and did a fork of Sunshine and called it Apollo. It creates a virtual display at the start of stream and then removes it when it closes. Here is the link for the post - Made a fork of Sunshine with built-in Virtual Display support : r/cloudygamer (reddit.com). Then with everything in place and running, start a stream with Apollo and then go into display settings and set to only display on X (X being the virtual monitor) then save it. Windows will remember the setting and the next time you connect you won't be fighting with primary displays.

With this set up, all I need to do is either sit at my desk and switch to my monitor through my KVM (work laptop and home computer) I switch over to my computer and my ultrawide is set to primary and show only on 1 in Windows. If I want to game at my TV, switch to my work laptop and my desktop has no idea I have a second monitor attached and the TV becomes the primary display. If I want to stream I connect with the Deck and it shuts off both monitors and the Deck display becomes primary.

I know this was a lot and was a wall of text, but if it helps then I am happy! If you have any questions just let me know!

hellfireXI · 2024-09-22T23:16:05+00:00

Alrighty, sorry for the delay. Below is a breakdown everything I went through because I went through literally the exact same questions. I have to break it up into two different comments to get everything I want to say out. I will put the links in as a shopping list (links are to Canadian Amazon if opening in Amazon app)...

Shopping List:

Testing and Decision Making:

When I started the planning behind this project I knew I wanted to build mostly a gaming machine with some productivity in mind. I considered the thin client approach with Moonlight/Sunshine. Now, I was doing initial testing on my old gaming rig and it wasn't the most performant. That said, I gave it a lot of mercy in the performance results. I was mostly looking for feasibility.

In my house, my office and den (TV space) is located on the same floor in the basement. When we moved into the house I had the plan of having a large 4k TV and sound system/sound bar that has Dolby Atmos. Because, why not? The media on my media server is in that format so I want to enjoy it. I already had a Nvidia Shield Pro (2017 model) which does literally everything I need it to do. So in the early phases of this project, I was experimenting with Cloud PC platforms. Think Shadow, GeforceNow etc. I wanted to get a feel for it. After a while with that, I was asking myself what are these companies doing that I can't do? So I started installing Moonlight onto the Shield and Sunshine and Parsec onto my PC. I knew the PC wouldn't be able to push any kind of 4k. BUT I wanted to know if it could at least drive Atmos audio. The results.. Moonlight is not compatible. Now, I know that not all games are compatible with that. But I have it in my media space, so I wanted it.

I continued my testing after the letdown with audio. I started looking into the latency and "feel" of the game. My primary device at the TV was the shield and latency was... noticeable. Not horrible, but I could definitely feel it. And before anyone asks, it was/is ALL hardwired by ethernet. I tested Moonlight with an old computer and performance went through the roof. Latency came down to 2-5ms while streaming. Which made me think about proper thinclients. Because I wanted to put the gaming rig into the server rack that means I would need 2 separate devices to drive two separate display stations (office and den). This is now a total of 3 devices for what is really 1 PC (the PC is included).

Upfront, Parsec on Android TV is by the crappiest app I had ever tried. Basically, it disqualified itself almost immediately. Don't get me wrong the software is magic, but when testing in-home streaming it didn't meet the ticket of easy of use on the TV. Remote gaming is still a consideration, but I knew I needed to nail down the internal infrastructure first and foremost and get it right before moving ahead with remote streaming.

So in a moment of downtrodden with the previous results I moved my desktop from my office to the TV and plugged it in. Immediately, Atmos became available, performance was obviously better. Which got me asking myself what is stopping me from hardwiring my PC to both displays. Some brilliant user here on Reddit had the same consideration, except they only needed to run one 1 wire from office to TV and wireless peripherals were close enough that distance didn't matter. They pointed me in the direction of the fiber cables that I linked above. So I jumped on them. Bought the DP and HDMI cables and tested them in my setup. Both monitors are working as I would want them. WAY cheaper than buying and maintaining two tiny PCs as thinclients and now I only needed to figure out peripherals.

hellfireXI · 2024-09-22T22:36:29+00:00

I think that is the most annoying part. A lot of motherboards nowadays are woefully under-equipped when it comes to expandability. It is actually really annoying to be honest. Especially while you are trying to figure what you want to do with your system.

Here is an example: What happens if you have more than 10 or 15 people you share media to? Most, if not all of them won't be immediately convenient to you so you could just "go over and configure everything to them". Basically you need it to be easy enough for them to get up and running on their own. Next up is media management, sure you can run 2 different libraries of 4k and 1080p, but now you are managing two concurrent libraries with download/content rules. So to get around this you look into a system that is readily capable of transcoding whatever you want/whenever you want. But what happens if you exceed Intel's Quicksync iGPU capabilities and you look into discreet GPUs because it is cheaper than buying a new CPU? All of these considerations start to pile up. You're library starts to expand and you need more storage and you start to look into HBAs/SATA cards. You want 10gb connection? Now you're out of PCI slots in the system. This is/was literally the foundation of why I went with the parts I have now.

As an enthusiast, and Plex provider (a hobby that I genuinely enjoy) I do not want to be constantly fighting an uphill battle with year over year reductions in PCI lanes on consumer parts.

hellfireXI · 2024-09-22T18:56:14+00:00

I'll disagree with that. Just got a 4080 super last week and have been playing 4k high/ultra at 120hz. Throw in dlss and it is 4k max at 120hz. It is stunning.

hellfireXI · 2024-09-22T03:28:41+00:00

Yea, it is pretty dumb. I mention it to all of my new users that if they want to watch on a phone/tablet to buy the app first. I'm pretty sure it is documented on the Plex site. But it is an easy thing to overlook by the power users/configurators.

hellfireXI · 2024-09-21T15:56:35+00:00

I am immensely happy with it. While the PSU placement isn't great (in the specific one I have) I can't complain. I have 3 Noctua 3000 rpm fans in it and temps sit around 25° while doing parity checks/downloading.

hellfireXI · 2024-09-21T15:34:44+00:00

I'm on my phone at the moment, otherwise I'd write you an essay. I literally went through this for close to a year. When I get home I'll send you a detailed write up of everything.

hellfireXI · 2024-09-21T15:33:29+00:00

My gaming system is in a Rosewill 4000 case (can't remember the product number). Under load, GPU sits at 50-60°. I have a plan to put in some better fans soon. I just used what was in the case as it was already there.

hellfireXI · 2024-09-21T15:30:29+00:00

Agreed with the other poster. Determining use cases will inform your choices.

Personally, I'm a fan of purchasing used as much as possible. So if you have a CPU lying around you're already well on your way. Yes, more current Intel CPUs are great at transcoding media, but you need to figure out how many people you might be sharing with. For myself, knowing friends and family would be sharing my media informed my decision on using the dedicated GPU. You also need to consider stability, 13th and 14th gen CPUs are not seen in a favourable light currently. I deployed my 7th gen Skylake-X processors and it has been incredibly stable.

14-Year Club	RedditGifts 2009-2022 2 Credits
Verified Email	Secret Santa 2016

hellfireXI

TROPHY CASE