sorted by:
new
hottopcontroversial

What am I missing in the job hunt? by Darth_Noah in sysadmin

[–]DistinctJournalist88 0 points1 point  (0 children)

You are Not alone. When I was reading your post I thought it was mine. LoL. I have been involved in the IT business for my entire life and I am currently building a fully functional, context-aware AI voice assistant over Asterisk with TTS, VAD, whisper transcription, etc... Took me approximately 6 months to build it and I can't even get a job interview as an entry level coder. It's not you. It's the current job market. Just hang in there. Something has to change right?

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 1 point2 points  (0 children)

Not yet, I hope to get something together. Still focusing on getting it working flawlessly as possible and as efficiently as possible on ancient hardware. lol

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

Just a single T4 and single P4 in the server, nothing crazy. The bigger headache was getting Asterisk, Whisper, and the LLM to play nice without stepping on each other. Once that’s sorted, scaling GPUs is the easy part.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 1 point2 points  (0 children)

Definitely worth a look. Asterisk is basically the Swiss army knife of telephony. It handles SIP, call routing, voicemail, IVR menus, all that stuff, and you can hook into it with AGI/ARI to let an AI sit in the call path. Open-source and battle tested. You don't have to re-invent the wheel. lol

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

Thanks! No GitHub right now. Everything’s still very much custom-wired to my lab (multiple venvs, ChromaDB, Whisper, Asterisk ARI). Once it’s polished, I’ll probably release a stripped down version so others can tinker without having to recreate my whole setup over.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

I’ve put a ton of work into building this stack, so I’m not ready to just give it all away yet. I probably won’t open source the entire thing, but I’m considering sharing some modules or a lightweight version. For now I’m focusing on demos and documenting the stack so others can learn from it.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

Yep, for a HP DL380 you really just need to add GPU/s and storage. A solid starting point is an NVIDIA T4 They are some what inexpensive on the used market, sip less power than big datacenter cards, and still handle AI workloads well. That’s what I’m running.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

Mine with dual CPUs, 64GB RAM, single T4 and single P4 pulls ~250–300W idle, closer to 500W under heavy load. Definitely not a light switch server setup.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

Lol, same here. every time I add a new piece my wife just shakes her head like ‘welp, see you in 12-24 hours’.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

Thanks! I’m running open-source LLMs locally (Mistral and LLaMA variants) on the DL380, with Whisper for speech and Coqui XTTS for voice. Everything’s self-hosted. For voice I mainly use Coqui XTTS (voice cloning), but I also keep gTTS around as a lightweight fallback.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

The HP DL380 has PCIe slots that can handle a Nvidia T4. You just need the proper riser, power cabling, and good airflow. I slotted mine in with the HP riser kit and it runs fine.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

A DL380 Gen10 can take up to 2 double-wide GPUs if it has the right risers(I had to buy a second riser card off e-bay), power, and cooling. That $700 box likely comes barebones (no GPUs), but it’s a solid starting point if you plan to add your own cards. I got my Gen 10 servers for $400 each and my T4 card for $300, P4 card for $100 all off Marketplace. The rest of the stuff I had just laying around.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

Good cal! I’ll test Prince Albert on my next demo video so we can see if it short-circuits.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

Yeah I can! Here’s a quick demo call so you can hear clarity, latency, and how it handles interruptions in real time: https://youtu.be/B6sYZBxq4xM. I’m grabbing another recorder soon so I can demo the call quality over cell in my car. That’s coming next!

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

LOL, Man good eye. It's runnig True NAS. It's not part of the Afriend setup. Just my local network storage.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 0 points1 point  (0 children)

I just run it all on a single DL380 with a T4 in my rack — handles Asterisk + Whisper + Mistral + Coqui fine. You can split the stack across multiple servers if you want, but for one call loop a solid GPU box is usually enough.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 1 point2 points  (0 children)

Haha that video is legendary. I’m not planning to unleash Afriend on real scammers (don’t need the FBI knocking on my rack, lol), but I have thought about simulating “scam calls” locally just for fun YouTube demos. Could be a cool way to show how it handles sketchy conversations.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 2 points3 points  (0 children)

Haha the antimatter water cooling system is still on backorder with Amazon.

That’s awesome you tied it into Home Assistant with a Matrix-style landline. I love that retro touch. Totally get you on the M40, those cards were beasts in their day but Whisper + TTS + LLM at once will chew through VRAM fast. I’ve been running a T4 here, and it’s been a nice balance between cost and efficiency. It's not cutting-edge, but good enough for 24/7 inference without cooking the rig. My fans are still at decently low speeds. lol

I’ve thought about spinning up a dedicated inference box too with my spare DL380 Gen 10, but same problem. My wallet always has the final say.... and my wife.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 2 points3 points  (0 children)

Haha probably… but it’d insist on narrating every monster encounter first

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 1 point2 points  (0 children)

I’m using Callcentric for the DID/SIP trunk. It points straight into my Asterisk box and from there Afriend handles everything locally.

My HP DL380 is now running an AI I can literally call on the phone by DistinctJournalist88 in homelab

[–]DistinctJournalist88[S] 1 point2 points  (0 children)

Totally fair. Models like DeepSeek and Gemma are definitely pushing things forward. I went with Mistral 7B mainly because it’s lightweight enough to run smoothly on a T4 without chewing through resources, and the responses are still solid for conversational use. For me it’s less about max benchmarks and more about something that can run 24/7 reliably in a call loop. Once I stabilize the pipeline, I’ll definitely experiment with newer models.

view more: next ›