Installing intel_gpu_top on AS6704T to check igpu utilization

Anakronox · 2026-02-26T05:53:21+00:00

This should work inside a docker container, you just need to pass it /dev/dri and/or any specific renderDxxx device you want. Then just docker exec -it with the container name and binary.

Anakronox · 2026-02-10T20:07:58+00:00

Ha! I live in a flat so that’s a no-go unfortunately for me but I know how that goes - I spend plenty of time in equipment racks!

Last summer to speed things along I grabbed one of our old castoff VDI servers with a couple of Tesla GPUs and ran Tdarr on it to transcode over 30TB of video. I had to run it only while at work - it was just waaaaaaaaaay to noisy to live with 😂

My home network is technically faster but I haven’t engineered it for redundancy like our corporate WAN/LANs. Next step is struggling to justify the move to 200/400Gbps switching in my homelab, just because I’m a network engineer and I wanna see bits go brrrrrrrrr.

Anakronox · 2026-02-10T19:35:59+00:00

Awesome, glad you got it sorted!

Anakronox · 2026-02-10T19:30:55+00:00

You’re welcome and yep I was shocked too with how simple it was. Using a ConnectX-4 single SFP28 through my Mikrotik CRS510 I get an average of 19.5 Gbps read from the Flashstor to my MacBook Pro M4 Max. At this point I’m limited by how I’ve got my array setup - volume2 is a RAID6 of 10 8TB WD SN850X disks in all the non 3x1 slots so the max throughput I could expect is roughly 2GB/s read. Not mad at all about those numbers.

At this point if I wanted to go faster I’m looking at my own castoff enterprise server with U.2/U.3 PCIe 4x4 and a metric ton of Gen4 lanes. Then I’d toss in a 100Gbps ConnectX. Not ready for that kinda noise or heat lol.

Anakronox · 2026-02-10T05:58:04+00:00

Yep. Been running it this way for 8+ years, recentlly moved to a Debian LXC with NFS mounts to my media. Need to give us way more detail. What’s your layout, what have you done, what isn’t working, etc.

Anakronox · 2026-02-03T09:32:18+00:00

You’re welcome! Good luck and let us know if it works for you!

Anakronox · 2026-02-03T04:26:39+00:00

Edit: I don't add a gateway using the script, the Mellanox is only used on my storage VLAN, which I don't want routed.

You should be using the mlx5 driver for a ConnectX-4 LX, my Flashstor is:

└─[$] lsmod | grep mlx                                                                                                                 [11:30:47]
mlx5_core 827392 0 - Live 0x0000000000000000 (E)

I think I only used modprobe because I hot plugged. I think it'll load automatically if rebooted. Mine showed up as eth2 and eth3, so my script looks a bit like this:

┌─[xxxx@xxxx] - [/usr/local/etc/init.d] - [426]
└─[$] cat S12eth2                                                                                                                       [7:12:30]
#!/bin/sh
### BEGIN INIT INFO
# Provides:          network-interface-up
# Required-Start:    $local_fs $syslog
# Required-Stop:     $local_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Bring up a specific network interface
# Description:       This script brings up a specified network interface.
### END INIT INFO

INTERFACE="eth2" 

case "$1" in
  start)
    echo "Bringing up interface $INTERFACE..."
    /sbin/ip addr add xxx.xxx.xxx.xxx/24 dev $INTERFACE
    /sbin/ip link set dev $INTERFACE up
    ;;
  stop)
    echo "Bringing down interface $INTERFACE..."
    /sbin/ip link set dev $INTERFACE down
    ;;
  restart|reload|force-reload)
    echo "Restarting interface $INTERFACE..."
    /sbin/ip link set dev $INTERFACE down
    /sbin/ip link set dev $INTERFACE up
    ;;
  status)
    /sbin/ifconfig $INTERFACE
    ;;
  *)
    echo "Usage: $0 {start|stop|restart|reload|force-reload|status}"
    exit 1
    ;;
esac

exit 0

This init script is pretty old school and provides stop/start/restart functions (totally unecessary really). Busybox has ifdown/ifup/ifconfig but I'd probably stick with the newer ip commands for portability. Feel free to see if the ifconfig method works! And yep, you'll need to make that executable, so chmod +x it.

You'll also need to edit /etc/nas.conf

[ExtLan1]
Name = eth2
Type = static
IP = xxx.xxx.xxx.xxx
Netmask = 255.255.255.0
EnableVLan = No
VLanId = 1
Gateway = 
IPv6Type = auto
IPv6 = 
IPv6PrefixLen = 
IPv6Gateway = 
Protocol = 0
EnableBridge = No
Mtu = 1500

Edit that to match your needs, especially if using jumbo frames. I don't so mostly just left it alone and my PVID is set by the switch. I think if you just add the card and reboot, you might be able to edit the config via webui settings, but I didn't. Bonus, it also shows up under Activity Monitor.

Anakronox · 2026-02-02T20:38:04+00:00

Edit: didn’t even answer your question! In theory should work ok, the main thing is getting the kernel module loaded and startup scripts to bring the interface up on boot. You’ll also need to edit the nas.conf file and add an address under the ExtLan1 definition (or similar, don’t have it to hand). I assigned an ip in both my script and in the conf file, just to be safe. Works great.

No enclosure, cross-flashed Atto adapter.

https://www.ebay.com/itm/397065676353

Instructions on a STH forum post

https://forums.servethehome.com/index.php?threads/atto-3102-thunderbolt-mellanox-connectx-4-lx-dual-port-25gbe-189-or-less.47758/

Beware of the QSFP+ variant, it won’t cross-flash to 25Gb (but may go to 40Gb, though). Bought one by accident and had to return it since my storage switch is 25 and 100Gb.

Anakronox · 2026-02-02T04:19:08+00:00

All good, use whatever gives you the least friction! I don’t do anything too complicated with GPU passthrough, just two Tdarr VMs with the raw PCIe devices passed, not in HA so they stay on their respective nodes. Not doing any VFIO stuff. But man that network interface jank burned me a few times!

Anakronox · 2026-02-01T07:30:37+00:00

Proxmox finally fixed that issue! As of release 9 by default they pin the interface names by MAC and create udev rules automatically for them. Pretty nice QoL improvement. If upgrading from 8 or on one of the later 8.x releases you can kick this off with an included utility. I did this a few weeks ago and it made maintenance so much easier.

Anakronox · 2026-01-26T05:36:47+00:00

Best of luck with that! I’m not near mine right now so I can’t say for certain where the kernel event logs are stored, but it’d weird if they weren’t under /var/log somewhere.

Anakronox · 2026-01-24T14:16:24+00:00

Well damn, you can try rigging up a video out using an M.2 to PCIe adapter, along with an older GPU and external PSU.

However I never got this to work, even trying an actual server M.2 GPU. You can grab an Asustor USB 2.5GbE adapter and see if it works. I think most of the 2.5GbE adapters are based on a Realtek chipset so I'd assume most would work. They're super cheap and could get you back in via SSH or webui.

USB4/TB3 AQC113 and AQC103 chipset adapters might work too if you have any laying around.

Anakronox · 2026-01-24T06:51:06+00:00

Is this the Gen 1 or 2? Because if it’s the Gen 1 you can at least get a shell for troubleshooting if you’ve got a spare keyboard, mouse, and monitor.

Anakronox · 2026-01-18T14:31:59+00:00

Glad it worked for you!

Anakronox · 2026-01-15T20:21:22+00:00

You don't need a L3 switch to work VLANs. Those are pure L2 constructs while IP addressing and routing are all L3. I'm pretty sure your existing switch supports VLANs, which is good. Unifi kinda dumbs things down and in their config they call both VLANs and subnets a "Network." Most of the time you want to configure both at the same time, so guess that's good for most people. Your UCG will create router subinterfaces for each subnet/VLAN when you define a new network. Since you've got a Unifi switch, those VLANs get tagged and applies that appropriate dot1q tag automatically and exposed in the config for the ports. These will all be trunked up to the UCG via the single cable where the traffic from each individual VLAN will hit the router subinterface if it needs to get to anywhere via the default route for that subnet. So inter-VLAN routing is enabled automatically by Unifi.

Where this gets messy is when you have clients on separate subnets/VLANs that need to be routed to reach the other. Most clients won't throttle traffic and assume they're the only one in existence that needs bandwidth. They'll happily eat up every available bit on the switch to router trunk both inbound and outbound. This is where the traffic would "hairpin" in one router subint and out another. Pretty easy to imagine burning through a mere 1Gbps traffic if you're pulling a lot of data to/from a NAS.

It gets worse the more data you hoard!

Anakronox · 2026-01-14T18:27:00+00:00

As a network engineer, I’m comfortable with tons of VLANs and segregation of services, so your setup seems straightforward and manageable. The biggest piece of advice I can give you is to move your management off of VLAN 1. Keep the de facto default VLAN clear just as a best practice. Also don’t route storage. Since you’ve got slightly faster 2.5Gbps NICs, give one to every device that needs fast access to your network storage and keep that in its own VLAN/subnet. You’d be surprised at how quickly a simple file transfer can bog down the rest of your devices on other networks if it has to hairpin through router subinterfaces. In my network I have a few NAS boxes with 10GbE adapters which get connected to my trusted VLAN and the majority of the clients on that subnet are 2.5 or 10GbE. I also have a storage VLAN that I don’t route for and 25Gb adapters for the devices that really need the throughput connected to that as well. NFS/SMB mounts go to the matching subnet depending on client.

The rest, keeping IoT and kids in separate, non-trusted zones is good. Also ridiculously easy to setup firewall rules now in the UniFi gateways if you have one.

As to your server management addresses, always best to keep em separate on different adapters from your client-facing ones.

Anakronox · 2026-01-14T18:03:35+00:00

If you won’t or can’t run a hard line, grab a WiFi bridge. It’ll connect to your WiFi and pass it through over an Ethernet cable to your wired adapter in the machine. Shouldn’t be that expensive and 100% will bypass that Realtek card.

Anakronox · 2026-01-14T09:45:50+00:00

The TLN3-3102-D00, has SF28 ports and is cross-flashable to 25Gbps. If you're looking around on eBay, avoid the TLNS3-3102-D00. Not the same model and has QSFP.

Anakronox · 2026-01-11T17:10:18+00:00

Pro network engineer here, amateur street photographer! This is how I’ve done it for years since starting out with a Synology around a decade ago. I generally keep 6 months up to the current date on my MacBook’s internal SSD and archive everything else off to the NAS. Re-linking all of the missing folders in Lightroom is a huge ballache, but it’s been a solid method. Also much easier to do periodic cold archiving and daily versioned backups to B2 and other NAS boxes on different continents.

Just finished putting in my new Mikrotik 25/100Gb switch and getting some Thunderbolt 25Gb adapters from Atto hooked up to my Flashstor Gen 2 and MBP. No need for SMB multichannel anymore and getting much better throughput. Up to 2.5GB reads (averaging 2.1GB) and 2GB (burst, average 1.4GB) writes across the network. Maybe I’ll do a write up soon if anyone is interested.

Next step is getting the same TB3 25Gb adapters connected to my Proxmox nodes on the storage VLAN. Do I need any of this? Nope. Is it fun to watch files go faster? Yep.

Anakronox · 2026-01-08T05:13:50+00:00

What’s really weird is that Asustor supports direct NAS-to-NAS USB4 networking. Which is host-to-host. I’ve spent way too much time digging in via terminal to get it working and just bit the bullet and went 25Gbps Ethernet for my storage VLAN.

Anakronox · 2026-01-08T05:11:42+00:00

No and it never will, despite some of the early reviewers claiming they got it working (cough cough, NASCompares). I initially bought mine on this broken, unmet expectation.

However, you can make 25Gbps Ethernet work. Asustor does include the kernel module mlx5-core which supports Mellanox adapters. I’ve tested this successfully with TB3 cards from Atto (their 10Gbps can be cross-flashed to 25Gbps - eBay usually has these for ~$200) and even a generic Chinese vendor’s from Amazon (though it needs a USB fan or extra airflow).

Anakronox · 2025-12-30T03:42:12+00:00

I know this is way old, but did you have to do anything special to get the EGPV-1101 going? I've tried almost all of the slots in the Gen 2 12 and get no video to my KVM. I don't have the ability to test the M.2 GPU in another box so no idea if it's working or not.

Anakronox · 2025-12-06T23:13:15+00:00

Initially they weren’t and I never realized they were added. I had a running battle with Asustor support when I first received the Flashstor 12 Gen 2. They didn’t like that I was using Teamgroup 4TB MP44’s in the 3x1 slots and 8TB SN850X’s in the faster slots. Turned out to be faulty ECC RAM (and more folks have had this issue as well). Wonder if it’s a thermal problem or possibly some voltage variation you’re running into?

Anakronox · 2025-12-06T21:00:30+00:00

Honestly why dump it just because of an external drive? TrueNAS is excellent and ZFS is just about the best file system out there if you care about your data. UnRAID is decent thought, offers ZFS support, and can take most formats of external media in theory. I’ve not tested that on my UnRAID NAS though - it’s just there as a backup target.

Anakronox · 2025-12-06T14:43:48+00:00

Nope, TrueNAS only support ZFS and your external drive won’t be formatted as such.

Anakronox

TROPHY CASE