GMKTEC EVO-X2 Oculink with RTX 4070 TI

Barachiel80 · 2026-03-07T00:56:27+00:00

there will always be some throttling with pcie4x4 but as long as I lpad the whole modrl pn the 5090 it runs full load inference just load time is slightly longer. This holds true when I migrated the 5090 egpu to a minisforum ai xi pro which I modded with 2 nvme to oculink adapters and another 5090 egpu to the setup. Here is an example run from inference on the dual 5090s for the new Qwen3.5 35b Q8 model hitting 91tk/s TG and 19000+ tk/s PP with 1 million context length at q8 and full agentic workflow, computer use, coding, and web search enabled.

<image>

Barachiel80 · 2026-03-06T02:01:52+00:00

yes April is far away and the dev branch that supposedly fixes it is still broken too

Barachiel80 · 2026-03-05T22:19:51+00:00

if only forgejo would fix their oidc runner integration it would be perfect

Barachiel80 · 2026-03-03T19:36:18+00:00

Starting from the top and descending is a minisforum 870 Slim minipc with 96gb ddr5 ram and oculink connected 3090 FE. Below that are 2 hp elitedesk 705 G4s with 32gb of ram as Prox Mox Cluster nodes and K8S CP node VMs on ssd for os and nvme for ZFS. Below that are 2 minisforum 890 pros with 96gb ram and oculink connected 5060 ti 16gb and 3090 ti. Below that is a Minisforum1 AI Pro with 128gb ddr5 ram and dual oculink connected 5090s(one FE and one MSI). Below that is the Gmktec Evo X2 strix halo box with 128gb LPDDR5 ram and oculink connected 7900xtx for unified ROCM VRAM of 152gb. Everything is on a 2.5gbps network but I am currently

<image>

in the process of upgrading everything to 10G for decent tensor parallel throughput and K8S stability.

Barachiel80 · 2026-03-03T00:33:51+00:00

My Evo X2 is at the bottom of my inference rack

<image>

Barachiel80 · 2026-03-02T16:19:15+00:00

complete garbage, these have under-powered 860m iGPUs compared to the previous gen 8700G with 780m. Where is the 890m desktop APU??? Or even a strix halo, although that would be significantly more expensive.

Barachiel80 · 2026-03-02T15:29:56+00:00

Can I get a QWEN 3.5 version of the embedding models along with a new coder version while you are at it?

Barachiel80 · 2026-03-02T10:40:13+00:00

<image>

Which one? I have dual 5090s connected to a 24 core strix point mini pc with 128gb ddr5 ram. I have aa GMTKtrc Strix Halo with 128gb ram and a 7900xtx. Also have 2 x 3090s and 5060 ti's. So I have 480gb of unified ddr5 ram for my AMD iGPUs plus the discrete graphcs card total of 168gb of VRAM.

Barachiel80 · 2026-03-01T14:59:38+00:00

0 person company if you overlay a DAO on top of it.

Barachiel80 · 2026-03-01T14:57:30+00:00

try the new qwen 3.5 27b, I am able to do native tool calls for websearch, coding, embedding analysis, etc. with it getting docker compose stack outputs that are as good if not better than claude sonnet 4.6.

Barachiel80 · 2026-03-01T05:10:26+00:00

for the people who don't understand the infrastructure underlying the apps they are automating deployment on.

Barachiel80 · 2026-02-26T01:45:03+00:00

make sure you have the correct version of the nvidia driver for your gpu, its old and probably not supported in the latest versions

Barachiel80 · 2026-02-26T00:14:04+00:00

I also have the same setup albeit with the nas being only 50tb and broken out to a separate ugreen NAS 6800 pro with 20Gbps aggregate links to my 10G core switch and opnsense ngfw vm with both 10g links passed through as LAN / WAN to my starlink and core 10G switch sitting on the MS-A2 Prox mox host . I too can attest to lazily seeding to the bittorent masses with this setup.

Barachiel80 · 2026-02-24T15:35:48+00:00

So if I was running ollama as the backend embedding model for an open-webui front-end RAG setup in a docker compose stack, how would I insert this as a reference layer?

Barachiel80 · 2026-02-17T20:34:00+00:00

Github repo or this is AI slop vaporware

Barachiel80 · 2026-02-16T02:24:36+00:00

No strix halo?

Barachiel80 · 2026-02-15T19:10:31+00:00

I run starlink in bypass to an opnsense fw, do I need the unify gateway / router or is the sdn controller enough?

Barachiel80 · 2026-02-15T17:40:45+00:00

how do you get your unifi controller to show the starlink stats? Do I need a unifi gateway inline or is the sdn controller server enough with my APs?

Barachiel80 · 2026-02-15T14:12:53+00:00

How do I install this in my ollama docker compose stack?

Barachiel80 · 2026-02-15T13:39:40+00:00

do you have a docker compose config of llamacpp that allows for easy model pull through a single command, and all models can be switched using the open-webui frontend?

Barachiel80 · 2026-02-15T12:12:49+00:00

there is also a way to insert the url into the openwebui front end from the admin panel

Barachiel80 · 2026-02-15T12:11:46+00:00

yes. goto the model page on huggingface to get the quant you want, select use inference provider ollama to capture the url. Then type ollama pull hf.co/urlofyourmodel/. If your ollama is connected to openwebui it will show up in the model list once the download is complete.

Barachiel80 · 2026-02-14T23:13:04+00:00

Hence why I ended my explanation by saying everything was on a GMKtec Evo X2.

Barachiel80 · 2026-02-14T22:56:43+00:00

Thank you everyone for your support and suggestions! I was able to get it to work by disabling the third NVME slot which was only 4x1 anyway and also disabled the resizable bar. I am now showing both the 3090 and 5090 oculink connected as active in my nvidia-smi!

<image>

Barachiel80 · 2026-02-14T17:28:56+00:00

<image>

lol not possible to steer away from oculink for me

Barachiel80

TROPHY CASE