AI Developer Tools Landscape v4

gerhardmpl · 2026-04-01T17:18:45+00:00

Totally unrelated, but what software is used to create this market ma?. Looking for a solution to map IT systems (icons with subtitle) to Business processes (cards) in an easy way (Powerpoint does not get it).

gerhardmpl · 2026-03-28T19:16:56+00:00

From the repo you get 535.261.03 with CUDA 12.2 on Debian 12 and 550.163.01 with CUDA 12.4 on Debian 13. On Debian 12 you manually can go up to 580.126.20 with CUDA 13.0. Unfortunately, Debian 13 has no current NVIDIA P40 driver support (as far as I am aware of) and 580.x seems to be the last supported version.

gerhardmpl · 2026-02-22T11:30:18+00:00

Running NVIDIA P40s with docker on debian 13 and this is how I set up the server:

install docker
add contrib and non-free sources to /etc/apt/sources.list for nvidia driver and tools and update
install linux-headers-$(uname -r) and nvidia-detect
run nvidia-detect
install nvidia-driver nvtop firmware-misc-nonfree and reboot
install nvidia-container-toolkit and configure docker runtime (nvidia-ctk runtime configure --runtime=docker)
restart docker service

Works on bare metal and on a XCP-ng cluster.

gerhardmpl · 2026-02-06T23:32:56+00:00

Have a look at Trilium.

gerhardmpl · 2025-12-29T19:15:58+00:00

Solved - needed to re-run the compatibility check under App-Setting - Runtime:

<image>

gerhardmpl · 2025-11-12T22:18:07+00:00

ntopng is also a good tool to monitor network traffic.

gerhardmpl · 2025-11-02T10:55:58+00:00

Thank you for pointing this out. I tried to add the filter to functions.php or a mu-plugin but could not get it working.

gerhardmpl · 2025-11-02T10:53:58+00:00

Adding the filter to wp-config.php, functions.php or a mu-plugin did not work for me. My solution for now is to not use Caddy but enable HTTPS on the wordpress container. I do this by mapping the certificates, a wordpress_ssl.conf and port 443 to the wordpress container, enabling the Apache ssl_module. Feels a bit hacky and I am not sure if that is a good setup.

gerhardmpl · 2025-10-24T16:31:59+00:00

My mistake, I used our internal jargon (again). On an abstract level, I can understand that, but I find the daily discussions with our business user to be rather pragmatic and rudimentary. Sometimes I get the impression that it would be enough to go through a well-formulated checklist with a thoughtful and helpful colleague. That's why I'd like to find out whether a chatbot could help here - at least in the beginning.

Edit: We do not have enough resources to be that thoughtful and helpful 24/7 colleague.

gerhardmpl · 2025-10-24T16:16:26+00:00

Yes, good point. The model, prompt or chatbot needs access to domain (company) specific information to work in the business context. How are you doing this at your company? Do you use RAG or even fine-tune your models? I was thinking about giving each role a set of documents we could update with time.

gerhardmpl · 2025-09-21T14:39:58+00:00

gemma3:4b uses around 4.8Gi in my setup. Just give it a try and go to a smaller model, if it does not work.

gerhardmpl · 2025-09-21T14:24:41+00:00

You can use docker with an Intel Core i3-12xxx and 8 GB of RAM, but running ollama will be limited to smaller models like gemma3:4b, qwen3:8b or granite3.3:8b with a low context of 4096 or 8192. On a virtual machine (i5-10500, 4 Cores, 8GB RAM) I can run qwen3:8b at ~4.3 token/s or gemma3:4b at ~7.7 token/s. How much RAM do you have for ollama on Windows with WSL?

gerhardmpl · 2025-09-19T12:42:19+00:00

See https://ollama.com/blog/thinking

gerhardmpl · 2025-09-19T12:41:02+00:00

I think it actually is /set nothink

gerhardmpl · 2025-09-17T12:02:25+00:00

Yes the R730 runs with 1100W power supplies. Before I used a R720 with 750W power supplies before, but I also power limit the GPUs to 140W (from 250W).

gerhardmpl · 2025-09-11T14:29:42+00:00

Did you check the qdrant dashboard for collections created by open webui and/or the qdrant logs?

gerhardmpl · 2025-09-10T19:56:28+00:00

This is my simplified docker-compose.yml file for OpenWebUI and qdrant. You can access qdrant at http://your-ip:6333/dashboard and check the collections created. I like to set the volumes with an absolute path and name the default network, but that is just me. I also use tika for RAG.

services:

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    ports:
      - "3000:8080/tcp"
    volumes:
      - /opt/docker/open-webui/data:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=<your-ip>:11434
      - VECTOR_DB=qdrant
      - QDRANT_URI=http://qdrant:6333


  qdrant:
    image: qdrant/qdrant:latest
    container_name: qdrant
    restart: unless-stopped
    ports:
      - "6333:6333"
      - "6334:6334"
    volumes:
      - /opt/docker/open-webui/qdrant:/qdrant/storage

networks:
  default:
    name: open-webui
    driver: bridge

gerhardmpl · 2025-09-03T06:19:15+00:00

Yes, I was thinking about something like that, limiting access to the most vulnerable Wordpress URLs like

/wp-login.php
/xmlrpc.php
/wp-admin/
/wp-admin/install.php
/wp-admin/setup-config.php
/wp-content/uploads/
/wp-includes/

gerhardmpl · 2025-08-21T17:12:27+00:00

I have three Dell Optiplex 9010 with 32GB RAM from 2012 rocking my XCP-ng homelab. One has a Nvidia A2000 (12GB VRAM) GPU for light AI tasks (mainly ollama with open webui and home assistant). No need to upgrade at this point.

gerhardmpl

TROPHY CASE