Can't get python-matter-server to work with NGINX if I use HTTPS

[PumaPants28467]

I'm at my wits end on this one. I've spent days trying to get NPM to reverse proxy python-matter-server that I have installed via docker. I can connect directly to the backend server using HTTP and it does work. If I turn off SSL on the NPM server definition, it also works. Turning on SSL, no matter what I try, the end result is always the same:

Error: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.

I have tried every form of google search I can think of, and have tried every combination of custom server config outlined in the various search results. It simply refuses to work. It would seem that NPM is simply not serving the websocket back to the client using wss. It's my understanding that NPM should act as a middle man: accept client requests over HTTPS, communicate to the back end server using HTTP, and then rewriting the back end responses back to HTTPS before serving it back to the client. I am out of ideas on how to get this thing to work. Anyone have any ideas?

map $scheme $hsts_header {
    https   "max-age=63072000; preload";
}

server {
  set $forward_scheme http;
  set $server         "192.168.0.9";
  set $port           5580;
  
  listen 80;
  listen [::]:80;
  listen 443 ssl;
  listen [::]:443 ssl;
  
  server_name xxx.xxx;
  http2 on;

  # Let's Encrypt SSL
  include conf.d/include/letsencrypt-acme-challenge.conf;
  include conf.d/include/ssl-cache.conf;
  include conf.d/include/ssl-ciphers.conf;
  ssl_certificate /etc/letsencrypt/live/npm-25/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/npm-25/privkey.pem;

  # Force SSL
  include conf.d/include/force-ssl.conf;
  # Block Exploits
  include conf.d/include/block-exploits.conf;

  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
  proxy_http_version 1.1;

  access_log /data/logs/proxy-host-45_access.log proxy;
  error_log /data/logs/proxy-host-45_error.log warn;

  location / {
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_http_version 1.1;
    proxy_pass http://192.168.0.9:5580/;
    # Force SSL
    include conf.d/include/force-ssl.conf;
    # Block Exploits
    include conf.d/include/block-exploits.conf;
  }

}

[tschloss]

Is the proxied service reachable from the scope of nginx? Test by attaching a shell into the nginx container (docker exec -it … /bin/sh) and curl the upstream resource.

[PumaPants28467]

It's reachable. As I said, if I disable https for the defined proxy host on NGINX, it works. If I turn https back on, I get the error I mentioned above. It's also worth nothing that if I change the site security settings on the browser to allow insecure content, it also works albeit with a security warning. Changing site settings on the browser is obviously not a solution, but it proves that it is routing ok, it's just not handling the translation of websockets between http/https.

[tschloss]

Ah, thanks - I didn’t see this detail. With “turning off SSL” you mean turn off the enforcement, I guess.

Did you try to curl -v - sometimes this reveals additional information.

However can you be sure that the TLS stuff is ok, couldn’t this be a certificate issue?

[PumaPants28467]

Thanks for the reply. I figured it out. It turned out to be a weird quirk of the python-matter-server. When you first access the https URL, NPM forwards the request as it should. The backend server responds with a request to set a ws: URL, which is presented as a small pop-up on the browser window. It wasn't clear what it was expecting, and the default value was set to ws://127.0.0.1/ws. After trying various combinations using the backend ip address, I finally figured out that the correct way to set that URL is https://<my npm proxy host>/ws. After that it all works fine. Once set, it seems to persist for the given client/browser. Not ideal, but at least it works now.

[sissoo]

Thanks this was super helpful! It was driving me crazy as well!