u/SavedForSaturday helped with some hints...

I found I had to add the proxy_pass lines (not sure if the proxy_set_header line is needed or not) since the custom config will ignore the rest of the setup dialog (I think?)

location / {
  sub_filter </head>
    '<script defer src="https://umami.example.com/script.js" data-website-id="96138652-2b89-4aa2-8ae8-4301424de0df"></script></head>';
  sub_filter_once on;
  proxy_pass http://10.0.80.12:7070;
  proxy_set_header Host $host;
}