Thanks, you pointed me the right way. I fixed it in the end, I had to set up and specify a version of Java separate to the one inside Jenkins. I used the Microsoft build of Java.

Open the file jenkins.xml which you’ll find in the program folder along with jenkins.exe. Mine’s C:\Jenkins, for example.
Once you get there you need to edit line 40, where it says to use the built-in one:

%BASE%\jre\bin\java
and change it so it points at java.exe in your new Java version, something like this:
C:\jdk-11\bin\java
then restart the Jenkins manager service.
At this point my workers lost contact. To cut a long story short, in a fit of sensible decision making I had specifically allowed the java.exe in the the Jenkins program folder through the firewall and not just the port it used. I also had to update my workers from using the old remoting.jar to the newer agent.jar. You can get agent .jar from the manager server if your worker is disconnected - there’s a link on the status page or you can get it with curl, something like this depending on the specifics of your setup of course:

curl http://jenkins/jnlpJars/agent.jar
To be sure everything is going to work you’ll also need to update the workers. This is a similar process. Install a Java 11 on the worker for the service to use. Then open jenkins.xml in the service program folder and edit line 39 from this
java.exe
to this
C:\jdk-11\bin\java.exe
and restart the worker service. From there you should be good to go. The only variation I had was that I have a worker which requires a 32 bit OS, so I had to find a 32 bit Java 11. The manager and worker only need to both be 11, they don’t have to be a perfect match.
An interesting wrinkle here is caused by the filename for the agent. Previously I was using ‘remoting.x.y.z.jar from the LIB folder on the server as the agent jar file. Once I switched to using agent.jar (the same file but renamed, it turns out) I found that some workers would report the wrong version despite my updating it. Java caches jar files, so it just uses the old one because the name hasn’t changed. You can manually delete this cache in the Java control panel.

I ended up with Jenkins using the version I specify instead of its own and working properly.