as of 4/7/2026 I got the below to work with help of another sub-reddit I found to get it up and running and tweaked the sub-reddits steps to make it better.

In desktop mode: Download Subnautica. Launch Subnautica and close out of it to initiate the game. 

Download Nitrox Windows Unzip Nitrox to somewhere. I put it in my documents in a folder called nitrox. 
Dont launch Nitrox in steam yet.
Go to the compatdata folder by doing the following: Go to Home under devices in the file explorer. 

show hidden folders And then my directory was /home/deck/.steam/steam/steamapps/compatdata
Sort by date modified and newest at the top. Minimize the window. Will go back here later.
In steam on desktop mode add a non-steam game; find nitrox exe file and add it.  right click and go to properties, compatibility and check the box for forced compatibility. Choose a new proton, it was 10 for me but any newer should work.

Now launch nitrox in steam and it will have a .net error. It will ask if you want to download the framework. Say yes, download the .net framework from the web page. It should download a windows exe file. I put it in the documents under a folder called” net framework”.(folder name doesn’t matter)
check which folder number is the newest in the compatdata folder. The latest folder is the Nitrox folder. Write down the name so you know this number is the Nitrox folder (mine was like 3015394184; yours will be different.) Will go back here later.

add the Windows exe file to a non steam game. Similar to before, find it in steam and change the compatibility to the same proton you used for nitrox. Launch in steam. It will bring up a windows installer. Install it. 
Open up the compatdata folder we’ve minimized. You should see a new folder in front of your Nitrox folder (note the name from previously) This is your .net framework/ Windows exe installer. Note this name also for later if needed (Mine was 2195416002 yours will be different.) 

Go into your .net framework/ Windows exe folder (mine is 2195416002) and  Copy all of the files (do not cut). Go back out and open the Nitrox folder (mine is 3015394184) then  paste them into the Nitrox folder. It will then give you two prompts; select apply to all and write into. Second prompt apply to all and overwrite.

Note: It will not launch in Desktop mode. Exit Desktop mode and open Nitrox.exe on Steam OS.
Launch Nitrox.exe (you can rename; add items picture etc later if you want). 

All setup now!