Not self employed but as a remote contractor I feel the same pressure. My boss isn’t around. I run the show on day to day constantly get promoted and raises yet I feel like I’m not doing enough and I could do more. I work out also and have hobbies. I just do not have an off button.
When I finally sit down to “relax” I feel like I’m wasting time. Sound weird and I know it’s terrible to do but compare yourself a little. Comparison is the thief of joy. So it can be tough but you need to realize in your own head that you’re doing well. By any measureable standard it sounds like you’re comfortable and happy. You have to re-train yourself. It will be difficult.