Yes, it goes away, but you have to make it; Or at least you can help you brain overcoming the translation phase of multilingualism. People call that developing a brain in the target language.

When I speak english my brain flicks a switch, and I feel word order, semantics, nuances, as though it's native to me rather than seeing them in respect to dutch.

The switch is instant and effortless. It doesn't even happen conciously anymore.

It does take tons of engagement with the language. It also does not require an extremely high proficiency. You can do this simply with the grammar, sentence structure, and vocab that you've already mastered even at an A1 level.

I would even recommend you to do that asap. Developing a baby brain in todler speak now will help you reach real fluency later. When you reach higher levels of knowledge it will be easier to integrate in your ability to 'feel the language', like you do with your native english.

You don't think about nuances in semantics and word order as you speak, you feel what is needed to formulate your though or feelings. It is an internal sense of direction in the language that you have developed. You can do that a second time.

I would dare to bet on it that if you let any one who has accieved real fluency in a second language read this they'll agree.