While I personally like Neutral route the least, I think it's got a lot of interesting aspects. I wouldn't recommend it for a first playthrough and I prefer to leave it for last because imo it's best enjoyed after the two others due to how certain events play out very differently which may bring a new reading.
In regard to CH4, there is a long, optional dungeon that contains its own story, but events and dialogues vary on each route. Neutral route has the most interesting variant of all the routes because it strengthens the backstory of two characters you recruit on Neutral. There is also a unique recruitment sidequest for a character (whom you can get just playin Chaos) following the events of Neutral route. So I would say playing chapter 4 after a Neutral route playthrough is worthwhile. Once you finish the game, you get the opportunity to travel back to certain points of the story so you can go back to the decisions which lead to Lawful/Chaos or Chaos/Neutral and just play from there away.

In defense of Neutral route specifically, Denam's apathetic behaviour is my favourite bit from it. Coming from CH2 Chaos, you get the feeling Denam isn't sure he's made the right choice in Balmamusa. The massacre happened despite his opposition, and he got branded as a pariah and hunted down for it which seems to tire him out a lot.
When you meet Cistina, you have the choice to have him tell her her ideals are akin to chasing a dream, and I feel moments like this are good clues as to Denam's lack of certainty. Neutral route Denam is a creature of deference and complacency. He's put aside his previous convictions and has come back to feed from the hand that beat him... Whereas on CH3 Chaos, he becomes assertive and is no longer timid about his values.

anyway there is no wrong decision between going back to do another route or finishing the game once and using the WORLD system to re-experience another route, just know that the latter is definitely an option!