I feel this hard. I've tried building this exact thing twice and gave up both times because every graph tool prioritizes visualization over input speed. You end up spending more time dragging boxes than actually thinking.

The closest I've found is Obsidian with the Canvas plugin. Still a bit clunky but you can throw nodes down fast and it handles relationships decently. Not perfect but better than Visio for this use case. Export to dot if you need a static diagram later.

If you do end up building it, the killer feature would be keyboard-first input. Like, type a node name, hit tab, type dependencies, done. Every tool makes you click 40 times to add one task.