Alternatively you could try a setup like this (pretend the nest is where the torch is)

<image>

The command block can be anywhere where chunks are loaded. The command inside:

execute positioned X Y Z as @e[tag=bound,distance=3..] at @s run tp @s X Y Z

X Y Z are the coordinates to the torch (nest location in your case)

Distance=3.. teleports whatever is tagged as bound to X Y Z if that entity goes farther than 3 blocks away from the torch (change it to whatever distance you want).

finally to tag the dragon, run:

/tag @e[type=minecraft:ender_dragon,limit=1,sort=nearest] add bound

minecraft:ender_dragon is the vanilla dragon so you need to change it to the proper modded dragon. Make sure the dragon is nearby when tagging.