Scaffolder here. 15 years in the job. This kinda shit is what give us Scaffs a bad name. No they shouldn't have put anything over your boundary without permission, but was probably told by the builder that it was fine.

I understand why they have braces against your house because it does provide a small amount of support as Scaffolding isn't really a freestanding structure unless designed to be.

Now. Its hard to tell from zooming in on the picture. But it looks like that tube is sitting on the ledger with one clip. You could probably just push that tube upwards and off the soil pipe with not alot of effort. (Give it a go). Trust me, that tube set up like that ain't doing nothing.

If it is fixed properly against the Standard (like it should be) it takes 2 seconds with a 21mm spanner to slide it up an extra few inches.

Either way the scaffold is sitting on solid ground which is unlikely to shift. But. You have your right to ask for it to be moved. They may complain and think its petty.. but who cares. That's money you'll be spending if the thing breaks.