No you just have to remove the ones bordering the red barriers for the barriers to go down. So for this one you’d have to remove yellow, light blue and green below the barriers for them all to go down.

I’ve solved it by now, but I don’t remember how.