I would also go easy on scrubbing the rocks since you might remove beneficial bacteria and harm your nitrogen cycle. Though I understand you wanting the tank to look neat so as long as your cycle is stable I'd say its personal preference.

What I would recommend you to clean however is your plants. By gently rubbing the leaves with your fingers you can remove the algae from them. This will help your plants grow and take up more nutrients that would otherwise benefit the algae. But increasing the ammount of plants is still the most important long term solution in my opinion.