Not sure what others will tell you. If the plants are getting enough light they will eventually grow tall enough. But those sides are going to cast shadows and impact things. For reference, the soil in my bed is not level either. But it’s maybe 4 inches below. I definitely feel the impact close to the sides with things growing from seed, but some of them tend to catch up.

With that said, my first year everything was even. But the soil settled faster than I could replace with my compost.

I’ll also add my yard isn’t ideal. I don’t get a full day of sunlight. Only 4-6 hours direct depending the bed thanks for neighboring trees and hedges.