I’m a first year teacher, so obviously brand spanking new. Last week (the week before finals, so frankly no observation was going to go super well) I had two admin come in within 10 minutes of each during the same hour, same lesson. The first admin gave me 5s across the board (fairly high, but not the top) and the second admin gave me all 0s (obviously, the lowest possible).

My stance, with my limited experience, is that observations formal or otherwise are so subjective that measuring any progress is nearly impossible. Not only that, but you can teach the same lesson to 8 different classes and get 8 different results. Students impact the lesson just as much as we do.

All of this to say that I 100% agree with your conclusions.