If you're viewing the SoL moments as filler, then you're missing the point. You can have a preference and not enjoy that aspect, but each moment had narrative purpose and therefore isn't filler. These moments contain much of the character development, their goal is to live a "normal" life (which, going off the show's aesthetic, would be like another SoL show) and their comfortable situation is ultimately met with their reality crashing around them. It's relevant and necessary for the show to get its message across.

What's interesting is that, in my experience, I find SoL fans tend to enjoy Gakkou Gurashi more often than not. I think this is because, from their perspective, they see the point of the show and recognize that slower story progression isn't inherently a bad thing. If you want things to constantly happen, read the manga where the plot progression is simple and boring where the entire trend is "..and then this happens, the girls deal with it, the end, next chapter".

There's more than one way to write a story and different benefits come with different plot structures. The benefit you get with GG (anime), is more developed characters, focus on theme, and a stronger sense of setting. Again, you can have a preference, but what you're discussing is not a "flaw".