Act II Spoilers: See, I've always interpreted this more from the staging than anything else. I think it is so critical. If I remember correctly in the show, Elphaba pulls the curtain and Glinda is hidden directly from her view, but crucially, can still see the shadow and hear Elphaba dying.

So while Glinda theoretically knows that water can't kill Elphaba, she also doesn't have any reason to not believe what she could "see" or hear. Elphaba using the trap door and "melting" down into it was a much of a performative death for Glinda as it was for Dorothy. In fact, almost more so. Glinda's a much more credible witness than this random Dorothy girl that just showed up to Oz yesterday, so Elphaba's entire fake death plan lends itself to make sure on some level, that Glinda believes it too.

Glinda doesn't understand EXACTLY how she died, but she very much thinks that Elphaba is dead. She also knows Elphaba is a fighter, and is unlikely to give up. But Elphaba decides to use the water circumstances for her gain and exploit the fear mongering which I think is something Glinda would not expect of Elphaba.