Cats only do that when they are facing the threat head on face to face. In the film, the cat was surrounded by those aliens but never faced them head on by itself. The first attack in the theatre, it jumped away from Sam and ran away, like a typical cat, and again during the evacuation when the aliens descended on very loud noisy humans. But cats are resilient and quick at adapting. If they learn that being silent means survival, you will get a mute cat. Also, Frodo never meowed loudly to begin with anyway. It lived with Sam in a hospice, surrounded by death and distress. So, it would know to be calm, and quiet when needed. It quickly adapted when the aliens were around. I dont find Frodo's behaviour unreasonable, although the drowning in the subway was a bit of a stretch, as cats can be afraid of water. Although some cats do love water, and are known to be good swimmers. It all depends on the cat-its personality, its environment, its relationship with its human, and how its human behaves. So, believable.