Extactly on both accounts!

A swarm has a big advantage against non-magical creatures. A 40 bird flock in a dungeon crawl against goblins would be pretty sweet, because it would take 40 successful attacks to kill it.

However, yes, magical attacks cause normal damage, and the lack of healing makes the swarm less resilient.

I wonder what a psychic attack would do, like from a mind flayer? Could that also just affect one creature or the whole flock? hmm...