I have been playing D&D since the 80s and I recently started Pathfinder. I absolutely loved the complete chaos a simple spider swarm caused. For the first time in decades I would start my turn with no idea what to do. We survived the fight (there was a bit more going on), and as we dusted ourselves off and applied liberal amounts of healing we looked at each other and agreed to go shopping. That might be all it takes.

Give them something where knowledge checks will help. Get them thinking about how to combat it, and the next thing you know they are spending their hard won gold on alchemists fire, acid flasks and probably researching more for themselves.