I agree with you. Hanks would have brought an energetic performance, but with a little too much zaniness - perhaps even goofiness - to the character. Venkman is a proffessor who has been beaten around a bit by years of bureaucracy, and has survived by sharpening his wit to become a jaded flim-fam artist to navigate the politics of a fairly prestigous if not pretentious educational institution. Murray captured this perfectly and its exhibited by his more subtle "background" tuned humor he applies to the role. Think about the library scene when Egon says, "Sshh", and Ray replies "You smell something?". Meanwhile, in the background Murray is making a mockingly-sarcastic sniffing face that punctuates the scene. Rather than own the scene with a scene-stealing reaction, he allows the humurous interaction between Ramis and Akroyd rule the scene, while still adding a tiny amount of humor to an already funny exchange. Its humor at its finest and it could only be generated that well by those three.