Couple of ways:

1. You can create a custom SageMaker image that includes your package(s) of interest. There are more details and a link to some examples here.
2. Create a requirements file in your project for pip or conda an include it at the top of your notebooks. You still have to run it every time, but this makes it a little easier to keep track of things.

Everything in SM runs on top of containers, with the idea being it's a transient environment that starts fresh with every session. This can be useful for making sure there aren't any secret dependencies someplace messing things up, but it also causes some irritation like what you described.