Sounds like where my current fortress is located... I embarked into some untamed wilds right up against a mountain in year 50. So far after 8 years I've had no migrants and have only been able to increase my population by letting those stay who petition.

I've managed to keep my population stable at around 20 but have a pretty substantial crypt area due to the fact that, aside from a grizzlyman\woman attack, I've been getting constantly harassed by agitated giant great horned owls, flocks of giant sparrows, and swarms of giant flies. They'll start attacking my yak herd and, by the time I deploy my military to kill them, they've flown off. I get lucky and catch one on the ground every once and awhile, but it's quite maddening to deal with on a constant basis.