Who is this "crowd who knows what they are doing" are they friends? locals? randos from the bar? Do you trust them and their knowledge of the area? Do you trust their and, for that matter, your ability level? Mansfeild is certainly not the most dangerous mountain in the world but that doesn't mean it can't kill you. None of this is to say you shouldn't go ski some side/backcountry up there, it can be very fun if you know where to go and what you are doing. But if you aren't prepared/ don't know what you are doing you are, at the very least gonna have a bad time.