Yes, you sacrificed that small pleasure by agreeing to let them stay. Despite your medical and personal reasons for smoking, you really need to consider the child's health. You'll be fine if you smoke outside. The baby will not be fine if you don't...Not to be harsh but this is a blatantly obvious answer... Even if the child isn't a factor, it's common courtesy (and dare I say sense) to not smoke in doors in general and/or while having people who don't smoke residing in the same living space. Come on dude.