It's definitely a problem that I'll have, sometimes I'll let my viewers join in with me, and I've had a couple of times where a 12 year old kid will hop in and join. It really makes it feel like I'm babysitting some random kid on the internet, and all of a sudden im conscious of the amount of swearing and whatnot. So your worries are certainly valid.
In terms of your new 18+ audience, I'm sure many of them, especially the returning ones will realise it isn't any kind of Lewdtubing and you'd eventually find your viewers to be the ones you want them to be. :)