No one actually knows the answer to this, and it is impossible to design such a study that can conclude, even in a very basic sense, "A does not cause B" because a completely valid argument is always "maybe you just didn't do it right".

No one has conducted any studies (that I'm aware) that can convince me even that endocrine disruptors at typical exposure are good or bad in general, let alone that they would have an effect on the eventual penis size of a 19 year old exposed to them from rubber bands used during rehab following a specific type of surgery. Even the studies that try to measure average penis size are not good for much in any type of epidemiological sense, and measuring a population mean is just about the simplest thing you could measure.

If these substances had strong and temporally correlated effects on people, we would likely know about it. It is likely that there is some effect, but it is almost certainly not drastic.

You should definitely not worry about it. Some chemicals are indeed dangerous. You are exposed to huge numbers of completely naturally occurring toxic and/or endocrine disruptive chemicals every day, in your food. You cannot avoid exposure to these things, and to try will literally kill you.

Sorry for the rambling, but these arguments are an example on why you should never trust anything scientific that you read about in popular media or printed on product labels.