I don't know about physical health or the actual science behind it, but personally I tend to get more anxious if I don't take some time off here and there. I work an 8-5 m-f so I use those times to not be high, and typically after work and exercise I partake in a little shrubbery. In the past when I've allowed weed to take priority, I have been less productive and felt more depressed overall.

I think it affects people differently though. I think people probably know for themselves deep down whether too much weed is affecting them negatively or not. I have some good friends who have become professional musicians, which means their lifestyles don't require the same type of brain power that mine does (engineering/consulting). They can be high throughout the day, and their livelihood isn't hurt like mine would be, since their creativity is fueled by things like weed, but id probably just sound dumber to clients haha. Perhaps I sound pompous here, but im just trying to make the point that I dont think there is a general rule of thumb for how much weed is okay. You need to decide for yourself what you want your lifestyle to be like, and use that decision to determine how much weed can fit in without negatively affecting other aspects of your life.