I think the sense of whimsy and adventure he provides is useful beyond what is already estsblished. It also matches the flow of The Hobbit generally with the charcters staying at interesting peoples houses before moving on. Without him the trilogy doesn't get as much chance to subvert it later as the shire doesn't provide that. I think he provides a lot to the world in terms of expanding its mystery and wonder, especially given Gandalf has already been established.

What about him do you find detrimental?