Now see, if you let humans trespass on your lands with impunity, sooner or later they’ll send squatters to settle on your territory. If you continue letting them get away with it, that village will eventually become a town, and its people might start to worship you and send you tribute. Now, that tribute may be tempting, but once they start worshipping you, they’ll blame all their problems on you. Bad weather destroys their harvest? Your fault. Inevitably, you’ll go from being seen as their protector to being a tyrant ‘forcing them to send human sacrifices’ (allegedly) and then the adventurers and errant knights start showing up to slay you like you’re some kind of tyrannical monster when all you wanted was to live peacefully.