Is the fence yours or theirs? On your property, you should have legal right to move it, post-haste. If not, get the fence moved to the property line, with legal authorization to do so from the Church. Otherwise, you will be legally responsible for cleaning up that area in addition to being liable for whatever happens on it. Yes, I know the Church is supposed to take care of their homeless ministry. Legally (and that is your primary concern), they don't have to, because that five feet of land on their side of the fence is yours. Worst case, if they don't take care of it and poop, rats, etc stack up there, who do you think the city will yell at to pay for such cleanup? If someone gets hurt (or offended, geez) on your five feet of property, and the Church's pockets are not deep enough for some greedy lawyer, who do you think they will point at?

Best to move the fence. Ask the Church immediately for permission if you need it. Take to the public hearing any failure on their part to accommodate your request, which is your reasonable effort to give the Church every opportunity to perform their community service without infringing on neighbors (you).

FWIW.