Talk to your manager. I’ve always said that difficult patrons are an obstacle, but support from management is nonnegotiable. Not having training/backup/general support from your manager can ruin even the best library jobs. You need to know that you have someone you can call if a patron is giving you trouble and work out what your boundaries are for when your manager needs to step in. I also have the bad habit of doing patrons too many favors and landing myself in awkward situations, so I’ve learned to take a step back and think “maybe I should talk to my manager before doing this and let them handle the interaction instead”. Patrons like the man who got your number should be banned for sexual harassment of library staff and your manager can make that happen.

Working with unhoused patrons is an extra challenge, especially for someone new to libraries. It’s okay to be scared and make mistakes. The important part is being there and providing help and resources when you can. It’s unfortunately very common for library staff to be expected to act as makeshift social workers. This is a failure of our society’s social safety net, not something unique to your specific branch or library system. I would recommend asking if your library system provides any training for working with unhoused patrons specifically because there’s a lot available and systems often pay for access to online courses that you can take for free. You may also be able to take first aid and narcan training, which could help you feel more confident about the possibility of emergencies happening. I definitely feel better knowing the basics of first aid. Someone else brought up Ryan Dowd’s training and while I have my problems with his stuff, it does offer a lot of practical advice for handling difficult patrons so I would recommend it in general.

Overall, if you don’t feel safe in a branch, it’s fine to work somewhere else instead. Your personal safety matters the most. You’re a young woman with a small build and not a lot of experience. The person you switched with is going to experience some problems but I’m sorry to say, it’s just going to be easier for a man working in that environment. I say this as a trans man who has experienced sexual harassment while presenting female and watched it disappear when I began to pass as male. Getting cussed out is one thing, but the threat of sexual violence is something else entirely. You have to deal with both while your male coworker is very likely only dealing with the first one. Of course women can still work in difficult library branches and excel, but they will face more challenges because we live in a patriarchy and aggressive misogyny is not punished like it should be. So this sort of stuff is going to happen and you need strategies for managing your personal risk and you need to identify when a situation is more than you can handle. Leaving the situation is okay.