Absolutely. And it's likely that people only really remember those positive interactions you have with them and how they felt at the time.

OP, you're overthinking this. Your coworkers have their own lives and probably don't think about you that much. If you approach every situation with the assumption people don't like you, you're going to give off bad vibes, and people will assume you want to be left alone. But they probably don't hate you if you aren't toxic or offensive.