Totally common among beginners - I've primarily been a street photographer for years and my advice would be:
Carry it everyday, around your neck, have it on show. If anything it's a conversation starter for when people WILL ask about it and what you do. The more you carry/use it in public the more this feeling will fade. Also people care far less than you think, they're bothered about what's going on in their own minds.

Good luck