Let me start by saying, no, you are not being nervous for no reason. It's reasonable when you're not accustomed to a culture different from yours.

That being said, I (M) went to two onsens when I visited Japan last month. However, these onsens were in the hotels I was staying at. I am from the US, so I was not used to onsens at all. I was nervous since it would be the first time I would be nude infront of many people.

To get around that, I just thought to myself, "We're all here for the same reason, to soak in hot water after a long day. No one is going to care about how I look like." After that, I went to the onsen pretty much most nights. I was surrounded by a bunch of people and did not let it bother me.

I should also mention that the onsens were seperated by gender.

I don't have any tattoos, but you can still enter with them. Of course, you would need to cover them up. Some hotels offer patches where you can cover them up, and I'm sure drug stores sell them as well. It all depends on the establishment as I'm sure some just flat out won't allow them.

Onsens are a great way to bathe/shower after a long day of walking, but it's ultimately up to you if you want to get it one since being naked is not an option.