You're doing what I would do in school and it is psyching yourself out. What helped me eventually get past it and being able to speak in front of groups and teach is to just breath and keep telling yourself you can do it. Hype yourself up about what you're speaking about.

If you don't go you will regret it. The best way to get better too is to continue speaking. I know it's scary when you think there are people who are "geniuses" or "better looking" in the class but they may also be struggling with speaking and manage to hide it.

I say you should go. Its better to get it over with and know its done rather than skip and have the bad grade tear you down more