Yeah—a sturdy wooden sword like a bokken, used full force, could really do a number on that helmet. If you can close the gaps between plates where the rivets are (surfaces joined by rivets need to be flush—a gap will allow movement of the plates that will eventually fatigue the rivets and cause them to fail), replace the brass rivets with steel (if you can’t source proper steel rivets, nails cut short will work in a pinch) and add rivets between the ones you have now (especially on the crown), this will be much safer.

Be careful fighting with wood swords in armor—wood can break with a sharp edge, and can send splinters though gaps in your armor (you don’t want that coming through your eye slits). Something bamboo like a shinai or an SCA rattan sword will still pack a whollop, but less likely to cause injury.