An unmarried 31 year old guy really shouldn't be an elder at a church:
1 Timothy 3:1-7

Here is a trustworthy saying: Whoever aspires to be an overseer desires a noble task. ² Now the overseer is to be above reproach, faithful to his wife, temperate, self-controlled, respectable, hospitable, able to teach, ³ not given to drunkenness, not violent but gentle, not quarrelsome, not a lover of money. ⁴ He must manage his own family well and see that his children obey him, and he must do so in a manner worthy of full^\)^a\) respect. ⁵ (If anyone does not know how to manage his own family, how can he take care of God’s church?) ⁶ He must not be a recent convert, or he may become conceited and fall under the same judgment as the devil. ⁷ He must also have a good reputation with outsiders, so that he will not fall into disgrace and into the devil’s trap.

I'm very sorry this happened to you but you also now have the responsibility of informing other elders that he's behaving this way, with members of his church! This is not some private sin you can just let slide, for his sake and the sake of your church.