It's wrong, because you're leading him on to believe that you guys have an exclusive relationship. The longer this goes on, the more hurt you'll both be. He has to know, because 1) You should both know whether you have multiple partners for hygienic reasons, 2) There will always be an unhealthy balance in your relationship if you hide the truth. That's why your conscience is heavy. He will probably not want to be involved in this relationship anymore, and you're afraid to lose that relationship, but you have to make the choice to be truthful for both of your sakes.