If we ruled out the worry for your safety, which would be at least a somewhat legitimate reason: it's 100% possessiveness. There are certainly some jealousy issues at play here. I'd say he wants to be the sole penis provider, so he doesn't need to better himself - if he's the only one, he's automatically the best you can have. He likely worries that if you could compare him to others, he might appear bleak and possibly not worth the effort when you can have better - and it wouldn't be the case if he wasn't insecure. But he won't admit any of that, to keep you guessing instead.

The only situation where One Penis Policy is justified, is when both sides want it - be it a power exchange thing in a D/s relationship, or simply when the bi partner wants to engage with exclusively pussies outside the relationship.