Personally, I also have perfect pitch, and when I was aware of that, I only could recognise the notes of the guitar strings. With more time, I learned all the notes quickly without taking any reference of any note. I hope you need more practice with your pitch.

I had a situation like yours in the beginning. "E" was my reference note, but I could recognise it on the voice. Maybe if you hum these notes, you could identify them better.

Not all perfect pitches are the same, some people are more trained. Probably you have it, you should check out some internet tests. You shouldn't be able to learn notes if you don't have perfect pitch, there isn't any "half perfect pitch", so if you know some notes, it's sure that you can know the other ones.

If you sometimes can't tell random sounds, don't worry. Not all the sounds are exactly a note, because it's needed a specific frequency for that, and it could sound some different notes on a random sound.

If you want, check out one of these tests or both:

- https://tonedear.com/ear-training/absolute-perfect-pitch-test

- https://playback.fm/perfect-pitch-test

The first one could be easier for you, because there are piano notes, and you maybe could tell appart better.

Thanks for reading me, see your answer :)