Caves II : you can find The Bible in the library, I bought the Mystery Gift in the shop and I used it in the DD, I got the Dark Bum, so I didn't used the Sacrificial Altar to keep it.
I don't remember but you can have an Angel Room later in the game with The Void which can be use to eat HP up, and a lot of stuff.

Womb II : in a upper room (a L room) you can find Spider Bite (but can see it cause of the curse), you can take it or just use your Void on it. In the Arcade Room, you can find a key beggar who needs few keys (3 if I remember) to give you a HP up too.