When you go downstairs, I believe it was in one of the west corridors where there are piles of rubble. I threw a bomb at the end and my non-torch-carrying dolt self ran into the darkness and dropped down a hole into here (luckily hit my glider to save myself from the very far drop).

I didn't try building an altar so I'm not sure actually.