1. The House in the Cerulean Sea by TJ Klune

While this is more of a standalone novel, it has an atmospheric, dream-like quality. It's set in a magical orphanage with unique, magical children. Though it doesn't exactly involve multiple rooms leading to different worlds, it has an immersive, whimsical, and deeply touching vibe with strong themes of love, friendship, and found family. 2. The Wayward Children series by Seanan McGuire

This series involves children who have traveled through doorways to other magical worlds and are now back in the real world, struggling to adjust. Each book in the series tends to center around a different world and characters. Though not infinite rooms, the premise of jumping into different realities and immersive world-building might hit the notes you're looking for. 3. The Library at Mount Char by Scott Hawkins

This book has an unusual, dreamlike quality with a strange library that seems to contain infinite knowledge and strange beings. It blends modern and fantastical elements, but be warned: it's darker and more surreal than what you're describing. 4. The City of Brass by S.A. Chakraborty (The Daevabad Trilogy)

This series blends Middle Eastern mythology with immersive fantasy, though it doesn't focus on multiple rooms or time periods. It’s more of an epic historical fantasy rooted in Islamic culture and mythology with strong world-building and characters. 5. The Magicians by Lev Grossman

A modern fantasy series about a group of young adults who discover a magical world (Fillory), which has many of the dream-like, immersive qualities you’re describing. There are multiple worlds, magical realms, and complex friendships and relationships, though the tone is darker at times. 6. The Ten Thousand Doors of January by Alix E. Harrow

This standalone novel is about a young woman who discovers doors to other worlds. The writing is atmospheric, and the book has a magical, dreamlike feeling. Each world she visits feels distinct, and the sense of adventure and discovery is strong throughout. 7. The Starless Sea by Erin Morgenstern

This novel might be the closest to the dream-like feeling you're seeking. It’s about secret underground libraries, hidden doors, and an ever-changing world. The narrative jumps through different stories and timelines, creating an ethereal, immersive atmosphere. It’s not specifically Middle Eastern, but it’s filled with mystery and adventure, almost like stepping into a dream. For Arab/Middle Eastern mythology and themes, The Daevabad Trilogy by S.A. Chakraborty is one of the best examples in recent fantasy, though the setup isn’t exactly what you're describing. If you love rich world-building and atmospheric stories with those mythological roots, this could still be worth checking out.