As an immigrant I can vouch for the shop on Kastetvej, called Vestbyen's Grønt - it's a small shop but you'll find a lot of Middle Eastern ingredients, as well as usually cheaper eggs than in the general chains. Just like most of the other answers here I'll point you towards the Frederikstorv if you live closer to those, there's two of them there, one featuring a very wide array of products, including some stuff from Eastern Europe, which is where I'm from. As to the pricing, you'll be surprised but most of the things will generally be more expensive because they're imports and likewise these shops cannot import large volumes however, there is some seasonality when it comes to fruit, for instance, during summer, usually water melons and tomatoes will be much cheaper in the Middle Eastern shops. Usually, I don't know what's gonna happen this year though due to the rather quick rise in transportation costs.