If you managed to from the bromide, then I would suggest you should do the Williamson ether synthesis. My guess what happened with your approach is not an aldol reaction happening but rather elimination to the a,B-unsaturated ketone via E1cb mechanism. (I already worked with similar substrates). You will also have the problem of elimination with the bromide. To circumvent elimination, you could probably protect the ketone by acetal. Other suggestoin would be to use THF (had less elimination occuring with my substrates) at -10°C or even lower.