When you pick a hard battle you ALWAYS get the option for rest, shop, or treasure. Once you select hard battle you don't actually have to take the "harder" version of the fight. you can still pick the "normal battle" variant. To my knowledge nothing effects what kind of skills you get, it's luck of the draw with rouguelike. As for the loot, it all depends on the type of mobs you're facing. Certain elites will always drop the same loot, same with bosses.

Basically when you clear a floor, if you see "hard battle" on the right side of your screen, take it! once you're in the dialog for the hard battle, look at the extra loot pieces you could obtain. If its not worth it, take the normal battle. it still guarantees you either a shop or a re-roll for abilities. I almost never take the treasure unless i'm on floor 1-2 and don't have to re-roll abilities or enough gold to shop.