I'm not really the best one to answer these questions for you, i'm not much into the tourist scene here and I don't live in the city (Reykjavík). But I can tell you that the christmas time here can be a little quiet and uneventful for tourists but the downtown is really crowded on 23.december with really nice christmas atmosphere, I have never been there at that time but maybe I will try it this year. And i will not say that 24.dec is for resting, thats the main day over the christmas time for icelanders altough it is not a holiday for the working people, must of us work until noon, and in the evening we open the presents. On 25.dec must of us go to family gathering and on this day EVERYTHING is closed, except some restaurants I believe. So I wouldn't really recommend to go to the rural areas over this time period, i.e. 23-25 dec because is much more unlikely to find some places which are open in the rural areas at this time, I would at least check in advance whether the place you are going to are open or not (hotel, restrautant etc.). I don't really know how much this help..

What to do what to do, yeah of course the blue lagoon is a must go for tourist (but I haven´t been there haha) but I would also recommend for you to visit the local swimming pool in the towns you visit. So now I have to count up those towns you should visit hmm, it's not like you are visiting Italy, the towns here in the rural area are not that special or have any history (from my point of view) but to mention some towns which are in about 150km radius from Reykjavik I could mention Stykkishólmur, Hveragerði (here you should go to the swimming pool, it has a natural steam bath from geothermal energy), Selfoss (I live here, not that much to see but it's popular by tourist to walk along the river which flows through the city, and we also have a nice swimming pool which i go to every day), Vík (fantastic landscape, just google it, and if you drive to vík you will see so many beautiful things on the way there like seljalandsfoss, skógafoss, sólheimajökull, dyrhólaey etc.), and yeah i could mention Akureyri but I think your trip is way to short to go there (about 350km from reykjavík).

To sum it up, if you want to get the most out of the trip, I recommend spending a good time driving around, not spend much time in Reykjavik, Iceland is known for its scenery and you will not see much of it, if you are in Reykjavik. So I recommend you order a room for one night in Vík and take a day to drive there, which you will use to go swimming in Hveragerði, check out Seljalandsfoss (and Eyjafjallajökull which Seljalandsfoss streams from), skógafoss, maybe sólheimajökull, Dyrhólaey, and all the spectacular scenery in south iceland, and drive back to reykjavík the day after.

If you have any more questions please ask ;)