I've been to Reykjavik several times and absolutely love it. Consider getting their City Card -- http://www.visitreykjavik.is/travel/reykjavik-city-card -- which gets you on the buses, into the museums, and into the city's thermal pools (which are fantastic!). If you want to splurge, the Blue Lagoon is a cool experience, but it's kind of pricey and crowded that time of year.
The food in Reykjavik is much more interesting these days, and there are some excellent restaurants, though they are also quite expensive. There's a nice organic fish restaurant called, appropriately, Icelandic Fish & Chips (http://www.fishandchips.is/). Not too expensive, by Icelandic standards, and the food is very fresh and tasty. The local sourdough bread is also worth trying, if you're a bread person. I still don't think I've had bread as good....
And whatever you do, DON'T eat the Hákarl! No matter how much brennivín they give you. 
Edit history
= new reply since forum marked as read