I don't know if he will be working during the winter, but the Reykjavik music walk is also really cool. It's lead by Arnar Thoroddsen a music history buff, and he's really entertaining as well.
And there is always I heart Reykjavik. It's a great outfit that has multiple tour guides all run Auður. The tours aren't music specific, but she hosts different types of events all throughout the year including dinner at her house once in a while. And her blog: https://iheartreykjavik.net/ has a lot of information about what to do around Iceland in the event you can't get things worked out otherwise.
You may want to reach to Ivar though as he may increase the tour size or have changes between then and now and you may be able to get into one of his tours. I took it in November and was really impressed, I even changed my traditional shopping habits because of it.