Yeah, depends on the game. When it's a case of the music being integral to the atmosphere and hearing characters talk, then I won't listen to music.
But I listen to music when replaying Tomb Raider, because I know what happens, and Lego Star Wars, because nobody talks. And of course fighting...