The game I think of as most consistently praised by Yahtzee is the first Condemned. This suggests he prizes things like storytelling, atmosphere, and innovation (such as that game's focus on melee weapons). He pretty much tells you directly that he values these things in other reviews, as well.
The fact that I think one of his most esteemed games is crap tells me that I should never let him influence my buying decisions. Instead, I listen to him for humor, vocabulary, obscure references, and to listen to a fast-paced British accent say hilariously lowbrow stuff (let's face it, ZP would be nowhere near as funny if it was dictated by a guy with a New Jersey brogue, at least not to me.).
Just because he criticizes a highly critiqued media does not make him a critic. More of a humorist and satirist. I wouldn't have any problem with him espousing a game's good points as long as he was equally funny about it.