For the purposes of this post, I'm going to ignore the words "biggest bastard" in the posed question, and replace it with "most evil person" (though for the purpose of completeness, Tormund Giantsbane was probably conceived out of wedlock, and is quite large. Is Hodor trueborn? Might be him, actually).
so...
There are so many characters who are just flat-out evil in that story... but I'd say Joffrey and Gregor (the Mountain) get a pass for mental incompetence. The insanity plea, if you will. They're both more like rabid dogs or forces of nature than people, and should have been put down accordingly before they had the chance to do the harm they did.
Tywin is not evil. Tywin is the most hidebound, acerbic, hateful sort of "Lawful" character out there... and I hate people like him more than almost any other type of person in this world... but he's not evil, he's just a dick.
Roose Bolton is like a lesser version of Tywin with a few kinks and a lot more empathy. Not evil, kind of a dick.
Walder Frey is an old man with a really powerful inferiority complex. He's also a coward. Only a bit evil... just a petty old man who has been bullied for his entire life and finally got his chance to strike back.
Ramsay Snow/Bolton is a rich boy sadist. He's not much of anything beyond that, though he is competent and relatively sane, so he's definitely up there in terms of evil. Bit of a one-note character, honestly.
Petyr Baelish (Littlefinger) is arguably the most successful evil character in the series... though his concerns are high-minded and strategic enough that it's hard to call him as evil as the more directly sadistic characters. He's more detached, slimier, more insidious and political... which would make him number one, if not for...
Cersei fucking Lannister (and yes, as far as I'm concerned, that is her middle name. It's so appropriate, like a double entendre... or triple... or maybe quadruple) is the most horrifically evil character I've ever encountered in fiction. Mind you, in the TV series they toned her down a bit... but this ***** would destroy anything or anyone to achieve her aims, if she only had the ability to do so. Politics, direct violence, physical or mental torture of family or anyone else... all more than acceptable tools for this sadistic, manipulative, and above all self-absorbed character.
...I typed four more paragraphs on Cersei and deleted them. Been reading these too recently, and I'm not sure those were even appropriate for spoiler tags.