Just following orders is a fair enough excuse.
Milgram proved that the average American is willing to potentially kill someone if ordered to by a figure of authority, with a compliance rate of above 90% if they cannot see their victim. Add deindividuization into the mix and any number of other proven psychological concepts and bam, Just following orders carries a hell of a lot of weight. You can either claim it does not, ignore all psychological evidence, or you can accept that if push came to shove, chances are you would follow an order that you personally find morally wrong. Many in the Milgram experiment cried, shook, hyperventilated. Yet they still carried out the order, despite no threat, no true coercion (Only phrases such as "The experiment must go on" were used). They hated what they were doing yet they continued (At least, two thirds of those involved continued when they could see and year the "Learner" screaming in pain).
When you consider the nature of the Camps (Because that is what is really being talked about here) the deindividuization of those involved (Both the minorities being annihilated and the guards committing the acts) and the fact that in many cases (The chambers) those committing the horrendous acts could not see the victims, could not hear them (After they moved away from the mobile gas chambers at least), they were simply conforming in the same way that it was proved 90% of those who participated in Milgram experiments would.
Combined with the propaganda and the fact that they were being ordered to do such things by their direct military superiors, well, their actions may have been "evil". But now we can understand them.
*heads off to find a quote from Milgram*
"I would say, on the basis of having observed a thousand people in the experiment and having my own intuition shaped and informed by these experiments, that if a system of death camps were set up in the United States of the sort we had seen in Nazi Germany, one would be able to find sufficient personnel for those camps in any medium-sized American town."
Chances are any of us would commit an act of extreme violence if asked to by someone in a position of authority over us. None of us think we would. Milgram thought very few people would. The experiment was carried out hundreds of times in different forms, including asking people to torture a puppy. Hell, in the 1920's someone managed to get countless students to decapitate a rat they were holding in their hand, simply through the power of suggestion (Not coercion) and being in a position of authority (Although he did not make the connection).
I wish the experiment had been carried out upon soldiers. I believe the rates of compliance would have been far higher.
All of us have the capability to be "Evil".
Never forget it.