Ooo, just watched a Criminal Minds episode on this! In that case, the man's memory did come trickling back, until he remembered everything (can't tell how long it took, since everything takes place in TV-show-time). Luckily, he's honest about his recovery, and conviction proceeds without further problems (he had killed 3 or so women). But while he was still confused, the issue of punishing a amnesiac was discussed.
Apparently, in Western philosophy, the concept of being a product of your past is dominant; without one, without the memory of one, you're practically a whole 'nother person. Whether that is a universal truth is definitely up for grabs....I personally think there should be some sort of middle ground, where the criminal is still being detained and monitored as a result of his unlawful actions, but where conviction and a sentence is postponed. The amnesia, hopefully, will not be permanent. If it is...well, perhaps a strict parole, rather than bleak imprisonment, would be more on the fair side of things. As for the victim's families...they'll certainly fight that kind of decision in court, but if the criminal truly can't remember, he shouldn't be punished...until he can recall.