Two feelings; one being feeling of trespassing in a place that is not kind to strangers, case in point, in the first Condemned (the only one I played), just about every goddamn place you have to go to in order to solve the mystery gives that feeling that you are not welcome, that you should not be there, That (to quote the Sin City comics) "this is a bad place, people have died here, the wrong way."
the second feeling would be that of insignificance. I'm not sure if the ideas behind Lovecraftien plots are scarier than the plots themselves, but in like Eternal Darkness Sanity's Requim, even though the main character keeps the Ancients out of the world, Mantarok is still there with an unbound essence and a cleared playing field. The idea that our actions are meaningless against the backdrop of beings of immeasurable power and that we as a species would go insane if we really understood our profound insignificance within the universe seems like some scary shit to me.
For example,
"Hey Bob, how was your day"
"Fine man, I just got thi-"
" That's great Bob, do you realize that your actions and very existence is meaningless since most of what we percieve is a sugar-coated illusion meant to keep us from realizing the abject terror of our own insignificance, which mind you can be easily snuffed out by imperceptible madness-inducing beings so powerful that to destroy the human race and earth would be to them like eating a rocky piece of candy filled with juicy souls or stomping out ants that scream?"
"Well now that you put it that way man, I believe I'm going to move to some remote location and start a cult that practises human sacrifice in hopes, however futile it may be, of placating to these aforementioned impercetpible beings of power so that I may be spared in the inevitable holocaust."
"I'm glad I got you to see things in a different way, Bob."