The problem, for me, was that the alien plot is, *ahem* well, alien, in an indy film.
The first 3 were based on the notion that religions mythology was true. The arc (jewish), the stones (i have no idea, but i guess some sort of indian tribe), and the grail (christian).
They were, in a sense, all "magical" plots.
Although the crystal skulls are painted as being deities, of some sort of Mayan civilization, the fact that they are explained as alien, crosses the line from (religious) fantasy, to sci-fi.
And I'm not saying that fantasy is better than sci-fi. It's just that this particular franchise is a fantasy one. Imagine that in "Lord of the Rings 4", Frodo boards a spaceship.
I guess had the movie not been an Indiana Jones film, but a stand alone summer adventure, I would've enjoyed it more. I wouldn't have had any problems with the aliens (although I would still be bitching about the refrigerator and the Tarzan scenes).