To respond to the OP, I can understand your feelings, and you are far from alone. A lot of people are conditioned to see cartoons as being for kids only. Anime is intended to entertain a wider audience. Some people can adjust, others cannot. It's no big thing.
Most of my interests which I broadly define as "fandom" (though I guess that isn't accurate) revolve around science fiction, fantasy, and horror in all of their incarnations, and that includes Anime (which like anything varies in quality).
If it helps at all, it's best to look at anime by stripping away the pretensions of Weeaboos and Wapanese who act like it's some kind of massively intellectual medium. In reality like 99% of the anime out there is aimed at an audience of teenagers ranging from like 13 to 17. Albeit it's done in a culture with differant standards about sex and violence. What it's doing pretty much is following through on a lot of the stuff that was dancing around the fringes of your consciousness back around that time. It's sort of like how with my generation for example you'd watch GI Joe and notice that nobody ever gets shot, and even the bad guys in exploding jets parachute away. Well, anime is pretty much what you get when you do something like GI Joe, but remove the enforced morality/pro-active censorship. A typical anime being a sort of science fiction action thriller, where unlike western cartoons the weapons have realistic impacts, and where the characters in a western cartoon are always chaste except for the occasional kiss, the sexuality is obvious and oftentimes as over the top as the rest of the overall storyline.
Where anime gets the occasional "intellectual" defense is that it sometimes will use science fiction concepts that are fairly complex, or generally only handled in text/novels. Anime was introducing and explaining concepts like nano technology, mental manipulation through memes, and other things before they were more commonly used science fiction stereotypes. They also played around with the idea of evil mega corperations and such before they became stock villains, things like "Genom" (from the original Bubble Gum Crisis) being the mould from which most of them derived. Indeed the whole "Aztechnology" thing and their CEO in Shadowrun seemed to be a homage to this.
Now don't get me wrong, anime can vary like anything in quality and intended audience. I'm talking in generalities here, and what most of it is about.
Something like Spongebob is directed at very small children, the majority of Anime is not, and it's easier to see it as like the guys from GI Joe firing at some Cobra troops and simply not missing, than say Spongebob going on a psycho murder rampage.
At least these are my thoughts.
As I have said in some other posts, I also think there has gotten to be a tremendous amount of absolute anime garbage shoveled into the US. I think one of the problems is that for a lot of Anime fanboys, the ones usually called "Weeaboos" they'll defend anything anime, or coming from Japan just because it's anime or from Japan. I still like anime to a great extent but I have little problem with calling a turd a turd, and right now we get like 9 turds released in the US for every decent series.
Anime became part of my overall sci-fi/fantasy/horror interests because when I first encountered it I was seeing a lot of stuff that was good. If my first experiences were with the majority of the stuff choking the shelves of your local Best Buy nowadays I might very well have become a hater.