Well I used to get mad at this stuff too. My friends used to tease me about watching cartoons and I used to fret and fume about it until I found out that the people who were saying those things only knew cartoons like "Tom and Jerry" and the age old ones.
They had no clue what anime was about. So I then convinced them into seeing an episode of deathnote and Poof! the jokes all vanished. I must admit they still didn't take to it because they were a little too ignorant but atleast they had to admit it wasn't lame and was totally bad ass!
Eventually, I agree with the verdict, Ignorant people will rant the loudest about stuff they don't know. The ones who really know about anime are those that don't blaspheme about it. Most people I know that know about anime really like it.
I should also state that I don't think it has anything to do with Asia or Japan in particular. People I've met don't like cartoons because they fell it's childish, not because it's made in the Asian continent. But then again that probably doesn't count for much because the people I've met are Asian and the wouldn't discriminate anime for being of Asian origin too. ^^