Well...
I lived here (in Osaka) for a year doing study abroad. I fell in love with the place almost immediately and decided early on that I wanted to spend at least a bunch of years of my life continuing to live here.
I came to Japan on a whim. I don't really like anime, I'm not exactly a weaboo or anything like that. I had been taking some Japanese courses at my college (also randomly), and one day just walked out of my apartment, signed some papers at the study abroad office, and a few months later found myself sweating my balls off in the late summer Osaka heat.
I never regretted a second of it.
I picked up the language extemely quickly, which greatly enhanced my experience. I made friends with the locals and hung out with them much more than my fellow ryuugakusei. The other foreign students hung out mostly exclusively with each other; there were several large pockets of those who were stuck in the "frustration" stage of culture shock and spent the majority of their time sucking at Japanese, hanging out in the dorms, and complaining about everything.
Personally, the fact that nobody would sit next to me (I'm not fat, I don't take up much room at all and I certainly scoot over when someone is nearby) on a bus or train, even if it was crowded and such. Also the stares, and the random bursts of racism. Instead of getting too huffy about it, though, I took the other approach and enrolled in some cross-cultural communication and Japanese modern sociology/anthropology courses to better understand the underlying reasons. Long story short, I figured it out, and I also figured out how to fix it for myself, and I enjoy life a lot more because of it.
I went back to America for three months to graduate from University, at which point I really reconnected with my American friends and culture. I was very sad to leave it behind, but I made it a point to, instead of trying to be as Japanese as I could to not stand out, hang on to my culture. It isn't exactly the Japanese way, but I am, as Oz said, outgoing and friendly with people. Hell, I say hello and thank you to clerks at shops and convenience stores. Never see a Japanese person doing that!
I've been here again for a little less than two weeks. I'm still looking for a company who's willing to sponsor my visa, no luck so far. Everyone, people who have done it before, and Japanese alike, have been telling me how easy it should be, especially because I can speak Japanese, and I'm TESOL/TEFL certified.
NOPE.
The one cultural hurdle I haven't been able to surpass is that it seems more and more like the country's trying to keep me out. Recently it's been revealed to me that all countries are like that--trying to keep the jobs for its own people and all that--and some are much, much worse about it than Japan. It is getting a little worse here, what with that bill in Diet about the computerised gaijin cards that can be scanned by police to point out all the foreigners in a crowd... but we'll see.
Anyway, I love it here. Not enough to spend the rest of my life in Japan, but certainly a bunch of years.