[Nyororin]

I am going to second this. I have said similar things in the past.
I think a huge amount of difficulties experienced by foreigners living in Japan are rooted almost entirely in language. It is like looking at the world in the dark - there are lots of things there, but either you can`t see them or can`t see them clearly enough, so the mind makes them into things they may not be. Sometimes this is just confusing, frustrating, and tiring - but other times it leads to the "everyone is talking about me" / "everyone is being racist toward me" syndrome experienced pretty frequently by foreigners in Japan.

I came to Japan with a handful of words, a couple of basic sentence patterns I could use, and a few full phrases. Nothing even close to conversation level. I think that I could read most of the Hiragana, and about half the Katakana.

Basically, about as much as someone who is studying could pick up in a week or two.

I never really actively made the choice to not associate with foreigners. I did a homestay, so really had no choice. The high school I attended did have exchange students, but we did not get along in any way whatsoever... So I went out of my way not to associate with them because we hated each other. I don`t recall feeling particularly lonely, even when I wasn`t all that good at speaking.
The most depressing experiences I had were bullying at the hands of the other exchange students...

Now I speak fluently, and have no problems related to cultural or language differences. The cultural stuff comes hand in hand with the language, so speaking Japanese is far more important than just communication.