[bELyVIS]

I had a Japanese friend that was mugged and beaten by 4 Japanese guys with sticks. I also have seen racism, mostly in my workplace. Japan isn't a magical place free from all Earthly problems. Racism is part of their culture, so you can't get too upset. Crime happens everywhere. I never felt unsafe or unwelcome there. If they don't want foreigners (probably caused trouble or was a "ugly" tourist) there usually is a reason because they will take your money like everyone else.
I wonder if the OP fits into this category. As for the mugging, maybe it was a misunderstanding and he gave his wallet to some guy asking directions to the knife sharpener's shop (OK, so this is unlikely but I know of one instance of something like this happening).
Japan is a polite, safe, society that makes me wonder what the OP did to have this happen (If it did).