Yep, fear not nukiuchi. All us wacky gaikokujin have something odd about us. You wear a hijab, I'm a six foot woman
You're much more likely to face discrimination because you're a woman than because you've got headwear, but most Japanese are polite enough that most tourists never get to experience this. The worst I've had is older businessmen expecting me to get out of their way when they've altered course to walk directly toward me, and that's no worse than the behaviour of London commuters (from whom I've had far more shoulder-bumps than Japanese businessmen in Japan).
When you come to live in Japan, being a woman will be your handicap, unless you want to live at home and do housework all day. Then you're fine
Basically people are people. The majority are harmless, but every now and then there's a nitwit.