It depends on where you go and what kind of kilt you are going to wear. There are some areas in Tokyo (i.e. 原宿) where men wearing a skirt isn't that much of a big deal. People bend over backwards to dress unique there. And many Japanese people do know what Scottish kilt is so it could be a great way to start a conversation with the locals if what you are wearing something that looks somewhat authentic. JasonTakeshi is right though. Make sure you wear underwear