I really can`t say without knowing him, but I`d make the guess that it`s one of two things;
He is attracted to you but now given the chance doesn`t know how to go about it or is scared...
Or - he was joking all along and didn`t think the opportunity would ever arise so didn`t put much thought into it, and doesn`t want to lead you on or have others think he`s going after you...
Japanese men are all individuals, so there is no possible way to say "this is what a Japanese guy does!" You`re just going to have to really feel it out yourself. An international relationship is something easy to joke about, but when it comes down to putting it into practice - particularly Japanese man / foreign woman - there are a lot of hoops and stress that may be too much to think through.
So even if he does like you, there may be a whole lot more to it. The stigma of a work relationship, of a interracial relationship, of a relationship very likely to end up long distance, of a relationship with a communication barrier (making the assumption that he is not perfectly fluent in English or you perfectly fluent in Japanese), etc etc...
Sort of like a dream that is great while it`s a dream, but in real life carries a lot of problems along with the dreamy bits.
Or - he might just be that type of personality and you`ve misinterpreted all along.
Why not just start out trying to be on friendly terms and getting to know him better in general? (Along with everyone else you`ll be working with too, of course - would be way too obvious if you JUST keep talking to him, etc) It doesn`t really sound like you know him that well at all - and that isn`t a good start for a relationship.