My top 3 guesses:
1.) Being really buff kinda went out in the 90's, leaner muscular with less mass, and more definition is what I think more women look for now days.
2.) I'd have to agree with my buddy Ronin4Hire, your approach is important. Some American guys are really flashy, or cocky and that's not really a good look in Japan.
3.) Maybe you haven't found "The One" There are plenty of fish out there you just have to be patient, and let them come to you.