As someone who doesn't enjoy hot and humid weather it would depend on which part of Japan you are talking about as the weather from one end to the other can be vastly different. Even by June most of Japan is starting to get too hot and humid for my liking except up in the northern areas. I wouldn't even consider travelling south of Hokkaido in July or August. So I'd go for June before it gets too hot but I understand down south this is the start of the wet season?
Although I've been here for 5 years now I've only been out of Hokkaido a couple of times!
If I was to travel down south I'd more look at late Oct into Nov I think when it's really started to cool down again.