I'm a bit confused...were you married in Japan or Germany? Did you divorce your spouse prior to their death or are you widowed?
If you were married in Japan, you should have been issued some sort of certificate by the Ward Office of the Prefecture you and your husband resided in (you had to have registered there in order to get married in the first place).
I'm not sure what German Laws require but here is some information that the American Embassy lists regarding marriage in Japan. You may need to check with the German Embassy in Japan to see if it is similar or if there are other processes that you need to follow to obtain the records pertaining to your marriage/divorce in Japan.
Marriage in Japan
Deutsche Botschaft Tokyo - Startseite
Perhaps your husband's family can help you in trying to obtain that information?