[sarasi]

Depending on where you will be wearing it, yes, casual clothing is fine. Japanese people don't wear suits everywhere, only if they are going to work in an office.