Is it worth seeing ? That's a tough one.
It is worth it to walk around the grounds though it's VERY limited.
If the gardens interest you, you might be better off going to Shinjuku-gyoen which is now a park but was the original Imperial gardens.
Think it was 200 Yen admisson
Bear in mind these are from early March so it's not as green as it would be later on.