It's called a visa run, you numpties! It is not illegal, yet some countries do have limits to how many times you can enter the country per year, and stuff.
For example, in Thailand, you're only alowed to leave the country (say, to Cambodia) and return the next day with the free visa on arrival 3 times every six months. That means you can only stay 3 months in Thailand every 6 months. If you want to stay longer, you're going to have to visit the Thai embassy in the country in which you are doing your visa run in and apply for a proper visa.
There will be a Japanese embassy in Korea in which you can apply for visas or something. If you have a visa for Japan stamped in your passport BEFORE you arrive in Japan, you will not be refused entry.
But you're going to have to visit embassy websites to find out the exact details.
www.kr.emb-japan.go.jp/