That's pretty much the reaction most people have
chryuop is right about the two main ways of learning and different people will give you different answers.
I clearly belong to the first category. We tend to forget what we don't use, especially abstract readings that are not connected to any concrete image. I find it much more effective to learn kanji in actual context.
I would also like to add that learning the meaning is as important as learning the pronunciation (if not more). It will help you understand some words you see for the first time, because you know the meaning of each character (sometimes even only one). It's hard to explain that to a beginner but trust me, it will develop your "kanji sense".
After some time studying the language, you will start seeing the same kanji used in different words, usually with related meanings and this will help reinforce what you will have learned.
Again, this is a personal opinion and not everyone will share it.
Don't think of it as a daunting task. It takes a lot of time and effort, but it pays off in the not-so-long term
.