[chryuop]

I am one of those who is wasting a bunch of time memorizing kanji with their readings (I lost the count of people who tried to have me change way of study LOL). Anyway...kanji can become difficult to recognize because they are very similar. However if you get to know the very basic kanji (the one with very few strokes) you can recognize kanji easily.
I will give you an example that actually happened to me while reading some examples in a dictionary.
In an example I saw a kanji which was 教 and at first I thought it was the one for number since back then it was the only one I knew with that radical. But it looked a little bit different so I thought at how it was the kanji for number and realized I was wrong. What helped me was that I new number (数) has in it a woman and the rice (女 and 米).

I actually do not know the system you are using, but instead of trying to learn by heart what they look like, try to learn the parts a knaji is made of.
So for example if you find 休 you can remember it is made by a man near a tree (which gets you close to "rest") and if you see 体 think that it is made by a man and the kanji for base/origin (thus it helps you as the base of a man it's his body). Not all kanji might have a logical explanation (at least not to us foreigners), but if you know what they are made of you can recognize them. 鳴 a bird and a mouth, 聞 gate and ears and so on.

Some are more difficult and some are easy...then of course up to you to find the method which helps you more.