[Troo]

Ideally, yes, you want to learn all possible readings of any given kanji. For many kanji you can only tell what word is intended within the context of the whole sentence, so if you can only read a handful of kanji you could get the wrong end of the stick.