I don't believe it was a habit that was "taken" from any one specific culture, but a habit that makes sense... as dogsbody pointed out, why would anyone want to carry dirt/mud/dog poops inside a perfectly clean house?

Exactly, in most public buildings you can keep your shoes on. Only in some restaurants you have to take them off.
However, I didn't visit every possible kind of public building.

You most definitely do not take your shoes off in most public buildings...
It is a home or certain restaurant sort of thing. The rule is more "Am I going to be walking on tatami?" - if so, then you will be taking your shoes off either at the door or at the edge of the tatami area. Most public buildings are not tatami, so taking your shoes off is fairly rare.
Another note - even if there are slippers provided, you never wear them on tatami. They are for walking in areas other than tatami, such as hallways, kitchen, etc. You will often see it written that you should always take your shoes off when inside, and that there are separate slippers for the toilet... But it is pretty rare to see the "no slippers on tatami (or carpet when in a private home)" thing mentioned.

We got it from being tired of having to wipe dog-shit and mud stains from our carpets.

Edit : Looks like my other fellow Canadian has said the same thing. Haha!

I don't have a specific cultural background that would make it so I had to take off my shoes, we just do it. As for public buildings, no we don't take off our shoes unless your at a doctors office or sometimes school buildings if your shoes are very dirty. It does indeed snow here but it is not cold all year round and we still take off our shoes in any weather. Like I said its second nature, and im not too sure about the other parts of Canada but seeing as people come to where I live frequently to find work, it seems like second nature to them too to take off their shoes in a home. The main factor is to not getting the floor dirty i suppose. Which makes sense cause I would not want to be cleaning up shoe marks from my floor everyday lol

I keep slippers by my bathroom door. But that bathroom is connected to my room so I only use it at night and when I shower and the slippers are really only for the winter since the tiles are cold. I also have a different pair of slippers I wear around the house but it's more so my feet stay warm.

Actually, in England, there are times when I've done the opposite and walked around outside barefoot! It's nice to do in the Summer heat, I think :p

(And I'm not the only one here to do it!)

Some Japanese companies also removed their shoes when entering certain buildings.

Japanese schools , traditional japanese ryokan, and some restaurants also will asked you to remove your shoes ..

I wonder if sometimes feet are rather smelly when shoes are removed?

I know that in schools, students wear what are known as wabaki (slippers/sandals) while on the premises.

I do remember visiting Ryoan-ji in Kyoto and I saw many sweat footprints from people on the temple's wooden floors....