I hear it can be very expensive to stay in Kyoto and difficult to find anywhere central.
How about staying in Osaka and travelling to Kyoto there are lots of diff trains that do this for varying cost and speed, and tons of cheap central hotels in Osaka. I rate both
Hotel Kinki - Osaka business hotel with superb location, cheap rates and friendly staff
and
Toyoko Inn Umeda Nakatsu
If you do go to Osaka don't miss dotombori leading to namba, shinsaibashi if you get the chance and umeda for shoppings and sights. (If you have a little otaku in you or are just curious visit den den town, it is Osaka's answer to the crazy toy and video game fest that is Akihabara and really great!)
<3 for Osaka, it is a great central base for day trips to places and the shops stay open late so after a day trip you can explore Osaka itself before heading to bed! (also you can get the bullet train direct from (shin)Osaka to Tokyo
Himeji castle is amazing and IMHO once you have seen that you don't need to see many other castles as they just won't live up! It is a lovely walk up to it as well, if not a tad too touristy.