If you do a google search, or if you just go to the VisitScotland.com website, you'll find loads of self-catering flats available. Keep in mind big events like the Tattoo and the Festival and all that, because it can seriously affect availability, and also prices are often higher during those periods. Also, keep in mind that if you're staying anywhere along or close to the Royal Mile, places are probably going to be fairly small. That wasn't a problem when my boyfriend and I went last April, but I'd recommend making sure there are good photos of the property available.
As for stuff to do with preschoolers, I'd definitely recommend the Edinburgh Zoo. It's a very nice zoo, and they do some good programs. Don't miss the penguin parade!
Beyond the zoo, though...I suppose it depends on your kid. Would s/he think castles and museums are really cool, or be bored stiff? Would s/he be able to keep up on one of the walking tours? There are a couple of other things that my boyfriend and I didn't see- the Dynamic Earth science centre, the Loch Ness Experience (some kind of 3D film)- that might be of interest.