Mr A told me last night that Lambeth supposedly has the highest crime rate for any council in all the UK (to which I replied "Good thing we don't live there"). A lot of people would probably list many places here in South London on the places not to live, but I love some of the places in Lambeth, Wandsworth, and so on. I absolutely love it here, and while I wouldn't choose to live in Croyden, I think it gets a bad rap as well.
Personally, I would avoid living in Westminster or the Square Mile again. Too expensive, crowded, noisey, and really difficult to get the things you need to run a household (surprisingly). Plus, where I lived near Victoria was rife with pickpockets because of the tourists. Sure seeing Westminster Abbey on a nearly daily basis and hearing Big Ben bong provided the wind was right never gets old, but a lot of the other stuff does.
I'm sort of glad we didn't seriously look in Luton and Slough though. We were considering it.