It's not totally unheard of to call it a lavatory on the east coast of the US, although it is probably a dying usage. It's weird that people find one of our words for toilet which pretty much all have roots that mean the same thing, that is a place to wash or fix yourself up, at some point in their history ("bathroom", "lavatory", "washroom", "toilet"). They're all euphemisms too. I am not entirely sure why I find it hard to say "toilet". It's not really because I find it too crude, which is why many Americans say they don't use the word. It might be that I find myself being self conscious about it, and that I am supposed to have a problem using the word as an American. I definitely know the weirdness of an American using the word "loo" is why I don't use that word in public.
It sort of reminds me of a teacher we had in school who called the bathrooms "basements". It took me about of week to figure out he was talking about the bathroom and not some cellar of the school I didn't know existed or that we could go into it. He's the only person I've ever met who called toilets that, but AFAIK it was used more commonly at one time to refer to specifically school or public toilets.