I think it's the hardest part. I grew up in a very close family and I've found that I've had to distance myself, almost make myself a colder person in order not to really miss them.
It's very difficult being away from family in good times and in bad. You miss out on births, weddings, birthday parties, children growing up, family jokes, illnesses, deaths, and funerals. The hardest, hardest part is when something goes wrong-a death or an illness and you can't get there. Can't just drive over and see for yourself that everything is okay. And if you do get there, it's a big deal. And you find yourself being the center of attention, when that'st the last thing you want.
And like Jemima pointed out, there's now an obligation to go visit when you go on holiday. And you sometimes find yourself lying about going other places so that you don't hurt people's feelings. Plus what your UK friends see as an exotic US vacation is actually staying in your mother's spare room with three kids.
Speaking of kids, there's the sort of guilt of taking your kids away from their family. They won't see them grow up like they would if you lived nearer-and in my case my mother sees them for a couple weeks at a time every two years.
It's hard to get home, the older your kids get-the fares are super expensive during the summer and 12 year olds pay full fares.
I don't want to sound like a complete downer, but for me this has been the hardest part, and I think sometimes we gloss over it and people who move over don't know what they're in for.