When I lived in a non-English speaking country, I thought the problems understanding English speakers after living there a while had to do with learning a second language.  I now think it has to do with rhythms more than anything.  
It's funny though, now when I hear American accents, they often sound fake.  My sister who lives in my hometown sounds like she's putting on a New York accent.  My sister from Maine sounds like she's pretending to be an extra on a Stephen King film with all the downeast in it.  And General American sometimes sounds like nails down a chalkboard (I never hear non-tourist Americans using it here, so none of you probably have it that strongly).  The recorded voices over the tannoy in JFK make me blush.  It's like a cross between General American and a southern belle.