I've been here since April 2012. I'm originally from North Carolina. When I first moved, people here (in Norwich) seemed enthralled with my accent and would just say 'You're not from around here are you?' My accent has now started changing and becoming more British, so is much softer than it once was. Now I get the "you're Canadian, aren't you?" I don't mind the former, but the latter is just rude to me (not the Canadian bit, the assumption bit). As long as they mean well, I just try to take it in stride and say, 'Nope, North Carolinian originally, British now!" If they are being mean about it though, woo boy. As my husband (UKC) would say: "there is nothing like getting a Southern woman mad. It's like repeatedly poking a bees nest and wondering why you get stung!"
It takes a lot to get me mad, but if someone would have said to me what that guy said to you, I would have probably come back at him just as hard. Knowing me, I would have said something along the line of "I may not have grown up here, but at least *I* was taught manners, unlike some obviously." I've learned that *most* British folk do not enjoy being called out in public, nor do they like confrontation. If I have a problem, I will confront the offending party. They usually very quickly back down and apologise.
If I worked at a store and someone said something like that to one of my customers, I would have escorted him or her off of the property!