Just my own 2 cents...
I moved here from Germany where I had lived for 6 years. I did the whole rigmarole driving school and test in German - which is a nightmare to relate on a different day.
When I moved to the UK I went to swap my German permit for a UK one, they refused because I was American. Even though I was swapping an EU permit, they said it didn't matter if the person was a non-EU citizen.
Legal proceedings ensued, and finally they relented and swapped my license. And today, an American with an EU driving permit can trade it in for a British one.
So things are not as bad as they once were.
Apropos, by the way, the Lithuanian government (who are EU) is currently issuing driving permits for a cash payment of 500 euro. These permits are swappable for UK permits, and since this has been happening traffic accidents are going up up up. And we all know what that means for insurance prices