The PC keyboard layouts have always been something of a mystery, and to me it seems as though IBM has never quite gotten it right for everyone.
Many older ASCII keyboards had " over 2 and ' over 7 to match up with some typewriter layouts and for hardware reasons (if you look at the ASCII table in hexadecimal or octal you'll see why). This goes right back to the electromechanically generated ASCII on old teletypes.
The original (American) IBM PC keyboard in 1981 adopted @ over 2 and " over ' (middle right) from the standard IBM Selectric typewriter layout. That's understandable given the original market for the PC and the fact that the Selectric had been widely used in offices for a good many years. Why they decided to swap @ and " for the U.K. version of the keyboard is unclear though, since the Selectric was common in Britain too.
That backslash key is another oddity. Both the original British and the original American PC keyboards had it located between Shift and Z. Nobody seems to know why they chose such an odd place. The awkward position was not popular, so when the revised PC/AT keyboard was launched around 1983 or 1984, the \ key was relocated to the upper right on the American version, but for some reason they left it next to Z on the British version, where it has been been ever since.
The enhanced keyboard (now the "standard" one with the extra cursor and function keys) was much improved in its U.S. version, but made the mistake of moving the Ctrl key below Shift. It probably isn't that big a deal for the average modern-day computer user, but it seems a peculiar move for the mid-1980s given that a generation of programmers using IBM machines hooked into mainframes made heavy use of the Ctrl key, as did all the office secretaries who were touchtyping with WordStar and were used to it being somewhere to the left of the A key on just about all other keyboards.