Actually, I'd be less worried about that. The same reason the cable will be fine also applies to the lamp holder, MOSTLY. The concern is that the earth resistance may be insufficient for 230V, but I suspect in most cases it'll be designed to be OK for that to make it possible to sell anywhere without modification. You'll need to do a lot of digging to confirm though, or find a friend with a Megger who can test it.
The part you will have to change is the bulb - that definitely won't be able to take 230V, and if it's a funny size could prove challenging, especially with the ban on incandescent bulbs over here.
Is the dimmer integrated into the lamp itself, or in the cord? If the latter, it's probably quite straightforward to replace. If not, that could also be a major problem (depending on the design used).