Can you press it at all with them? Explain that while it may be fine for the passport agency, it is also your legal ID, and you will be turned away from some places in the UK for not having your name in exactly the same format as is on your legal ID. I'd maybe say, they may say it is fine, but it isn't your name, and it was their error, so they should fix it.
I imagine most places would be fine with you spelling your name the right way even if they don't fix it, but there are some who like to be sticklers with the proverbial stick up . . .