So - honestly, in my situation, do I really need to worry? What is the worst the U?S can do?
A really blunt reply:
If you had been born in the UK to a documented US parent (thereby becoming a US citizen); but never had your birth recorded at a US Embassy; never been present in the US; never been heir to a US estate or beneficiary of any US account; never had a US Social Security number; never had a US passport; never had an acquaintance who turned your name into the IRS to claim a reward; you
might be able to fly under the radar for the rest of your life, provided you never have
any contact with the US in the future. That includes being a passenger on an airliner bound for Canada, Mexico, or elsewhere that is diverted to the US.
Since you were born in the US and your birth was recorded (thereby automatically becoming a US citizen); you have had a US passport at one time; possibly have a US Social Security number; if male, possibly registered for the US Draft; and you lived a recorded life there (school records, drivers license, etc.) for 19 years; you may find on opening any new financial account in the UK (simple bank account for example) you may be required (by a decree of Parliament) to disclose your US citizenship. You may lie, but if found out it could have severe consequences for your financial future in the UK. If an employee of a UK financial institution has any reason to believe your are a US Person (by indicia such as a US place of birth) you will be classified as a US Person. If you cannot prove otherwise (by a CLN for example) the financial institution may be required to give your name to HMRC who will then forward it to the IRS.
What will the IRS do? I don't know.
You may want to research 'renouncing US citizenship', 'relinquishing US citizenship', the IRS New Streamlined Voluntary Disclosure Programme', or a reasonable cause plea. You may want to read IRS Publication 54, and the requirements for filing an FBAR. You may wish to investigate the possibility of HMRC collecting a debt in the UK for a US tax or FBAR claim; and if it's possible to collect both from a UK citizen resident in the UK (I don't know the answer to that).
I'm afraid no one can advise you as to what to do at this point, or whether you should worry. This is a very personal decision.