Tax implications for two dual citizens

I'm just about to apply for British citizenship.  The plan is to move back to the States in the next few years and go through the process for DH to have US citizenship, thinking we'll be free from all things immigration from now until the end of time!  But that means we'll both need to file taxes in both countries forever, doesn't it? 

The United States is the only country that taxes based on citizenship.

If you leave the UK permanently not intending to return here & have no UK income you would no longer have to file UK tax returns.

Your husband's estate would still be subject to UK inheritance tax however unless and until he relinquishes his domicile within the United Kingdom.

carrying on from Guya's comment...

...and the process of relinquishing domicile is easy to do; but you have to be gone from the UK for a while beforehand.  Without looking it up, I believe it's three years. Guya can correct me if I'm mistaken.