It depends how "benefits" and "Contracting State" are interpreted, I think.
The Technical Explanation for 1.5.b says:
The benefits referred to are all intended to be granted to temporary residents of a Contracting State (for example, in the case of the United States, holders of non-immigrant visas), but not to citizens or to persons who have acquired permanent residence in that State. If beneficiaries of these provisions travel from one of the Contracting States to the other, and remain in the other long enough to become residents under its internal law, but do not acquire permanent residence status (i.e., in the U.S. context, they do not become "green card" holders) and are not citizens of that State, the host State will continue to grant these benefits even if they conflict with statutory rules. The benefits preserved by this paragraph are: the host country exemptions for the following items: government service salaries and pensions under Article 19 (Government Service);
In my case, the US isn't the host country, because I'm resident in the same State that pays the pension. If I were US tax-resident (and still a US citizen), I'd lose the exemption.
There's a paragraph in Pub. 901 that perhaps makes the intention a bit clearer:
Tax Exemptions Provided by Treaties
In addition to the tables in the back of this publication, this publication contains discussions of the exemptions from tax and certain other effects of the tax treaties on the following types of income.
- Pay for certain personal services performed in the United States.
- Pay of a professor, teacher, or researcher who teaches or performs research in the United States for a limited time.
- Amounts received for maintenance and studies by a foreign student or apprentice who is here for study or experience.
- Wages, salaries, and pensions paid by a foreign government
They don't want fear of double taxation on pay or pensions to put people off from coming to the US (or the UK) on a temporary basis, (as teachers, students, and government workers often do), but equally they don't want them to remain exempt if they are or become citizens or Green Card holders. Is my guess.