While living in the USA we moved States 4 times, address 6 times, and never informed the IRS except on the next tax return so I don’t think it really matters. When we moved from the USA to England we did not inform them except when we filed our first tax return many months after we moved. (We are dual citizens).
Maybe we should have informed them each time, I don’t really know.