So, if someone born and raised in the UK, went to live in the US for 3 years or more, but then moved back to the UK, they would have to pay international fees if they wanted to go to university.
Not necessarily.
If they moved to the US without maintaining their Ordinary Residence in the UK,
any length of absence would be a problem, not just 3 years or more.
Ordinary Residence is different from physical presence, and in practice many people who move away from the UK for a while do maintain their Ordinary Residence in the UK throughout the absence, even if they are away for a number of years or, occasionally, even a decade or more.
For example, Ordinary Residence in the UK is not broken by moving away from the UK for employment on temporary contracts (including military posting), for study or for "gap-year"-type travel. You are automatically considered to be Ordinarily Resident in the UK throughout your absence.
Even if your absence doesn't come under the above exceptions, it is still possible for you to maintain Ordinarily Residence in the UK if you can show that you maintained a "regular and habitual mode of life" here throughout. This is normally shown by regular physical presence. Things like owning property, maintaining a bank account, having relatives here, being British or having ILR can be part of the overall picture, but do not on their own mean you were Ordinarily Resident. Case law shows some surprisingly small amounts of physical presence have been accepted as maintaining Ordinary Residence. UKCISA has a helpful
summary of the case law for tuition fees and student support.
In OP's case of course, he appears to have no previous residence in the UK anyway, so his Ordinary Residence would only start when he moved here.