Are you filing quietly or noisily? Why did you not file in 2004 if you had US source income that year?
I honestly just had no idea about it. I was there starting a PhD but I returned to the UK because I got mono so I was generally having a bad time that year. I didn't realise I would have to file as a student and assumed it worked the same way as in the UK (that any tax due would be removed from my funding at source - I was working as a TA) and that I would be below any thresholds for individual filing. I received no advice from the university.
I am now filing "noisily" I think - I plan to submit 2010-2012 under the Streamlined compliance program. I have no tax to pay for any of those years for sure (earned income is always <40% of foreign threshold, unearned income < 10% of standard deduction) so I hope I will be "low risk" although I do have bank accounts in different countries. So I am putting a cover letter explaning that I was resident in the different countries at different times (I never opened an account in a country I was not resident in at the time).
The main outstanding things I am not sure of are: how to handle my UK pension (only 4 months in 2012 but still would like to get it right), how to handle a large-ish moving allowance (I have just put it in the earned income, still safely within foreign exclusion limit), and for 2013 what to do about my 6 months of holding a UK Vanguard fund in a stocks and shares ISA (no gain).
I have to say on my second week of this now I am wondering if there is any light at the end of the tunnel! It seems unbelievable burdonsome - I don't mean the streamlined program, I appreciate I should have been filing and those earlier years would have been simple enough to file as I had no pension and no savings, but now I do it seems impossibly complicated just to do the most standard things - pensions, standard UK savings etc.
I have engaged an advisor but at some £150 an hour already 2/3 hours a month will absorb all my disposable income and it seems some of these issues (particularly the Vanguard fund for 2013) are going to take much longer.