Here in the UK much of a qualified accountants workpapers are subject to professional privilege so the issues described are only rarely of any consequence (ie they only become material in criminal cases).
Fair enough, but aren't we talking about IRS enforcement and a potential prosecution in the USA? I'm thinking of the particularly aggressive sort of IRS investigations that we saw in the UBS case, and in others where the IRS has served a subpoena on an officer of a foreign bank who happened to be visiting the US, or more frequently on the US branch of a foreign bank. Here are some examples from a footnote from a scholarly article that also cites some UK cases: United States v. Bank of Nova Scotia (In re Grand Jury Proceedings), 691 F.2d 1384 (11th Cir. 1982), cert. denied 462 U.S. 1119 (1983); United States v. Field (In re Grand Jury Proceedings), 532 F.2d 404 (5th Cir. 1976), cert. denied 429 U.S. 940 (1976). Compare the issue of self-incrimination risk in the context of the "same-sovereign" rule: United States v. Balsys, 524 U.S. 666 (1998); King of the Two Sicilies, (1851) 1 Sim. (N.S.) 301, 61 E.R. 116; Rio Tinto Zinc Corp. v. Westinghouse Electric Corp., [1978] A.C. 547. And see R. v. Spencer, 31 C.P.C. 162, 145 D.L.R. (3d) 344 (Ont. S. Ct., 1983) (Crown witness subpoenaed to give evidence as to customers and transactions of foreign bank), Murphy v. Waterfront Commission of New York Harbor, 378 U.S. 52 (1964) (effect of immunity granted by a different sovereign).
There's another issue: when a foreign country uses a tax treaty to ask for assistance (i.e., seizure of documents or collection of information) from the treaty partner, the target taxpayer does not necessarily have the usual defences.
Only the OP knows the extent of his or her risk. My own advice would be to look for an attorney-CPA in the USA. There is a professional association of attorney-CPAs; some of them are well-versed in dealing with nonfilers and potential targets of prosecution.
As you must know the biggest risk is where a (serious) nonfiler discusses his or her options with an accountant and then, despite the criminal liability, decides not to file. The press has published IRS press releases about recent arrests and convictions; here's one from the other day:
http://article.wn.com/view/2010/04/23/UBS_tax_evader_cites_Holocaust_in_seeking_leniency/ What the press does not publish are stories about absconders who never again come within the jurisdiction of the IRS, and unlike, say, Norman F. Dacey
http://www.uniset.ca/other/cs2/TCM1992-187.html , don't have US assets or income.
And aren't in a position to get a presidential pardon like Marc Rich. Who still can't visit the US because of state-law arrest warrants and civil liability. But he can't be extradited either. (His major offence wasn't tax evasion but Iran sanctions-busting dating from 1979-80.)