Virginia has a set procedure for change of domicile.You first approach the state dept. of revenue, and a form is sent. You fil in the form and send with all the "proof" they require that you no longer live in VA or have ties to the state. In our case, hubby had only been in VA since 2000 when we moved to the US so that I could work (long story). At the time, I had ILR here and was a US citizen only; hubby was the Brit, and applied for a green card, and became a US citizen in 2010.
Fast forward to 2011 when we decided to return to the UK and I started the visa process again. Hubs was retired and I was semi-retired (I did consulting work, and did one assignment in early 2012 before calling it quits. In late 2013, we initiated the request for an official ruling on change of domicile.
We had to prove that we had no ties at all to VA - no property, no vehicles registered in the state, no drivers license (easy as VA requires that any change of address is reported within 30 days, and automatically cancels if you report a foreign address), no financial accounts (we kept a US bank account, but changed our address to the UK address), cancelled voter registration (this doesn't affect your right to vote in federal elections by absentee ballot), affirmed that we had no children in any public school, paid taxes in the UK, were on the electoral roll (applied to hubby only initially), possessed a UK drivers license, and proved ownership of a vehicle here, and a pile of other papers, which thankfully they allowed me to scan to PDF and email. It then took over a year for VA to sift through the "proof". By this time I was in my mid-60s, and it helped that I had lived outside of VA for over 50 years and was born in a different state. I also submitted additional evidence of change of domicile when I applied for British citizenship in 2014.
We finally got an official letter stating that our request for change of domicile had been approved, but admonished us that should we decide to return to VA, we had to notify the state.
Every state with a state income tax likely has a similar procedure. Interestingly, if you move from VA to another state, they still TRY to hold onto you if you have any ties. In the past, failure to cancel a library card was considered a "tie".
Were I you, I would start researching what you need to do. California may not pursue you, but you never know. Good luck.