It took us 1 month after our wedding to get all the paperwork together and in the mail for DH's green card. He got it 1yr 1month later. The first green card is conditional (expires in 2 years), so you can prove your marriage is real. We're renewing DH's green card this spring, and next summer we can submit his citizenship paperwork. We will have been married for 4 years, but the number they used it how long you have the green card for. Last I look into it, it can take 6 months - 1 year to get your citizenship after filing the paperwork.
We're thinking the same as you. As we will have been married for 4 years, moving back to England should use fewer steps.
As to the original question, once you decide to "live" in another country, the green card is invalid. You can cut it up or save it. As to your finances, I don't know if his SSN will be valid, our bank allows foreign residents to be on US bank accounts as long as they have an address outside of the US. I'd ask your bank what their specific requirements are.