Congrats!
Reason he gave you is also because you won't have any identifying status documentation other than this really large, very formal certificate you get at the end of your ceremony declaring you are now a US citizen. I think they take up collection of your green card when you check in for the ceremony. Ergo, the best thing you can do is get that US passport right after (they will have a booth explaining the benefits and to pick up forms), so you have immediate proof of citizenship, without having to get the giant certificate out. Just have the forms filled out and ready to be mailed the day or two after, though I seem to remember that the Swede took his certificate to the post office, they said it was ok, and then he didn't have to mail it with the application.
Also, as a citizen, you have to enter the US on a US passport now, but the UK on your British one. Welcome to the fun world of dual passports - nothing like having an extra one to worry about and possibly lose!