I moved to Canterbury a month ago from NJ on a spousal visa. I called the local Job Center Plus as a couple of days after I arrived and they gave me the information for applying for a NI number number. You don't actually need the NI number to start applying for jobs, but I think it's good to apply as soon as possible.
And yes, talk with some of the banks to see what their requirements are for opening a new account.
Canterbury is a cute little place. They call it a city, but it is quite small for a city, if you ask me. Because of the 2 universities located here, the cathederal, and the proximity to france, there are lots of young people, and lots of tourists. There are also many decent pubs and places to hang out if you are into pubs and such.
PM me if you need any other specific info.