Hello there
I live in Surrey, not too far from you in Camberley. Milford is a beautiful place to be in the summer. You will be very near some of my favourite village pubs! Make sure to spend as many sunny afternoons as possible in the garden at Elstead Mill. You'll also be near Tilford which has a lovely cricket green and (of course) pub on the green called the Barley Mow. Definetly go watch the cricket at the weekend.
Re: outdoor festivals, there is a music festival in Guildford called
Guilfest, which is only 20 minutes in the car or maybe 15 by train from you. Its the 15-17 of July.
You'll miss Royal Ascot, but you wont be far from Ascot for other races and events at the race course thru the summer.
The other great thing about all the villages near you is that they often hold their own festivals or fetes in the summer. Keep an eye on signs and community boards for these types of things.
Re: restaurants - I've never eaten in Milford, the Thai Terrace in Guildford is very nice as is The Donkey in Tilford.
Let me know if you have any other questions - I'm very happy to help.