Yeah at this time of year Didcot Railway Centre is only open w/ends and Wednesdays. There is another steam railway place at Cholsey and another railway museum at Swindon. Haven't been to either so can't say how good they are. There is also Beale Park (wild animal place) near Reading which is meant to be really good, again don't have personal experience of it. You should be able to find details of all these online though.
Awesome kids playpark in Henley too, we were just there this afternoon!