I grew up in San Jose!
I just came back from a visit two weeks ago.
If you have a car it will make getting around a lot easier, but there is Light Rail and buses if you are central.
Happy Hollow is an old family favorite which has been majorly renovated in recent years. Day entrance is about $12.95 or annual membership is $35.
http://www.hhpz.org/In the same plot of land there is also the Japanese Friendship Gardens (free) and History San Jose (Tuesdays - Fridays free).
http://www.historysanjose.org/wp/ Bring a picnic and enjoy the outdoors. Likewise there are parks dotted all over San Jose so depending on your location you should easily find greenspace.
The Children's Discovery Museum is a good place to take kids too. $10 each
http://www.cdm.org/index.asp?f=1There is also a farmer's market downtown on Fridays.
http://www.sjdowntown.com/FarmerMarket.htmlSJ libraries are also good. The one downtown is large.
Then of course the shopping malls, movie theatres... Target! The usual good old American stuff.
On the weekends in May and June when you will be there, Bay Area cities host free arts festivals.
I am happy to answer any questions.