Well, I did my PhD in Swansea, just down the road. I can't tell you much about the university, though the library is pretty decent. (Better than Swansea) And it's a nice location, central to the city. What are you planning on studying? I think the experience really differs from department to department, and what course you'll be on. I ended up at Swansea b/c of supervision, but would've loved Cardiff. (Mind you, I'm soooo glad I went to Swansea - I met DH there!)
Cardiff is a great city. It's not just a good-sized city, that's fairly up-and-coming, with good entertainments and shopping (and jobs I've been told - but better if you have Welsh), but it's really vibrant as the capital. There's nothing more exciting than being in Cardiff for an international rugby game. It just comes alive. Otherwise, like PP said, it's got great transport links - though Cardiff intl airport doesn't do long-haul flights, and you'll still need to go to London (or possibly Bristol, v. nearby) to fly to the US.
The Welsh language won't really handicap you at all. I studied it for three years here, as part of my research, but everyone speaks English. The accent is no harder to cope with than any other regional one, and everything is written bilingually, like with spanish in the US. The only thing I'd say is that if you're looking for a professional job in any public/cultural sector, you'll need the language - at least teh basics. But you can easily study it thru the university's adult learning centre.
I think it'd be a great place to study - good luck!!