I always go Amtrak if I can. It's not just because buses can be seedier, I like reading (which I can't do on a coach), and the pricing is about the same. It doesn't go everywhere though, and you can get delayed significantly if you have to wait a long time for freight to pass. But part of the romance of riding the bus in the US is the danger and seediness. I never really encountered the bums with guns that you did, but yeah, it's not always nice. If you think that's bad, you should see how my dad used to cross the US. He did the freight train thing back in the day, and it was dangerous. But romantic enough to helps spawn an entire school of literature in the US.
That said, I only had a car for part of my adult life. Granted, some of that was in NYC, but most of it was in cities that didn't have as good of public transport and in the country. It can be done, but it is totally not easy. The US (and Canada to an extent) are much more car-centred, but part of that is definitely because things are further apart.
That said, if a city has public transport, it seems to be better than what I've encountered in London, at least on an inner-city level. The continent is much better in that regard, at least in my experience, miles better than the UK and better than America as well.