1.The competitive nature of US railroad system
2.The vast distances between major traffic areas
3.Difficult terrain on locations (well this applies world-wide)
4.The differences between freight and passenger traffic
5.The lack of standards applied on selected routes. (or rather, there are too many of them)
Now, US is a railroad country. US has plenty railroads (wikipeedia states 226097km, but this would reach over five times round the globe so I doubt the number, possibly this is the total amount of installed tracks (incl. sidings). The routes are also plenty, see for example the route map of seven largest companies. Then there are plenty smaller companies some of which hold important connecting lines and get their revenue partly from trackage fees. Class III railroads do the same if they can, though many of them are more sort of a train assembly lines for larger railroads.
But if one wants to have intercity and long-distance passenger traffic, the distinction here would be that an intercity might travel by night but the long-distance has to, there has to be some common rules on the routes used. One simply cannot do a complete separation of passenger and freigth traffic on a country as big as US. Well maybe on selected corridors. The man on DailyKos (BruceMcF) has diaries that would convince me of his expertise. But all the same, on all locations these two cannot use the same tracks. Passenger rail depends on passengers thus it might be helpful to know where there are people. Wikipedia has a great map on this. The only problem with it, it hasn't got the terrain features such as Appalachians, Mississippi and Rockies (and Sierra) which present some great difficulties for rail transport and travel, if one doesn't mind the scenery.
Well I'm not that interested in getting involved with this (at least what it comes to US) so I just stack the images I made. These are probably nothing new for railroaders but might be interesting for those who don't know about anything of this ecologically pretty friendly way of travel (Click for larger versions).
Late addition: One may of course go into details, and find out many things are not quite that simple in practise:
which is near impossible to build without disruption to freight traffic.