About 100 million of our 330 million residents have a USA passport. There are excellent reasons for this, including the low number of children who hold passports and the low number who have traveled abroad. Considering how it takes days to go from Texas to the West Coast, and hours to go from Germany to France, we have a much lower ratio of holders even after 9/11 changes. But the number has doubled since 9/11.
Well, since there is all types of places and climates in the US, most Americans don't have to travel abroad to visit a tropical places while people from England would need a passport to get to the Mediterranean or something.