It depends on the place. Private HS do cost money, because they aren't paid for by taxes, like public.
Some may be religious, and if they are, it's typically because it's a school from a church. For example, if there's a Christian church called "St. Mary's", there might be a school called "St. Mary's" which teaches its students about God.
In addition, sometimes there isn't as much extracurricular activities. Some do have clubs, but typically not as much as the public schools.
Private mainly is not open to the public, so not part of a school district. They mainly have uniforms and cost money to be enrolled into (we're talking thousands...4-5 digit numbers). And they mainly teach a sense of religion.