It really depends on the environment, training, and person. I personally think rodeo. You take at least 4 years raising your horse(most likely more), and years to fully train them. I feed my barrel horses twice a day, and ride them each for about a hour every day/every other day. It's a very time consuming sport. Not even counting training the horses, you spend a long time perfecting your riding style on each horse, and it can me harder then it looks.
Crew (rowing) it involves brute strength, mental toughness, and great endurance. It's very grueling on the body. Crew is rather painful at times (try like all the time) but a lot of fun. (In my opinion at least)