Depends on what kind of lifestyle you want. Those are very stressful, time-intensive careers, with a lot of responsibility, long hours, and high stakes. They also have the potential to make you a lot of money.
You need to think about what's important to you. Investment banking is good on money and prestige; not so hot on down time (or family time). Which isn't to say that you couldn't balance the two--just that it can be a challenge.