Famous people should have every right to the same degree of privacy as anyone else. Being famous does not mean the general public should have complete access to every private area of a celebrity's life.
This is really a question of ethics. To turn the question around - is it ethical to allow access to peoples private lives, to go so far as to take away much of their sense of security, simply because they're famous? When the question's like that people have no choice but to agree that it's wrong. When you're stumped about things like this, or when you need to try and persuade people for or against something, always consider rephrasing the question. It's a powerful method if you're good at it.
When it comes to things like personal stuff (like medical issues) or as it pertains to their children. Thing is, they are happy to use the media when it is to their benefit. So, there's also going to be a down side to that.