I believe so, though it'd be better safe than sorry. You should probably have at least one proof that the child is yours with you. And if your last names don't match, bring the birth certificate or a letter from the other parent (I've known people with this problem on cruise lines, since they gave their kid the father's name they had to prove it's their kid.)
It depends on the country they are traveling to and from. My sister and her husband are US citizens living in Mexico. When my sister came home without her husband for a vacation, she was told to get legal papers drawn up to show she had his permission to travel with their daughter.