Actually, men can produce milk if stimulated enough. :)
Beyond that, it's a part of the anatomy that's formed in utero before the definitive sex characteristics are determined, so both men and women have them, just like both men and women have estrogen and testosterone in their bodies. Up until a certain point, an embryo or fetus doesn't have any specific sex, just a prototype of either one, basically. The genetics and growth in utero determine what parts the baby has when it's born.