You only capitalize the word kindergarten if you are referring to a specific school. For example, you would capitalize kindergarten if you said that your child was attending James Woods Kindergarten.
1 Additional Answer
If you are referring to a specific kindergarten, such as Edmonton Kindergarten, then it should be capitalized. If you are referring to kindergartens in general, then unless it is as the beginning of the sentence it should not be capitalized.