I don't think there is a right time to take your tree down, excpet maybe January. If you are planning on having any New Years events and your tree still looks good, keep it up until after. If your tree is turning brown and the needles are all coming off, then the day after Christmas would be a good time.
That is completely up to you and your family's preferences. Some people do take their trees down the day after Christmas and other people I have seen leave their trees up past New Year's. There isn't really a right or wrong answer, just don't leave it up until July or anything.