You stop wearing linen typically in the spring and summer seasons. Usually during the summer and spring time, people are bringing out there more lighter wardrobes such as silks, halter tops and lighter cottons.
1 Additional Answer
Linen can be worn year round if you are in a warmer or tropical climate. It is very popular in warmer climates for suits and dresses because it provides coverage without the weight.