Wearing shoes and socks constantly never letting your feet breathe isn't good for your feet. When you walk barefoot this gives your feet time to form to their natural self. The foot will spread out and tone up the muscles.
Your feet will still grow some with or without shoes.
Japanese women will bind up their feet very tightly too keep their feet small.
The main determiner of foot size is genetics. So, going outside without shoes shouldn't have an effect on the size of your feet. What Rheada says is partially true. By wearing shoes that are too small for you, your shoe size can be inhibited. But, that probably means your toes and feet will be scrunched up instead of growing in a healthy way.