I don't know how cheap you want...but I get all my stuff from Homegoods. Excellent quality stuff for great prices. I LOVE that place. Also, Target has wonderful home decor. It's so trendy and well priced. I find Walmart has ugly things and Ikea is good but things tend to break easily.
Depending on where you live, but the Dollar Tree seems to be the best for indoor decor; and the best thing is, everything is 1$ or less. If you do not have a store near you, you can always order it online from dollartree.com. Hope this helps.
Tuesday Morning has cute stuff sometimes, but not always cheap....I also like Hobby Lobby...shop their sales (for most home decor items if it isn't on sale this week, it will be next week). They rotate between Ceramics, Metals, Glassware, and Wooden items as to what is on sale.