I like to buy the grape tomatoes for mine. She loves them. They like fresh lettuces and fruit too, plus hay and dry food. It seems like I buy more food for her than us. She eats good. There is also bedding hay to buy. Go to :http://www.happycavy.com/. It is a great website. There are great cages to buy (not cheap). Also think about getting a rescue piggie. Mine was at a store, brought in by someone who found it in their backyard.
Guinea pig itself is $20-$30. Plus the cage, well you can actually make one yourself as it saves money and gives your guinea pig more room to run around. The cages are called C&C cages. Hope this helped!