You'd be crazy not to. Kidding....it's fun buying baby stuff. Don't hold back either. Babies grow so fast and mom&dad are so excited (maybe a tad nervous too). If u can't purchase or get to a baby store, another idea is to make them a home cooked meal. Maybe ur favorite cassarole or potroast anything to help make their new adjustment a little easier. Have Fun ;)
If you feel inclined to do it, then buy a gift. It would not be inappropriate. If you don't feel inclined, don't. However, contrary to the post above, it is NOT rude if you don't buy a gift. Expecting a gift is rude. If you are obligated, then it is not a gift.