A PS Vita is bigger, weighs more, has a bigger display, a better processor, better graphics, more ram, has 3G and wi-fi. the biggest long term competition is with smart phones and tablets. Costs $249 for the wi-fi version, and $299 for the 3G/wi-fi option. A 3ds has internal storage, and an SD card that's included, wi-fi. Costs $169.
I would recommend the Vita, based on the specs. But if you're hard up for cash, the 3ds would do nicely.