Men like home cooked meals, back rubs, their beer in your fridge, watch the game with em, a pair of lounge pants at your place, a bag of goodies - razor, tooth brush. Listen to their day after he has relaxed a bit. If it's only physical then these things aren't happening either. He wants all of it, physical LAST.
He might be saying one of many things; 1. Maybe he doesn't trust you, or seeing you going anywhere in the future 2. Maybe he sees the relationship he is in with you, as childish, and he feels he wants something serious. 3. He has different ideas of a relationship than you do & he doesn't feel it would work. etc. There is obviously something going on, that makes him feel that your relationship isn't normal, or worth trying & he wants to find someone he is more compatible with.