If you really do not like him or the idea, it is better that you tell him off. Delaying will only send him the idea that you like him and he may never stop doing the same thing or even try something else. Only you have the power to make him stop.
Are you a guy or a girl? If you are a guy and another guy is trying to get close to you tell him you are not gay. If you are a girl depends on if he is a nice guy or a jerk. If all else fail just let him have it.