If he says it won't last, then it won't last, sorry to say. Of course both partners must agree for a relationship to continue. If he is making you feel scared of loss, you should reconsider the relationship because it is not right for you really, whatever reasons you have for being involved still.
Your boyfriend is probably insecure that he might lose you and you should make him understand how much you care about him. On the other hand, he probably doesn't have confidence in the relationship and he does not see a future with you.