Maybe he thinks you're hogging his friend and he just wants some bro time. I would talk it over with your boyfriend and make it clear that you want to be with him and he should trust you. Trust is very very important. Maybe also, the friend is jealous because he likes you. That's a bad friend if you try to break up a relationship because you're jealous of the people in it.