depends. there are several types of annoying. there nice annoying bugging annoying and mean annoying. if they are nice then they probably dont know they are being annoying. if they are bugging then they know there being annoying and are bugging you. mean are like trolls and haters.
Well, Im no expert on having "annoying people" around me but, I would ask them nicely to leave, or say something like "I'm sorry, but I just can't hang around you anymore." It wouldn't be lying at least. ^^
Just ignore them. They'll keep trying to annoy you if they know they're getting to you. So if you stop responding to them hopefully they'll get the hint. If they don't, then be honest and straight up tell them that they're being really annoying and to leave you alone.