You need to practice saying no. You will probably feel bad about it at first, but after a while you may realize that by saying no you are giving other people a chance to participate and that the people who are asking you to do things don't feel offended by you saying no.
I know exactly how you feel. I do the exact same thing. This is why I was a member of like 5 banks. They ask and then I'm like well...ok. I've been doing something so that I wouldn't feel bad. Basically If I have any doubt about saying yes I say no. I now am more used to saying no and am now only a member of 2 banks. Haha. C ;
I was like you. My best friend told me the easiest way is to just say the word "no"....forget explanations, excuses, reasons, just "no". It's very hard to argue with and the more you practice the easier it becomes.