You need to find the nest and then get some Hornet/Wasp spray at home depot. Ive had this happen twice but luckily the nests were on the outside. Look around the eaves and window sills of your house, they can fit into pretty small spaces. If all else fails call a professional but they are expensive. Good luck!
If you don't know where the nest is right now, watch them at dusk, they will begin to return to the nest.WATCH THEM! Once you locate it, use wasp killer on the nest in the dusk or early morning when they are the least active.
And before you go to treat the nest, go to a hunting supply store & buy the veil they sell to keep bugs from biting. Put gloves on with a long-sleeved shirt, socks & long pants, & duct tape where those come together, so they can't crawl into your clothes. Better yet, get an exterminator--it'll save you a world of trouble!