For the past few weeks I've been noticing an ever increasing number of wasps hovering through the garden. Mostly they're going after small caterpillars that have been nibbling on the Violets. Here it's hard to make out but this wasp is chewing up it's catch into a squishy ball of mush to make it easier to carry.
The general rule about wasp nests is if they're not bothering you, leave them there. Tolerate them for as long as you can, and they'll remove almost all of the pests in your gardens. This is bad for a butterfly garden but great for every other type.
Wasps and Hornets are mostly predatory. Chewed up pest insects they catch are fed right to the larva.
They don't alway make their nests in the right spot though. If you can catch it early on it's simple enough to dislodge a young queen's nest before she gets to far along. Usually she'll hang around it for a few hours before giving up and moving on to another nesting spot. This should be done as quickly as possible.
If they do become a problem though I can say small nests of paper wasps (the ones that don't get big and are only 40 wasps at most,) are easy to take care of. Larger hives such as yellow jackets, and bald faced hornets should be handled with care. Call a pest control agent if you don't feel up to it. Most sprays like Raid are great at firing the poison at such speeds that most of the wasps are completely blown away and taken out by the poison before they're able to fly. Again thought, this is a last resort.