Yellow jackets are wasps, belonging to the same order as honey bees, ants, horntails, and hornets. They can develop pretty large colonies, especially certain species. They are cavity-nesters and will often nest in a burrow they find in the ground, or in man-made structures. Again this varies with different species. Some tend to build aerial nests under eaves or in tree branches. They can often find space, too, behind or inside old railroad-ties used for landscaping. This year a neighbor "discovered" them in his compost heap! You are very observant to notice the queen-like ones. Normally the colony is established in springtime by a single, fertilized female (queen). She lays a small patch of eggs in a tiny comb, which she constructs from wood pulp. Eventually the first batch of younsters hatches and they continue with the foraging and nest construction. Then the queen eventually settles down to egg-laying only, and the subsequent cycles of brood continue to populate and enlarge the colony. Some species will grow to a few thousand wasps; the population peaks in late summer. This is when most stings are encountered. In late summer/early autumn, the colony begins rearing "reproductives." These are fertile females (queens) and males. These are probably what you have seen around the nest. Each new queens will mate and find a place to overwinter, which she does individually. Sometimes you can find a young queen in winter, under some loose bark, hiding in a wall, or under a log. She endures the winter, all alone, and in spring searches for a new nest site and starts the process all over again. The non-reproductives (workers) and old queen perish soon after frost. So they won't be around much longer to bug you. However, towards the end of the season they get pretty aggressive, so be careful!