~Hi TK,
I am SO glad you don't want to eradicate the carpenter bees on your property! Many homeowners despise them for the damage they can do to wooden decks, swing sets, sheds, etc. Yet this fact remains: they are excellent pollinators and steps can be taken to keep them away from property.
Place your pollinator garden far away from any wooden items you don't want damaged and provide them with plenty of wood they CAN use like assorted sizes of logs and branches placed under bushes and in waste areas and brush piles. Mounds of rock or dry stone walls and a few random patches of bare earth provides habitat for all kinds of native bees.
Milkweeds are a favorite( I posted a photo of Asclepias incarnata recently in the database of a male carpenter enjoying himself)
If you have room, bushes are a perfect and much needed addition to pollinator gardens. In particular a top choice of the bees on this property is Hydrangea arborescens but ONLY use types that have "fluffy" petals like: "White dome", "Haas Halo", or straight species "Wild hydrangea" Also, Buttonbush(Cephalanthus occidentalis 'Sputnik') I watch the bees literally run through the hydrangea fluff like children playing in a bubble bath!! Perennials on the list: Culver's Root, Black Cohosh, American Wisteria, and Lupines. Because Carpenter Bees are so heavy and strong they seem to prefer flowers they can hang from often using only one or two legs!
Sorry for the winded reply-- but I am passionate about restoring health to our endangered pollinators.
Best of luck~ Cheryl