The work involves the cutting, chemical treatment and burning of Rhododendron ponticum and cherry laurel at Parkwood, near Parkmill, and Millwood, near Penrice.
The aim is to eradicate the two invasive species from the woods, to allow native trees and ground flora to grow.
Parkwood and Millwood are both important pockets of native woodland within the Gower Area of Outstanding Natural Beauty (AONB).
The woodlands are also part of the Gower Ash Woods Special Area of Conservation, a European designation to preserve and enhance significant biodiversity sites.
Forestry Commission Wales conservation manager Rosalind Codd said: "Rhododendron ponticum and cherry laurel are invasive species which shade out the ground and stop native trees and ground flora from becoming established, reducing biodiversity in the woodlands.
"Removing these shrubs at Parkwood and Millwood will help restore the native woodlands and preserve the special landscape character of Wales."
The shrub clearance work is part of Forestry Commission Wales's four year programme to control invasive species in the Gower AONB.
The work is being carried out by M and D Services.