A statutory Plant Health Notice has been issued on the trees which requires the treatment and felling of infested material.
The caterpillars (larvae) defoliate and weaken oak trees, and are a hazard to human and animal health.
The trees near Cardiff Central Station will be chemically sprayed using an insecticide tonight and will be removed tomorrow.
Work will be completed by 12th July.
The first priority is to ensure containment and clearance of the disease however in the longer term we will work with NRW and Cardiff Council on the replacement of the trees.
OPM is established in most of Greater London and in some surrounding counties. Efforts are being made across the UK to minimise their population, spread and impact on the environment.
The affected trees in Cardiff had been imported from an area of Europe where OPM is widespread.
The Welsh Government is advising the public not to attempt to destroy OPM nests themselves as they and the caterpillars can pose some risks to human health. For more on how to identify OPM, visit the Forest Research website.
Anyone who has planted larger oaks imported from the continent, should urgently check the trees for signs of OPM and report any findings to TreeAlert.
Biosecurity minister Lord Gardiner told an environmental partliamentary committee this week that the government's biosecurity strategy should "be more ambitious - there is a lot more to do. More resource is needed to be put into this major contribution to environmental degradation."
He told committee chair Mary Creagh MP: "Yes, we have an ambitious spending review request because we think this is a very important area," adding: "We haven't candidly had the resources animal health has had."