Roberts, speaking at a British Tomato Growers' Association conference in Coventry last week, told growers that he was uncertain how Big Society and localism - allowing local people to effectively run their communities - will work.
He said: "I am not sure I even understand it. I think instinctively we would all like to have a greater say in how Government is run but how is this going to be implemented?
"This Government has already taken away all of the regional development agencies, which were driving investment in their local communities, but these have been swept away with no solution as to what goes in their place. They swept away the old infrastructure without having a new one in place of it."
He added that the NFU was particular concerned about the impact that Big Society and localism will have on planning.
Roberts said: "Planning will fall into the hands of the local communities. It will be less easy for developers to get on with it.
"So we are pushing forward sector guidelines on how planning should be done. If not, the Big Society becomes a charter for nimby-ism and the needs of the industry will not be met."
As reported in Grower in June, the NFU is intervening directly in a legal battle over the use of polytunnels on a farm in Herefordshire after a Wye Valley local resident action group won a High Court challenge against a local authority's decision.
Roberts did, however, say in his speech to tomato growers that despite his concerns the new government does "on the whole" represent more opportunities to growers than challenges.
One of the reasons for this, he explained, was the fact that supporting and developing British farming and food production was on the top of ministers' list as part of Defra's current structural reform plan.
Roberts said: "The structural reform plan includes implementing the Taylor Review on scientific research in agriculture (which recommended more long-term investment in applied research) and we will press the Government to deliver on the review's recommendations, remove barriers to investment, encourage private-sector investment and promote levy boards."
Roberts also told delegates at the conference that the Government's need to make spending cuts could work in growers' favour because "regulation is expensive".
He said: "The Government needs to save money. It needs to find spending cuts and regulation is expensive. So there's slightly more opportunity here than there has been in the past but the key is to let them know where these regulatory burdens are."
The consultation is open until 31 October. Growers can submit their views online at www.tiny.cc/farmregulation, email farmregulation email@example.com or write to: Task Force on Farming Regulation, Area 8D, Millbank, Department for Environment, Food & Rural Affairs, Nobel House, 17 Smith Square, London SW1P 3JR.