Those who have reported seeing less wildlife has increased from 19 per cent in 2011 to 32 per cent last year, the annual Gardeners' World State of the Wildlife Nation poll found.
This is despite 93 per cent of gardeners taking steps to protect wildlife, up from 72 per cent in 2009 when the survey was first conducted.
Birds were spotted in 97 per cent of gardens, up from 92 per cent in 2011, with the next most common visitors being squirrels, frogs and mice.
And foxes have entered the top five for the first time, seen in 37 per cent of gardens - a five per cent increase on the previous year.
Bats are out of the top five, down from 39 per cent in 2011 to 34 per cent last year.
The migratory painted lady butterfly was seen in 42 per cent of gardens last year compared with 17 per cent in 2011.
But another more invasive visitor is the thriving harlequin ladybird, seen in 23 per cent of gardens, while parakeets were spotted by seven per cent of respondents and rosemary beetles by three per cent.
Gardeners' World magazine editor Lucy Hall said: "Wildlife appears to be relying more than ever on our gardens in times of hardship, so this makes it even more worthwhile that gardeners provide for wildlife, with uncertain weather ahead.
"It is encouraging that more of us than ever are taking steps to protect our wildlife visitors."