Lifestyle sales dropped -2.4 per cent compared to May 2015 and fashion was down -1.9 per cent.
Sales of homewares grew by just one per cent year-on-year and non-store sales rose 18.5 per cent.
Aside from an uplift during the January sales, BDO’s HSST figures have shown negative growth since October 2015.
Consumer confidence has been knocked by a combination of factors, including uncertainty caused by the June referendum on the UK’s EU membership, and a recent report confirming the pressure on household budgets is as fierce as it has been in nearly two years.
In contrast, garden centre sales took off after a slow start to the season and many garden retailers recorded record days during May, as plant and outdoor living sales took off thanks to better weather and pent up demand.
Barclaycard found that in May garden centres enjoyed a 9.4 per cent upturn in consumer spending, while DIY stores saw growth of 4.8 per cent.
Barclaycard said high street spending rose by 0.7 per cent in May, while total consumer spending climbed 3.6 per cent.
Clothing retailers made a 'big comeback' last month and across all categories, total sales grew by 1.4 per cent, the British Retail Consortium said.
But Sophie Michael, head of retail and wholesale at BDO LLP, said May had been a bad month across the whole high street. "A fall in retail sales is often sparked by consumers choosing to spend their disposable income on things like eating out, but the evidence is that people are reducing spending across the board.
"Things have been tough for retailers since the end of last summer and there’s not much to suggest confidence will pick up soon. Retailers will have to be strategic with their discounting and confident their product is strong enough to appeal to their customers when they decide to spend."