Garden designer West said that plants were ready and he and herb grower McVicar were looking for another sponsor, but that he was "not holding his breath".
The RHS said the garden was still going ahead earlier this month (HW, 5 December).
Neal's Yard communications manager Simon Ford said: "When costing for each element came through, we decided the project wasn't financially viable. It was designed continuously and then we worked out the cost. Due to financial restrictions on the garden we weren't able to continue. But there's nothing like Chelsea for any brand that uses plant material."
Ford would not give the cost of the garden. Neal's Yard has opened five new shops in London during autumn 2008, taking the chain to 34 stores. Ford said: "In 2009 we will consolidate that. There's nothing in the pipeline for Chelsea in 2010, 2011 or 2012."
And Matthewman's Sweetpeas, Sheila Chapman Clematis and Squire's Garden Centres have also pulled out of Chelsea 2009 after many years of exhibiting.
Squire's managing director Dennis Espley said the move would save £30,000 and free up staff for the firm's 11 South East-based garden centres: "We're trying to conserve our marketing spend and give Chelsea a miss for a year or two and just do Hampton Court. The show can mean staffing issues so this will save money and make life easier.
"Personally, we all love going but you have to look at the commercial side. We might go back in a year or two," Espley added.