Conifer sales rose by 10% to £15m in 2008, at odds with Swift's claim, which he made on Gardeners' World on 24 April.
HTA marketing director Andrew Maxted said: "One million pounds extra sales figures in 2008 shows Joe Swift was wrong."
The HTA will extend National Conifer Week this year to make conifers its plant of the month for October.
Maxted added: "I was very disppointed to hear Joe say that. He is mistaken — conifers are becoming fashionable."
New Hopetoun Gardens owner Dougal Philip said conifer sales were good at his West Lothian gareden centre, with a conifer garden being built at its Avant-Gardens garden deisgn festival, which opens on 9 May.