Diarmuid Gavin's garden builder Dermot Kerins said on Sunday: "It won't be going up today. If the wind stays like this it won't fly I'm afraid."
Gavin's crane is believed to not be allowed to operate if winds gust more than 20mph. Gusts reached 30mph in London yesterday and the Met Office says strong southwesterlies will blow until Thursday.
The show runs at the Royal Hospital in Chelsea until 28 May.
RHS representative Phil McCann said: "It's up to health and safety. We can't take any risks."
Meanwhile, B&Q's 9m tower garden, the tallest structure ever built at the flower show, was also being buffeted by the wind.
B&Q designer Patrick Collins said: "The wind and sun have been quite a challenge but the tomatoes are sheltered by the tower and are holding up well."
RBC Canada rain garden designer Nigel Dunnett Dunnett said the gusts were likely to mean some gardens would receive lower medals than if it was calm: "Alliums for instance are very vulnerable and we don't want to put stakes in because they would look awful when it comes to judging. The heat we can deal with but there's little we can do about the wind.
We're at the stage where we've used all the spare plants up."
One designer who was pleased about the weather was 'Winds of Change' garden designer Jamie Dunstan, who features wind turbines on his Chelsea plot.
Dunstan said: "It's an advantage for me definitely. The turbines will be turning and Monday is forecast to be very windy, which is good for the judging. But it won't do anyone else any favours."
The RHS releases results of judging on Tuesday morning.
Mark Fane, founder of online garden centre Crocus, who supplies more gardens with plants than any other single grower, said:
"For me the wind is the biggest issue. It's more dangerous than the heat. Allliums and irises are fragile plants and are really taking a battering. It's been very difficult this year - the most difficult in the last 20 years. Four months ago we were sweeping snow off polytunnels and now it hasn't rained in two months, then the wind struck. We've had to do a lot of changing of plants."
Designer Robert Myers added: "The wind is the worry factor. You don't know what's going to come down from the trees and squash the plants. And it makes the pollen worse, which is no good for anyone's hayfever."