Murphy has held the role for four years since Terra Firma bought the business for £276m in 2012.
King is vice chairman and head of portfolio business at Terra Firma. He has held previous roles at Marks & Spencer, Asda, Haagen-Dazs, PepsiCo and Mars, as well as Sainsbury's.
Wyevale posted results in what Murphy called "a challenging year for our profitbility" last month. EBITDA was down from £56.1m to £55m.
Wyevale is in a transformational phase to become more retail focussed after Roger Mclaughlan replaced Kevin Bradshaw as chief executive this year and King will see that through for the next year.
Murphy was previously group chief executive officer of the Virgin Group from 2005-2011. King met Wyevale staff at the annual spring show this February.
Two more Terra Firma personnel will also be joining the board: Alex Williams, a managing director at Terra Firma who is leading its refinancing negotiations, and David Hobbs, Terra Firma’s new strategy director.
A Wyevale Garden Centres spokesperson said: "Following the successful introduction of new executive management, and as Wyevale Garden Centres enters the next phase of growth, both Terra Firma and Stephen Murphy agree that, for the next phase of its journey, the business would greatly benefit from Justin's experience and knowledge of Wyevale Garden Centres.
"Justin has over thirty years of experience in retail and consumer goods businesses and a strong knowledge of Wyevale Garden Centres’ business model. In order to effect a seamless transition Justin will be taking over as Chairman with immediate effect."