Brown has held the post since 2010 and his work is described by property owner Lady Emma Barnard as "outstanding".
Described as having a passion for innovation with an experimental planting style, Brown’s achievements include the re-creation of the double blue border within the Walled Garden as well as the introduction of Parham's Planting Trials.
The blue borders were re-planted with 3,000 new blue and silver perennials and annuals to re-create the romantic opulent borders that Parham is renowned for.
In 2014 Brown also introduced Planting Trials at Parham inspired by his previous work at RHS Wisley, Tulips, sweet peas, gladioli and most recently annual climbers have been some of the varieties trialled, to much interest from visitors.
The garden won a Gold award in the Heritage category of the South & South East In Bloom Awards in 2017.
Brown said: "I’ve really enjoyed the freedom to be able to create a garden and give it quite a unique identity, which I think is a lovely thing. There’s a danger that we may be moving to some kind of generic way of dealing with gardens now.
"I think head gardeners have a responsibility for a sense of place and to create somewhere that’s sensitive to the area. Staying somewhere for several years allows you to do that."
He said it was a good time to leave as his son was now reaching secondary school age and he wanted to find a new role in a place where the family could settle for another decade while he finished school.
Barnard said, "It is with great sadness that we say farewell to Tom Brown. We are hugely indebted to him for his outstanding work over the years. His great knowledge of all things horticultural has brought rigour, excitement, fun and beauty to the garden and has been a constant source of inspiration to all of us here at Parham as well as to so many of our visitors.
"Tom has helped us go from strength to strength during his tenure and whilst he will be sorely missed, we know that the gardens will continue to thrive and will always provide a haven of beauty and tranquillity for our visitors."
The garden is currently recruting for Brown's replacement.