"Discussing the stories behind plants is what I enjoy doing most," he says. He cannot look at a plant simply as a beautiful object - for him it is an opening to discuss where the plant grows in the wild, how it came to be collected, by whom and when, and how it is cultivated. Not bad for someone who confesses that "geography and history were my two no-go subjects at school".
He is currently working on one of his own stories, revising his book Travels in China. Of the first edition, published in 1989, he says: "It broke my heart to see hazy pictures in black and white of the people and places I had visited." So he is excited that the book's 1,000 images have been digitally remastered and will appear in colour. This edition will also include new images of Chinese plants that have been established in cultivation since he first went to China. "It's been a chance to relive the seven adventures I had there in the 1980s."
Lancaster has another book ready to emerge, one that's been in the making for many years. The subject is his own garden in Hampshire, which covers only one third of an acre (0.1ha) but contains 1,000 plants. He says: "It will give me the chance to write about gardeners connected with the plants who you don't hear about, the ones who hide their light under a bushel or whose light burned brightly for a short time."
Lancaster is not in the public eye as much as he used to be, having given up regular broadcasting, but he continues to write and give lectures, often abroad. This year he will lecture at the Scott Arboretum in the US, followed by a holiday there with his wife, Sue. She probably needs a holiday - as Lancaster remains endearingly attached to writing his work by hand, Sue types up all of his copy, including all 500 pages of Travels in China. Not surprisingly, he calls her "100 people in one" when describing their "great partnership". She will accompany Lancaster to China for the first time at the end of this year.
He takes an ambassadorial role for several of the major horticultural organisations including being vice-president of the RHS and a member of the National Trust gardens panel - and he relishes part of his remit being to "visit gardens and encourage gardeners".
He is increasingly conducting Plantman's Days, when he leads a tour around a garden. "I say to the group when I start - this is not so much a practical lesson as an adventure. You decide where we go. Then on the walk we stop and talk about the plants we come across."
He recently took a group of National Trust gardeners around Sheringham Hall, Norfolk. He was struck by how "everyone began to share their experiences of the plants, so it was not just me being the Pied Piper". "It was brilliant," he says. "It's important, drawing out the spark that everyone has in them, so that they believe that what they do matters."
Lancaster's early enthusiasm was shaped by the two park managers who took him on as an apprentice aged 15 and tutored him in Latin names half an hour before work every day. Within months, a passion for plants had developed, with him using his 3km walk to work as a challenge to name every plant in the front gardens he passed. A traineeship at Cambridge University Botanic Gardens followed, which he says was the "turning point of his life - I had a wonderful time, learning on the hoof".
He established his reputation after joining Hillier, by becoming curator of the Hillier Arboretum. "I am so pleased to have a permanent connection with Hillier, being patron now. I can't forget my time there." Lancaster went freelance in 1980 and began the successful career that continues to this day.
It would be easy for Lancaster to sit back and enjoy the efforts of his labours. But he still helps to motivate the next generation of gardeners, inviting courses from Royal Botanic Gardens, Kew, RHS Wisley and the English Gardening School to meet him every year. "You don't know who you might get through to. If one person says 'yes, that was worth it', it's a wonderful feeling."
1953-1960: Bolton parks department
1960-1962: Trainee at Cambridge University Botanic Garden
1962-1980: Hillier catalogue compiler and horticultural botanist; Hillier Arboretum curator
1971: Involved in putting together The Hillier Manual of Trees and Shrubs
1980-date: Freelance lecturer, writer and broadcaster on shows including BBC Gardeners' World and Radio 4's Gardeners' Question Time
1981: Publishes Plant Hunting in Nepal
1988: RHS Victoria Medal of Honour
1989: Publishes Travels in China
1996: Institute of Horticulture Outstanding Service to Horticulture Award
1999: OBE for services to horticulture.