"When I was a child I would spend lots of time with my grandmas," he says. "Sometimes I would selfishly think spending time with them could be boring but when I got them on a subject like living in London during World War II when bombs were raining down, they would become very animated and I would be enthralled. I realised everyone has a story to tell."
And there are some cheeky elements of their characters in Gangsta Granny.
"There was definitely a smell of cabbages in one of my grandmas' houses," David admits. "The other did break wind like a duck quacking when she walked across the room."
But they were also greatly loved – just as there is a special bond between Ben and his gran.
"I think grandparents love being grandparents because they get to give the children back to the parents," says David. "Children love spending time with their grandparents because they love hearing their stories and being allowed to stay up past their bedtime."
"Ten years ago I had an idea for a story," he recalls. "What if a boy went to school dressed as a girl? I thought it would be a thought-provoking children's book. That became The Boy in the Dress, my first of eight children's novels."
Those novels, including Ratburger, Demon Dentist, Mr Stink and Billionaire Boy, have all been hugely successful, topping charts and winning a host of awards. David now has more than 6 million book sales to his name, with his stories translated into 40 languages. His most recent book, Awful Auntie, was the fastest selling hardback of 2014 and went on to win the National Book Awards for Children's Book of the Year and Audiobook of the Year.
Writing for children is a real passion for him.
"The only limitation in a children's book is your imagination. You can take children on magical journeys in books that many adults would be reluctant to go on," he says.
"Children love to be scared but it can't be too horrifying. Children love to laugh but it can't be too rude. You always have to be the right side of the line."
David has frequently been compared to Roald Dahl, his own childhood writing hero.
"I think Dahl's books always feel a little bit forbidden. He manages to balance the humour and scary elements in his stories perfectly. My favourite is The Twits which is utterly hilarious and I love that it is a children's book with no child characters."
David was also a fan of American writer Dr Seuss.
"I loved Dr Seuss books as a child, especially Green Eggs and Ham. His books are like nightmares come to life. They are rich and strange and utterly unlike anybody else's work."
Keen to ensure his own novels prove to be just as memorable for children, 44-year-old David believes his popularity is down to the fact his books are laced with humour and never patronise youngsters.
"I deal with quite big topics - cross-dressing, homelessness, grief etc," he says. "I know children are a lot smarter than most grown-ups think."
Two years after it was published, the BBC made a film version of Gangsta Granny which was shown as part of its Christmas schedule. With Reece Buttery as Ben and Julia McKenzie as Granny, its star-studded cast also included Joanna Lumley, Rob Brydon, the singer Robbie Williams, Miranda Hart as Ben's mum and David Walliams as his dad.
After the success of the television film, it seemed only natural that Gangsta Granny should become a stage show so David joined forces with Birmingham Stage Company. With a string of Roald Dahl adaptations under its belt including James and the Giant Peach, The Witches and George's Marvellous Medicine, BSC has a strong track record so was the obvious choice to bring Gangsta Granny to the theatre.
Birmingham Stage Company has built a strong reputation for quality children's theatre since being formed more than 20 years ago. As well as Dahl classics, it has also staged a number of Horrible Histories adaptations, The Borrowers, Skellig, Tom's Midnight Garden and The Jungle Book.
David has plenty of stage experience having taken roles as diverse as Foster in Harold Pinter's No Man's Land and Bottom in Shakespeare's A Midsummer Night's Dream. So he needed to ensure his Gangsta Granny was in safe hands.
It was after seeing BSC's Horrible Histories that he decided it was the theatre company to take on the stage adaptation.
"Their Horrible Histories show was superb. I loved the humour and the interaction with the audience so I look forward to seeing more of that. Productions for children need to be fun and fast-paced. I think from pantomimes, children like audience interaction too. They like to be involved.
"It's a huge thrill seeing Gangsta Granny have this whole new life on the stage. There is lots of action in Gangsta Granny, especially when they try to steal the Crown Jewels. The challenge will be bringing those scenes to life."
David is looking forward to watching some of his characters take shape.
"I like the characters of Ben's mum and dad. They have an obsession with ballroom dancing which should be funny so I can't wait to see them brought to life. The great thing about seeing Gangsta Granny on stage is you will get to share it with an audience. So hopefully you will laugh and cry along with everyone else. That's what makes theatre so special."
It's been a busy year for David. Not only has he been working with BSC on Gangsta Granny but he also took the role of Agatha Christie's detective Tommy in the BBC drama series Partners in Crime.
David's varied career has seen him in films such as The Look of Love with Steve Coogan, Stephen Fry and Anna Friel and Run Fatboy Run with Simon Pegg and Thandie Newton. He played Frankie Howerd in the biopic Rather You Than Me and has written and acted in the sitcom Big School with Catherine Tate.
A trustee of BBC's Comic Relief, he has undertaken a number of challenges for the charity including swimming the English Channel in 2006 and the River Thames in 2011.
In 2012 he became a judge on ITV's Britain's Got Talent. So, bearing in mind the obsession Ben's parents have for ballroom, has David ever considered donning his rhinestones to enter the television competition Strictly Come Dancing?
"I can't dance at all (as you might have seen in the TV adaptation of Gangsta Granny when I tried to dance with Miranda Hart). So I would say my chances are less than zero," he laughs.
David is hoping BSC's Gangsta Granny will be the perfect outing for families – and that a granny or two are in the audience. And, casting his mind back to his own childhood, he says that in all the craziness of Gangsta Granny, at its heart is a very special relationship.
"The moral of the story is 'don't assume old people are boring just because they are old'," he says. "In fact they are likely to have had a much more interesting life than yours. Talk to old folk, listen to their stories. They are bound to be full of magic and wonder."
Gangsta Granny tours from November 2015 to January 2017 with dates at towns and cities including Stoke, Coventry, Nottingham and Milton Keynes. For the full tour dates click here.