The 20-year-old has been one of the stars of the season so far, scoring five goals for West Brom in all competitions and four more for England Under-21s.
Berahino has been installed by some bookmakers as a 20-1 shot to join Roy Hodgson's squad on the plane to Brazil next summer.
When that was put to Clarke, he laughed, and said: "Six weeks ago nobody spoke about Saido. Nobody. And now people want to put him in the World Cup squad.
"Come on, you have to keep a sense of realism. Saido's had a fantastic start to the season. He's scored some goals for the England Under-21s against opposition that is not the strongest. He's scored a Premier League goal for us and he's doing all right.
"Let's just give him time to develop and see where his career ends up."
Clarke did not rule out the possibility completely though.
"It can happen," he said. "In football you never say never but for me at the moment (I want) everybody to back off Saido and give him time to grow as a player. That's what I want more than anything. I want to give the boy time to grow and develop as a player."
Berahino's modest contract has less than a year to run and the club are currently in negotiations over a new deal but Clarke is reluctant to put much emphasis on the matter.
"I'm going to speak this time about Saido's contract but don't ask me next week or the week after or the week after that," he said.
"Saido's contract is under discussion, everybody is talking about it. Three weeks ago I said I was confident something would be sorted and I'm still confident."