Spurs saw an initial bounce in form when Mourinho joined the club in November, but have stuttered recently after being hampered by injuries to some of their key stars.
Cole has sympathy for his former boss, claiming he is “trying to turn a cruise ship” at Spurs and deserves more time before being judged.
“Jose has gone into a difficult situation,” Cole told PA Sport.
“In the media in particular there is a lot of anti-Jose, people who are very quick to jump on Jose’s back, especially for someone who has been as successful as him.
“You need to look at the job in perspective, I think it had run its course under Pochettino. I think the team wasn’t playing well, it needed a change.
“But it’s like turning a cruise ship, you can’t just turn it on a sixpence, it takes time. He needs time, he needs to bring the right players in.”
Spurs are without a win in six matches across all competitions and many believe Mourinho isn’t the right man to take them forward.
And Cole believes club chairman Daniel Levy will play a big part if the Portuguese is to succeed in north London.
“I think the relationship with Daniel Levy is important,” Cole added.
“If Levy gets him the players he wants and gives him time I think it will be a success for him.
“It’s just how every time Spurs lose a game… Spurs were losing games under Pochettino, players weren’t playing well.
“I think some of the players may have run their course at Tottenham. They are all great players but sometimes your time is up. They are in a rebuild situation.”