Torres was dismissed in the 81st minute and the Blues were left to hang on for a draw after John Terry cancelled out Gylfi Sigurdsson's opener for Spurs.
Mourinho felt the momentum had been with Chelsea before referee Mike Dean's decision to give the Spaniard a second yellow card following an innocuous aerial collision with Vertonghen.
Mourinho was unhappy with Vertonghen's reaction and also pointed to the Spurs defender's tug on Nicklas Helenius' shorts in the Capital One Cup match against Aston Villa in midweek, which went unpunished and meant he was available to face Chelsea.
"Football is football and things happen which we don't know why. At that time we were much better and stronger, they were in trouble," said Mourinho.
"When I arrived here in 2004 I had players arriving from other countries and I had to educate them because England is England and other countries are other countries. In other countries people say the guy is clever and the guy is intelligent because he pretends it's a penalty or a red card. Not in England, that's not what I learned here.
"That's why I don't think the referee is guilty, he made the wrong decision but he trusts the player. When you see the player with his hand on his face and pretending it was a violent action the natural tendency is to follow and make the decision.
"It was a situation where the player wasn't helping the referee. He's a fantastic player but he's a special guy because three days ago he left the player naked against Aston Villa. It wasn't a penalty or a red card and he shouldn't have played today, he should have been suspended from a red card against Villa. Today he changed the game, I'm not happy."
Mourinho is confident that Chelsea will be able to sustain a title challenge and refused to be too downbeat about getting a point from such a difficult away game, while he was full of praise for his side's performance in the second half.
He said: "In the second half we played very well. We played with very good spirit, a strong mentality, we were strong tactically and the second half was very good.
"When you get a draw at the home of a title contender it's never a bad result. When you get to the end of the season, March, April, May and June you will see if you get a lot of points at home and win some matches away, and don't lose against your direct opponents for the title, we will be there.
"I'm not saying we are going to win but we will be there. We won't be like two seasons ago, 20 points behind, or like last season, 15 points behind, we will be there.
"If you told me before the match I would get a point at Tottenham I would say not bad. If you said at half-time we would get a point I would say okay, but the way we played in the second half we should have won the game."
Mourinho also gave his backing to Juan Mata, who started on the substitutes' bench but helped inspire Chelsea's strong response after coming on in the second half, and he will now keep his place for the UEFA Champions League clash on Tuesday.
He said: "I think this is the way players have to say I want to play. Blah blah blah is not good, conversations with the press aren't good, what's good is the effort he made against Swindon, it's the way he changed the team in the second half and because of that I'm very happy and I can say with 72 hours in advance, he plays in Steaua Bucharest, he earned that himself."