Defoe took his season tally to seven goals on Thursday night when he scored the opening goal in Tottenham's 2-0 win over Anzhi Makhachkala.
Defoe has played just 50 minutes of Premier League football so far this term, however, as £26million signing Roberto Soldado has been Villas-Boas' first choice.
The Portuguese is a fan of pitting players against their old clubs as he thinks it brings out the best in them, meaning Defoe could possibly be handed his first league start of the season on Sunday when Spurs welcome West Ham to White Hart Lane for what promises to be a fiery contest.
Cole and Defoe spent four years together at Upton Park, where the striker scored 41 goals in five years, and the 31-year-old midfielder is afraid his former team-mate will come back to haunt his old team this weekend.
"Tottenham have brought in some good players but personally, I'll be glad if my old West Ham and England team-mate Jermain Defoe's name is not on their team sheet on Sunday," said Cole, who rejoined West Ham last season after spells at Chelsea and Liverpool.
"He seems to have scored every time he has played this season and we all know what he can do.
"If you give him a chance he can hurt you, so I think he's probably impressed me the most of Tottenham's players.
"We've got to stop him from scoring this weekend, but Jermain will be eager to impress.
"He wants to get back into the England squad and he'll be right up for it against his old club.
"We'll need to be right on it, so I'll be happy if he is not playing against us."
Defoe is not the only familiar face that Cole will come up against on Sunday.
Villas-Boas is another man the former England midfielder knows well from the Portuguese's time on Jose Mourinho's backroom staff at Stamford Bridge.
Villas-Boas went on to manage Chelsea in 2011, but lasted just nine months before being sacked.
The Portuguese has rebuilt his reputation at Spurs, where he is mounting another top-four challenge after just missing out on Champions League qualification last term.
Spurs have won four of their opening six Premier League games this season and Cole thinks the north London club have a top manager at the helm.
"Andre Villas-Boas is a really nice guy, a top manager and I think he can go on and do amazing things in his career. Tottenham are lucky to have a top-drawer manager in Andre," Cole told West Ham TV.
"Jose is proven as the number one manager in the world, but Andre is up and coming and doing his thing."
While all the talk around Spurs has been of a possible title challenge, West Ham have been struggling at the other end of the table.
The Hammers finished 10th last term, but they have struggled so far this season, collecting just one win from their opening six games.
Cole has been a frustrated bystander for most of the season because of a hamstring injury, but he hopes to return on Sunday.
Cole, who scored against Spurs at Upton Park in February, is braced for a difficult comeback game.
"Tottenham did some good business in the summer and it's going to be a really tough game," Cole said.
"It's been a tough few weeks for us. We have not got the number of points on the board that we wanted and we're going into a run of very tough games."