Reidar Stenersen Jr, a 29-year-old hairdresser from the Norwegian city, subjected Villas-Boas to derogatory abuse from the stands when the score was goalless.
The Spurs boss took offence to the chants and notified a steward, who confronted Stenersen. The official asked Stenersen, who was sat on the front row just behind the Spurs dugout, to move stands but he rejected the idea and walked out to watch the remainder of the game in a pub in the city centre.
Stenersen, who also supports Manchester United, claims Villas-Boas objected to the chant of "You're getting sacked in the morning".
Stenersen reckons Villas-Boas, who is under pressure following last week's 6-0 loss at Manchester City, overreacted.
"I first sung after five minutes that he would be 'sacked in the morning' and he looked at me," Stenersen told the Norwegian newspaper Nordlys.
"At the half-time whistle, when it was still 0-0 and I started the same song, he pointed at me and suddenly the security came and threw me out.
"I know he is under a lot of pressure so I think my words hit him, even though I am only a little guy in little Tromso. He was being a bit petulant. This is the same thing that can be sung by 60,000 at the Emirates Stadium or other grounds."
Tottenham won the Europa League group game 2-0 thanks to second-half goals from Vlad Chiriches and Mousa Dembele.