Spurs were leading through an Emmanuel Adebayor goal when the striker was sent off for a lunge at Santi Cazorla, leaving his team-mates to play more than 70 minutes a man short.
Arsenal raced into a 4-1 lead and, although Gareth Bale's strike briefly threatened to spark a fightback, Theo Walcott scored in injury-time to make it a repeat of February's derby scoreline.
Villas-Boas said: "I think the sending-off determined the game, but the referee made the right decision. Players want to dispute the ball, there was no intention to do any harm.
"I'm very, very proud of the team. The feeling of the result is indifferent for us when we played so well and we took the risk in the second half to be so bold and play with three at the back. Men against men.
"Yes, we suffered but there were chances that, had we converted and brought the score to only a one-goal difference, I think it could've gone our way."
The Spurs manager rejected suggestions Adebayor was an accident waiting to happen against his old club.
"I don't think so," he said. "We make the most out of players' revolts, players' motivations. It's something that we respect.
"I think Ade was in full control of his emotions and in a north London derby you want players to challenge for balls, you want them to win second balls, you want them to be strong, to be brave."
"Cazorla is quick, he reacted, he took the ball away before Ade's feet met his foot and the ref has to make a decision and he decided for the red card.
"We understand that it could've been any other player on any other day. This is football and challenges like this happen."