Belgium striker Benteke was dismissed after 65 minutes at Villa Park following a skirmish with young Spurs midfielder Ryan Mason ahead of Nacer Chadli firing in his fifth league goal of the season in the 84th minute which looked like snatching a point for the visitors.
But Kane’s deadball effort was then deflected by Nathan Baker’s head past his own goalkeeper Brad Guzan to lift the north Londoners up to eighth in the table and stretch Villa’s run of league defeats to six.
Andreas Weimann had earlier ended 547 barren minutes in front of the goal for the hosts with a wonderful opportunist effort.
The Austrian forward slid home Charles N’Zogbia’s cross to end Villa’s scoreless nine hours in front of goal in some style.
It was Villa’s worst run in front of goal in their 140-year history but it was Spurs striker Robert Soldado who missed the first decent chance of the game for the visitors with his head.
Benteke then almost broke the 50-day drought for Villa in the 13th minute.
The robust striker pulled away to meet a cross from the left and cannoned a fierce left-foot drive against the outside of the post from 15 yards out.
Villa Park would only have to wait three minutes before the deadlock was broken as N’Zogbia, in for the dropped forward Gabriel Agbonlahor, broke down the right and crossed for Weimann to brilliantly slide the ball past Hugo Lloris in the Spurs goal.
Tottenham had a clear chance to equalise when Christian Eriksen played in Emmanuel Adebayor to go one-on-one with Guzan but the Togo forward’s tentative shot was well turned away by the United States goalkeeper.
Weimann, fresh from his third goal of the season, hooked a Villa cross over Lloris’s crossbar ahead of the industrious Benteke flicking the outside of the post with a looping header ahead of half-time.
With both clubs desperate for something from the game it turned somewhat bad-tempered after the interval.
And Benteke received a straight red card with 25 minutes to go for raising his hand to Mason after the English midfielder had pushed his head into the Belgian.
Spurs sensed blood but it would take until the 84th minute for the unmarked Chadli to volley home substitute Erik Lamela’s corner to equalise at the far post.
The stage was set for a grandstand finish and, after Carlos Sanchez hauled down Andros Townsend, Kane snatched the ball off Lamela to hit a Spurs free-kick in off Baker for a priceless winner and a first away league victory since the opening day of the season.