Second-placed Rangers lost defender Steven Smith in the 21st minute when he was sent off for a crunching tackle on Hearts defender Callum Paterson in the early kick-off game on Saturday.
Hearts midfielder Jason Holt then found himself in space in the Rangers box 11 minutes into the second half and struck the loose ball cleanly to score the opening goal past visiting keeper Steve Simonsen.
Rangers immediately responded, forcing Hearts keeper Neil Alexander to block a volley from Nicky Law before Lewis McLeod hit the post.
But, with the visitors’ ten men tiring and Hearts dominating the latter stages of the fierce contest, in the 88th minute Jamie Walker drew a foul in the Rangers box from Ian Black to earn a penalty.
Walker fired home the penalty to seal the win that gives unbeaten Hearts a commanding lead at the top of the table over their nearest challengers, who they also beat at Ibrox in the opening game of the season.
Hearts have now won 12 of their 14 league games, adding two draws as they battle for the one automatic promotion spot back to the Scottish Premiership.
The defeat heaps pressure on Rangers manager Ally McCoist who has suffered three defeats in their 14 matches, with the visiting supporters making their frustrations heard at the end.
Rangers made three changes from their disappointing draw with Alloa with Kris Boyd dropped to the bench to allow Jon Daly to partner Kenny Miller up front while Lee Wallace and Bilel Mohsni returned in defence.
The opening exchanges were peppered with nasty challenges as both sides tried to exert their influence on the match.
Rangers were the first to make some headway with Lee Wallace earning a corner after some good work down the left but Darren McGregor fired wide from the resultant corner.
However, it was the 21st minute red card for Smith from his reckless challenge on Paterson that eventually told.
Rangers initially continued to force the play and Paterson soon picked up a yellow card for dragging down Law just outside the box. The resultant free-kick was played low to Wallace on the penalty spot but he blazed his effort high over.
The visitors then earned another free-kick 20 yards out which Law curled around the defensive wall only to see former Rangers keeper Alexander comfortably push it around the post.
But a minute later Alim Ozturk’s effort from a similar range forced Rangers keeper Steve Simonsen into a similar save at the other end.
And Hearts slowly began to make their extra man count as they dominated possession. Morgaro Gomis was inches wide with a bending shot from 25 yards although Simonsen appeared to have it covered.
They almost had the opener just before the break but Simonsen made a sensational save to tip Brad McKay’s wonderfully guided header on to the bar.
Then in the 56th minute Paterson’s cross was acrobatically cut out by Lee McCulloch. The ball took an unfortunate deflection off teammate McGregor into the path of Holt who hit his unstoppable strike beyond Simonsen, with Hearts sealing the match 32 minutes later with their penalty .