Danny Wilson put Hearts ahead before Nicky Law levelled on 90 minutes, but just moments later Osman Sow snatched the match for the visitors in a thrilling finale.
For newly promoted Rangers it was their first defeat in league football since the 2-1 loss to Peterhead in Division Three in April 2013.
Wilson netted a rare goal when he climbed above his markers to meet a Billy King corner and power his header high into the net with the aid of a slight deflection.
Law appeared to have earned a point for last season’s League One winners when he latched on to a Kenny Miller header to hook the ball past substitute goalkeeper Scott Gallacher.
But just 60 seconds later Sam Nicholson drifted through the midfield and slipped the ball to Sow who strode past two defenders and dispatched a low drive into the far corner of the net.
Neil Alexander earlier almost gifted Rangers a goal on six minutes when he tried to dribble the ball past and ran into Kris Boyd, but the keeper managed to block the latter’s strike and smothered at the second attempt.
David Templeton then headed over the target when it looked easier to score as the ball spun off Alexander’s leg from a Miller drive inside the box.
Hearts were again caught out through sloppy defensive play as Lee Wallace stole possession when Alim Ozturk tried to shepherd the ball out for a goal-kick but Templeton’s attempted cross for Boyd drifted over the bar.
Nine minutes before the break two former Hearts players combined as Marius Zaliukas met a free-kick from Ian Black but the defender glanced his header wide.
The visitors finally created a chance as Wilson sent a low delivery across the face of goal and Ozturk clipped the ball past the far post.
And Hearts had another opportunity when a cross from Sow found Nicholson ghosting between two defenders but his downward header was straight at keeper Cammy Bell.
Five minutes after the opening goal Hearts survived a scare when Lee McCulloch’s mis-hit shot broke to Miller inside the six-yard box but Alexander blocked his effort before the ball was scrambled clear.
Alexander then had to be replaced by Gallacher after he clashed heads with Ozturk when catching a free-kick.