Dundee, fresh from their 4-0 opening day win last week, dominated the early stages and took the lead after just five minutes when an unmarked Kane Hemmings headed in from a Greg Stewart corner.
Hearts should have equalised against the run of play just before half-time, but Juanma Delgado sent a free header inches wide.
The Spanish striker atoned for the miss 10 minutes after the restart when he comfortably converted a penalty following a James McPake hand ball.
Juanma, who has already courted some controversy during his brief spell in Scotland, completed the comeback moments later with a fine individual goal, winning the ball on the edge of the Dundee box and jinking into the area before driving the ball into the bottom right corner of the net..
The win puts Hearts alone at the top of the Scottish Premiership table, but Celtic and Aberdeen can join them on maximum points if they win their games against Partick Thistle and Kilmarnock respectively on Sunday.
It was a better day for Dundee’s local rivals Dundee United, who won their first points of the season with a 2-0 victory at Motherwell.
The visitors’ first Premiership goal of the season came with just seven minutes of normal time remaining, courtesy of an own goal from Motherwell defender Louis Laing, who sliced the ball into his own net from a United corner.
In their search for an equaliser, things got worse for Motherwell and goalkeeper Connor Ripley, who was sent up for a late corner, only for United to break and for Simon Murray to put the ball past the rapidly retreating Ripley.
It later emerged that United midfielder Rodney Sneijder has returned home to the Netherlands to undergo tests for what is suspected to be a viral infection and will not be available for the midweek Tayside derby.
There were also first goals and points of the season for Ross County, who comfortably overcame Hamilton 2-0 at the Global Energy Stadium thanks to two goals from Craig Curran.
The hosts had to recover from an early scare, however, as goalkeeper Scott Fox conceded a penalty after just five minutes when he brought down striker Christian Nade in the area.
But Fox soon redeemed himself by producing a brilliant stop to deny Ali Crawford from the spot.
Jim McIntyre’s team took the lead five minutes later as Curran earned and scored from a corner.
Curran then doubled his and Ross County’s tally just before the break, finishing from a clever Marcus Fraser cutback.
Meanwhile, a late goal from St Johnstone rescued a 1-1 draw with Inverness at McDiarmid Park.
The away side went ahead after just nine minutes, when midfielder Ryan Christie found enough space just inside the area to slam a sweet strike past Alan Mannus in the St Johnstone goal.