There were near misses at both ends before Skacel broke the deadlock in the 58th minute with an unstoppable drive from the edge of the box.
Celtic were further disarmed when midfielder Kris Commons, who had just returned to the Hoops starting line-up, was shown a straight card by referee Craig Thomson in the 65th minute for a reckless challenge on Adrian Mrowiec.
And as the 10-man visitors piled forward in search of an equaliser, Stevenson scored a breakaway second in the 80th minute to leave Celtic in third place behind Motherwell and 10 points behind leaders Rangers, albeit with one game in hand.
The Parkhead club will now have the international break to regroup but that is of little concern to the home supporters who cheered their side off the pitch at the end of an exciting afternoon.
There was pressure on Celtic manager Neil Lennon before the game after Rangers had increased their lead with victory against Hibernian on Saturday, but all eyes were on the Hoops boss for other reasons, the Irishman having being attacked at the touchline the last time the Parkhead side visited Gorgie in May.
As a consequence of that eventful night, security was beefed up.
One fan was ejected from the Hearts section of the ground before the game began which drew huge cheers from the travelling support and which cranked up the atmosphere.
Jambos owner, Vladimir Romanov, the Russian-born Lithuanian businessman, was in attendance, two months after his last visit to Tynecastle for a game signalled the end for previous manager Jim Jefferies.
So the ingredients were all there for yet another classic tussle between these two clubs and the game, predictably, began amid a din and at breakneck speed.
In the fourth minute Celtic keeper Fraser Forster made a good save from Jambos' lone-striker Stevenson who had escaped his marker to get in a header.
Moments later, Hearts skipper Marius Zaliukas struck the post with a left-footed shot from the edge of the box.
Celtic roared back and in the 20th minute they should have taken the lead when James Forrest's cross from the right found Mohamed Bangura but his close-range header was brilliantly saved by Jamie MacDonald.
Seconds later, following the corner, the Hearts keeper made another good save at his near post from Commons' drive.
The game roared from end to end with tackles flying in all over the pitch.
In the 26th minute Skacel was booked for a late challenge on Hoops stand-in skipper Charlie Mulgrew, before Parkhead playmaker Ki Sung-yueng was shown yellow for fouling Skacel.
On the half-hour mark, after Hoops midfielder Victor Wanyama was penalised for his challenge on Ian Black at the edge of the box, with Forster diving to prevent Jamie Hamill's free-kick from going in.
Celtic responded again and MacDonald made a fine block from Commons' powerful drive from just inside the box, beating the ball away to safety and the pace never let up until the break.
The interval allowed everyone to get their breath back but when play commenced it was the visitors who took the initiative and Commons had another long-distance effort which was dealt with comfortably by MacDonald.
Celtic pressed with purpose but in the 58th minute Hearts took the lead with a fine strike by Skacel.
Hamill's high ball from the right fell out of the air and, when it broke from David Templeton, the Czech midfielder arrowed in a unstoppable left-footed drive from the edge of the box to give Forster no chance.
With the home fans still celebrating, Lennon immediately replaced Mohamed Bangura with Anthony Stokes.
In the 65th minutes there was more trouble for the Parkhead men when Commons was sent off for a reckless challenge on Mrowiec in the middle of the park.
It was mayhem inside Tynecastle as Celtic reshuffled and threw everything in search of an equaliser, while leaving themselves open to the counter-attack.
In the 72nd minute Mark Wilson had to replace the injured Adam Matthews and he drew a save from MacDonald moments later with an angled shot.
Hearts had to defend desperately at times but in the 80th minute Stevenson made it 2-0 when, with the Hoops defence stretched and struggling with a long Danny Grainger punt, he ran on to clever back-heel from Templeton and drove high past Forster from 10 yards.
It was the final blow to the visitors who know they have some work to do if they are to catch Rangers, even this early in the season.