Killie took the lead in the 19th minute when Portuguese striker Rui Miguel took a pass from Alexei Eremenko at the edge of the box before driving low past Jambos keeper Marian Kello.
The Gorgie side had been disappointing in the first half but came back strongly after the break and levelled in the 72nd minute when Elliott, on for Arvydas Novikovas at the interval, volleyed in from 12 yards.
There was more drama to come with four minutes remaining when he drove past Killie keeper Cameron Bell from inside the box although it looked as if a stronger hand might have kept the ball out.
But there is little doubt that the Gorgie men merited their win in the end, despite their unconvincing start.
Miguel had only started the game because Kilmarnock's top scorer Conor Sammon started on the bench after travelling up from Scunthorpe following his decision to reject a transfer to the English club.
Hearts were more disjointed. Boss Jim Jefferies was without injured duo Kevin Kyle and Calum Elliot, and dropped Ryan Stevenson to the bench.
In came Lee Wallace, David Obua and David Templeton with Marian Kello replacing Jamie MacDonald in the Jambos' goal.
Killie, one place behind the third-placed visitors in the table, had been the last team to beat the Gorgie side in the SPL at the end of October with a 3-0 win at Tynecastle.
However, Jefferies' men had gone on an unbeaten run in the league which had seen them garner 25 from a possible 27 points before they lost at home to St Johnstone in the Scottish Cup last week.
Neither side, though, looked like European contenders in the opening stages.
In the sixth minute Jambos midfielder Adrian Mrowiec almost sliced a free-kick from Eremenko past Kello from 12 yards, the Tynecastle side happy to concede a corner which came to nothing.
The game stumbled along and after Hearts defender Eggert Jonsson fouled Miguel at the edge of the Gorgie box, Eremenko fired the free-kick over the bar but he was soon to make amends.
In the 19th minute the Finnish midfielder robbed Ian Black who had hesitated in possession and fed the ball to Miguel who was quick to dispatch a low drive from the edge of the box past the diving Kello.
Kilmarnock took immediate confidence from the goal with the silky Eremenko catching the eye and Manuel Pascali protecting the back four with impressive determination.
The visitors began to press but showed little in an attacking sense until the 33rd minute when defender Ismael Bouzid headed Black's inswinging corner just past the near post.
Stephen Elliott replaced Arvydas Novikovas for the start of the second half and he went up alongside Obua, who had been isolated in the first 45 minutes.
The visitors looked as if they had received a typically robust half-time pep talk from manager Jefferies as they set about the task of grabbing the equaliser, with Wallace firing in a couple of crosses from the left which were just too deep.
Killie found themselves pinned inside their own half and in the 58th minute Wallace made space 25 yards from goal but his left-footed shot zipped past the far post, moments before Eggert Jonsson drove high over from a similar distance.
Just after the hour mark Jefferies introduced Stevenson and Gary Glen for Obua and Black in a bid to get back in to the game. For Killie, Billy Berntsson came on for Mehdi Taouil.
However, in the 72nd minute Hearts deservedly drew level when Elliott contorted himself to hook in from 12 yards from Bouzid's cross.
The game opened up and Sammon set up Frazer Wright in a breakaway with Kello saving the angled drive at the expense of a corner which was well defended.
Then, as the home side broke again, Eremenko's cross intended for Sammon was put on to the bar by back-tracking Hearts skipper Marius Zaliukas.
But in the 86th minute, Elliott took advantage when Killie failed to clear a Jambos attack and drove past Bell from the edge of the box, the Killie keeper getting a hand to the ball but failing to stop it reaching the back of the net.
There was time for Kello to make a great point-blank save from Pascali before Bell prevented Elliott getting his hat-trick but in the end, the points were Tynecastle-bound.