The scores were level at half-time on Sunday after Robbie Keane's opener had been cancelled out by Jambos skipper Marius Zaliukas.
However, a wonderful volley from 16 yards by the Chinese midfielder on the hour mark gave him his first Hoops goal and the visitors all three points to finish off the season on a high.
The final whistle signalled the end of Lennon's period as caretaker, with Hoops fans left to discover if he is to be appointed manager on a permanent basis.
Tynecastle was bathed in sunshine for the lunchtime kick-off but the opening period was rather dull.
It was the seventh minute before Gorgie midfielder Craig Thomson drove wide of the target from 25 yards, the first real effort on goal.
The game remained scrappy and littered with errors but in the 23rd minute the visitors took the lead through Keane who scored his 16th goal since arriving on loan from Tottenham in January.
Aiden McGeady sent his fellow Republic of Ireland striker through with a pass that should have been cut out by Zaliukas and from 14 yards Keane knocked it confidently past Jamie MacDonald.
The well-taken goal turned the game firmly in Celtic's favour.
The home side were offering little threat until the 35th minute when Hoops stand-in keeper Lukasz Zaluska had to stretch to make a great save from David Obua's header after Jason Thomson had provided a fine cross from the right.
However, a minute later, after Parkhead defender Darren O'Dea had conceded a foul on Calum Elliot to the right of the Celtic box, Hearts were level.
Craig Thomson's low, driven free-kick was met at the front post by Zaliukas who from 14 yards swept it past Zaluska.
The tempo increased and Obua came close with a header moments later.
Keane then had a half chance when Jason Thomson got into a mess at the other end but MacDonald was able to gather.
The Gorgie side started the second half well and, within seconds, David Templeton had flashed a shot from 16 yards wide of the far post while at the other end Keane drove just over the bar from 25 yards.
In the 49th minute, Celtic skipper Scott Brown was booked by referee Craig Thomson for a foul on Eggert Jonsson but there was a moment of real controversy a minute later after Obua's high foot on Wilson was penalised 25 yards out.
Braafheid drove the free-kick straight past the diving MacDonald but while he ran away to celebrate, referee Thomson ruled it off, indicating that he had awarded an indirect free-kick.
Referee Thomson had to have a word with Lennon who complained bitterly about the decision.
But on the hour mark Celtic did go ahead with a wonderful Zheng goal, his first for the club since signing for the club last September as a free agent.
Keane's cross from the left was only partially cleared and from 16 yards the Chinese midfielder took a touch with his right foot, turned and hooked an unstoppable left-footed volley past the flailing MacDonald.
Celtic substitute striker Morten Rasmussen replaced the ineffective Marc-Antoine Fortune but it was Hearts who were pressing for the equaliser as tempers occasionally flared.
The Gorgie men kept plugging away and Ryan Stevenson headed a Jason Thomson cross over the bar in the 78th minute, moments before Keane's dangerous cross from the left was cleared away by Jonsson, with Rasmussen waiting to pounce.
But the game ran out of steam and Celtic held on before the travelling support gave a vocal endorsement to Lennon.