Rudi Skacel marked his first start for Hearts with a first-half opener, before Kyle Lafferty came off the bench to haul Rangers back into the match.
Just when it looked as though both teams would have to settle for a draw, Naismith - who also grabbed the winner against Bursaspor in the Champions League - secured all three points deep into injury time.
Nikica Jelavic - ineligible for Europe - and Vladimir Weiss were both back in the starting line-up for the lunchtime clash. Lee McCulloch and Kirk Broadfoot both dropped out of the side, with hip and foot injuries respectively.
Skacel made his first start for Hearts, as boss Jim Jefferies made five changes from last week's defeat against Motherwell.
Only some last-gasp defending from Hearts prevented Rangers from taking the lead early on.
An attempt from Jelavic was blocked by goalkeeper Marian Kello and fell to the feet of Steven Davis but he was denied the opener when Ismael Bouzid cleared his shot off the line.
It was Hearts who opened the scoring with 12 minutes gone. Kevin Kyle flicked into the path of Skacel and a mix-up between Madjid Bougherra and goalkeeper Allan McGregor allowed him through on goal and to bundle into the back of the net.
Miller then went to ground in the box but pleas for a penalty were ignored by referee Craig Thomson.
Rangers pushed forward in search of the equaliser in a bid to haul themselves back into the game before the interval.
They were not too far away when Sasa Papac's cross was touched on by Naismith for Jelavic but an acrobatic overhead kick from the Croatian - similar to the one he scored against Dunfermline - was pushed over the crossbar by Kello.
Hearts could have doubled their advantage after the restart when Calum Elliot shrugged off David Weir and tried his luck with a shot at goal that fell inches wide of the post.
At the other end, Weiss was only able to produce a weak effort that was comfortably smothered by the goalkeeper as the home side defiantly clung on to their lead.
The champions threatened again when the clearance from a Weiss corner fell to the feet of Papac but his poweful volley was blocked by Kello, before Davis blasted well over after collecting a flick on from Jelavic.
With time running out for Rangers, Steven Whittaker had a go with a shot from distance that clipped team-mate Naismith but was still gathered by Kello at the near post.
Lafferty was then thrown into the action for Jelavic with 15 minutes to go, while Hearts replaced goalscorer Skacel with Ryan Stevenson.
Four minutes later, Lafferty equalised. Darren Barr was booked for hauling down Naismith 25 yards out and the Northern Irishman stepped up to dispatch a low free-kick into the back of the net.
Hearts should have restored their lead just moments later when Kyle met a cross from Craig Thomson only to squander a great chance by nodding over from six yards.
But it was Naismith who had the final say three and a half minutes into injury time when he embarked on a solo run before sending a left-foot shot past Kello to snatch the points for Rangers.