The Croatian ensured the points would be heading back to Ibrox with a late winner after his opener was cancelled out by an own goal from Carlos Bocanegra.
The Light Blues had lost back-to-back SPL away games prior to this encounter but Ally McCoist's men ensured they kept the pressure on Old Firm rivals Celtic.
Rangers made two changes from the side which defeated Arbroath in last week's William Hill Scottish Cup tie.
Goalkeeper Allan McGregor was out with a thigh injury and John Fleck was dropped to the bench, with Neil Alexander making his first SPL appearance of the season in goal and Steven Davis back from suspension.
Saints also made two changes, with Callum Davidson and David Robertson given the nod ahead of Alan Maybury and Carl Finnigan.
The first real chance fell to the Perth side when Francisco Sandaza - who is reportedly set to reject a pre-contract offer from Rangers - beat several players before slotting into the path of Marcus Haber.
Haber had the ball in the back of the net after 13 minutes when he rifled past Alexander but Rangers were able to breathe a sigh of relief when the effort was flagged offside.
Dorin Goian became the first player in the game to be booked a minute later when referee Craig Thomson penalised him for a foul on Murray Davidson, before David Healy dragged a low shot wide for the visitors.
Rangers surged ahead after 23 minutes when Sone Aluko collected a quick free-kick from Davis and weaved his way into the box before unleashing a shot which was blocked, before Jelavic pounced to stab home from close range.
The home side were unhappy with the positioning of the set-piece and St Johnstone skipper Jody Morris was booked for dissent. Thomson then warned both managers about their behaviour towards each other in the dug-out.
Jelavic required treatment following a challenge by Steven Anderson but Healy failed to make the resultant free-kick count and goalkeeper Peter Enckelman was able to smother comfortably, before producing a decent save to deny Jelavic his second of the day.
Rangers emerged for the second half determined to strengthen their lead and Davis saw his shot blocked by Morris after being teed up for the effort by a lovely ball from Aluko.
Davis whipped the corner into the packed goalmouth where Goian rose above everyone else only to be denied a goal when his header was nodded off the line by Robertson.
Saints were back on level terms after 67 minutes when a Liam Craig cross was nodded down towards goal by Haber and deflected into the back of his own net by American defender Bocanegra.
The home side were threatening again just moments later when Sandaza squared for Robertson on the edge of the box and his swerving effort fell just wide.
Rangers tried desperately to reclaim their lead and a well-struck shot from Healy was blocked by Enckelman and Lee Wallace failed to connect at the far post to nod home the loose ball.
The Ibrox side were celebrating with 81 minutes gone when Aluko's free-kick was nodded onto his own crossbar by Murray Davidson and dropped for Jelavic to force over the line from a couple of yards out.
The away fans then clamoured for a penalty when Jelavic went to ground under pressure from Dave Mackay but no award was forthcoming as the Scottish champions held on for the win.
