Jonny Hayes' 30-yard thunderbolt just after the half-hour had given the Highlanders the lead before Miller equalised with his 20th goal of the season in the 57th minute.
But the league's top scorer blew the chance to send the visitors five points clear at the SPL summit when his 82nd-minute spot-kick was saved by Ryan Esson as Terry Butcher - a legend during his time as captain at Rangers - remained unbeaten as Inverness boss against the Glasgow giants.
The draw moved Rangers three points clear at the top and put Caley Thistle level with third-placed Hearts.
Impressive Inverness youngster Nick Ross was dropped to the bench in favour of attacker Eric Odhiambo, who netted the late leveller in the 1-1 draw at Ibrox in October.
Rangers manager Walter Smith made two changes to the side which drew 1-1 in Turkey against Bursaspor in the Champions League on Tuesday, with Sasa Papac and Vladimir Weiss replacing youngsters Kyle Hutton and Darren Cole.
Caley Thistle started brightly at Tulloch Caledonian Stadium with Rangers stopper Allan McGregor thwarting Adam Rooney.
The Irishman was played in by Odhiambo in the ninth minute but the keeper raced out to block the best chance of the opening stages.
Rangers threatened with a Steven Naismith header going inches over and Kenny Miller squandering a chance to break the deadlock.
Miller chipped over from the edge of the box after being played through by strike partner Naismith and tried to delicately lift the ball over the onrushing Esson, but it went just too high.
Moments later Hayes rocked the visitors with his stunning opener.
The winger strode forward unchallenged before rifling in a swerving 30-yard blast which beat a full-stretch McGregor, hitting the post on its way in.
Things got a bit feisty towards the end of the first half with referee Calum Murray showing yellow cards to Inverness' Richie Foran and Russell Duncan, as well as Rangers' David Weir and Richard Foster.
Rangers showed an attacking threat after the half-time introduction of former England frontman James Beattie for midfielder Weiss.
The ex-Stoke striker, who made a substitute appearance midweek after recovering from a thigh injury, provided the impetus for their first shot on target when Steven Davis' effort from the edge of the box was stopped in the 47th minute by Esson.
Two minutes later Miller rounded Esson but fired over from a tight angle with defenders covering the Caley Thistle goalmouth.
Miller did not have to wait much longer, however, with his leveller coming in the 57th minute.
The Scot beat a diving Esson with a low drive from 20 yards after being played in by Davis.
In response, Inverness forced a double save from McGregor which started when Foran found Rooney but his shot was palmed out to Hayes, whose header was tipped over the crossbar.
Beattie might have put Rangers ahead had he connected with a low cross from Steven Whittaker with the goal gaping.
Naismith missed a glaring chance to put the visitors ahead with just under 20 minutes to go when he was one on one with Esson, lobbing the ball over the bar from 10 yards.
The home fans were breathing another collective sigh of relief eight minutes from time when Miller's penalty was saved by man-of-the-match Esson.
Murray pointed to the spot after Stuart Duff felled Beattie, who had been played in by Davis. Miller's effort went straight down the middle and was stopped by the feet of the diving Esson.
It was end to end in the closing stages but neither side could find a winner.