It was the first time in 47 years that the home side had hosted Celtic in a Scottish Cup tie but seldom in the intervening time could this famous fixture have produced such an afternoon of drama.
Light Blues' midfielder Jamie Ness began the mayhem when he scored his first Rangers goal in the third minute with a thundering drive from outside the box.
Old Firm debutant Kris Commons levelled in the 15th minute before Steven Whittaker regained the Light Blues' advantage after Hoops keeper Fraser Forster had been sent off for bringing down Steven Naismith in the box.
Celtic refused to be beaten though and skipper Scott Brown equalised in the 64th minute before Naismith, booked in the first half for a foul on Mark Wilson, was sent off for simulation inside the visitors' penalty area.
When the final whistle sounded it was Celtic fans and manager Neil Lennon who celebrated the result and after dominating for most of the game they will fancy their chances at home in the replay.
Lennon had resisted the temptation to recall fit-again striker Georgios Samaras - who scored both goals in the 2-0 win at Ibrox in January - but there was some surprise that he split up his on-form strike duo of Gary Hooper and Anthony Stokes to leave the former playing the lone striker's role.
Midfielder Ki Sung-yueng, just back from Asian Cup duty with South Korea, was brought back in and Lennon solidified the centre of his defence by recalling Daniel Majstorovic.
Much of the attention was on Rangers' new signing El-Hadji Diouf, who had been infamously fined and suspended in 2003 for spitting at a Celtic fan at Parkhead when he played for Liverpool.
And it was his involvement which led to Ness' opener in a whirlwind start.
Diouf's corner from the right was cleared to the edge of the box and the young midfielder took a touch before arrowing a drive from 25 yards through a ruck of bodies and high past Forster.
Moments later Light Blues' striker Nikica Jelavic, another Old Firm debutant, made his way down the left to the byline before cutting the ball back to Steven Davis who rattled the bar with a thunderous drive.
Celtic's response was swift, impressive and long-lasting and in the 15th minute, soon after Wilson headed over from a Ki corner, they were level.
A break of the ball at the edge of the Rangers box allowed Joe Ledley to take possession and his cut-back found Commons, who side-footed it past Allan McGregor from 12 yards.
The Parkhead side took control and a fine move in the 23rd minute involving Beram Kayal and Brown ended with Hooper dragging his shot from 16 yards wide of the far post.
Rangers could hardly get out of their own half but in the 33rd minute Forster took a heavy touch from a Majstorovic backpass and struck his clearance against the onrushing Jelavic, the big keeper relieved to get back in time before the ball trickled over the line.
Four minutes later, however, Forster compounded his error when he was red-carded by referee for upending Naismith, who had raced on to a fine Jelavic pass.
Substitute keeper Lukasz Zaluska came on for Commons as Celtic re-jigged, and the first thing he had to do was pick Whittaker's spot-kick out of the net after he had been sent the wrong way.
Despite being down to 10 men the visitors started the second half in possession and within minutes Ki had sent a shot from 30 yards past McGregor's left-hand post.
Just after the hour mark Samaras replaced Kayal to give Hooper a helping hand and four minutes later they were again level, this time through Brown.
After the tireless Emilio Izaguirre had again bombed down the left, his cross to the back post was gathered by Wilson who laid it off to the Scotland midfielder at the edge of the box and he sent a left-footed shot curling past McGregor and into the far corner.
After the celebrations had subsided, Brown was booked for what appeared to be have been a provocative stance in front of Diouf after scoring.
Moments later Jelavic broke free on the left and his low cross took out Zaluska but had too much pace on it for the sliding Davis.
Still Celtic kept pressing and in the 73rd minute McGregor made a point-blank save from Samaras at the expense of a corner, which came to nothing.
More drama was to come a minute later when Naismith, already on a yellow, was sent off for simulation after going to ground in the Celtic box after Charlie Mulgrew had challenged.
Vladimir Weiss and Kyle Lafferty immediately replaced Jelavic and Diouf but the Hoops remained in control, pressing for the winner as Rangers seemed unable to get a foothold in the game.
In the end it was the home side who were happy to hear the final whistle although it will be Celtic who will relish the replay at Parkhead.