Naismith's opening goal was cancelled out by Gary Hooper, before Badr El Kaddouri marked his debut in the fixture with a goal which came from a blunder by Gers goalkeeper Allan McGregor.
Nikica Jelavic hauled Rangers level again and Kyle Lafferty also netted before Charlie Mulgrew saw red later on and Naismith had the final say with an injury-time strike.
The victory allowed the Scottish champions to move four points clear of their rivals at the Scottish Premier League summit.
Both sides made two changes from their previous outings; a win over Dundee United in the SPL for Rangers and a Europa League defeat against Atletico Madrid for Celtic.
Jelavic and Sasa Papac recovered from injury and illness respectively to start for Rangers, with Kirk Broadfoot and Juan Manuel Ortiz dropping to the bench.
Celtic drafted fit-again captain Scott Brown and El Kaddouri into the side in place of James Forrest and Joe Ledley.
The match began at a frantic pace but without many clear-cut chances from either side in the opening spell.
Georgios Samaras collected a short pass from Hooper but blasted well over from long distance, with McGregor untroubled in the Rangers goal.
At the other end, Maurice Edu took a tumble in the box but there was no suggestion of a penalty claim from the United States international.
Mulgrew was the first name in referee Craig Thomson's book when he was cautioned after 16 minutes for a late challenge on Naismith, which would prove costly later on.
Rangers opened the scoring in style with 22 minutes on the clock when Kelvin Wilson attempted to hook a Lafferty cross to safety.
But his clearance fell to the feet of Naismith, who left Fraser Forster with no chance with an unstoppable right-footed shot from just inside the box.
Lafferty then had the ball in the back of the net but only after being ruled offside when running onto a pass from Steven Davis.
Celtic were back on level terms with 34 minutes gone, with Brown setting up Hooper with a pass from the left and the striker producing a lovely finish from 10 yards.
The first two goals were all about fantastic finishes but the third came courtesy of a huge gaffe from the Rangers goalkeeper.
McGregor looked set to block a long-range attempt from El Kaddouri, only to allow the effort to slip between his legs and over the line.
Rangers emerged for the second half determined to haul themselves back into the game and Davis was denied by the woodwork when his shot from 25 yards crashed off the crossbar within seconds of the restart.
Lafferty was then denied by the offside flag for the second time in the match, with Jelavic the provider on this occasion, before Mark Wilson earned a booking for a foul on the Croatian.
Jelavic quickly recovered from the challenge to add his name to the scoresheet with 55 minutes gone when he met a Davis cross and bulleted a downward header past Forster from 10 yards.
Rangers pushed forward in search of more goals and Lafferty conspired to pass up two great opportunities, firstly sending a header over from six yards before failing to capitalise on an El Kaddouri error by firing wide of the gaping net.
Celtic also looked capable of finding the back of the net again and Glenn Loovens looked on in disbelief as his header smacked off the post.
However, Rangers reclaimed their lead after 67 minutes when Lafferty finally grabbed a goal, stabbing home from close range after seeing his initial effort blocked.
Celtic were dealt another blow when Mulgrew was dismissed with 15 minutes to go when he received a second yellow card, this time for a challenge on Davis.
Rangers made the most of their numerical advantage and it was Naismith who put the gloss on the win with a left-footed shot in injury time, as Ally McCoist claimed an impressive victory in his first clash as manager against Celtic.