Substitute Leigh Griffiths scored a hat-trick after Darryl Westlake had given Killie the lead, with Kris Commons netting the equaliser which he and Scott Brown both appeared to claim.
Celtic penned the visitors back from the start of the match and Emilio Izaguirre got in behind the Kilmarnock defence to cut the ball back for James Forrest, but the winger’s shot was deflected wide for a corner.
Izaguirre was involved again when he again got to the goal-line and crossed for Forrest, who laid that ball off to Brown, but the Celtic skipper fired wide.
Virgil van Dijk was inches away from opening the scoring with a header on 33 minutes when he timed his run to perfection to meet a driven corner from Commons.
Celtic continued to press forward at the start of the second half with Forrest cutting inside Mark O’Hara and passing to John Guidetti, who held off two players to tee up Brown but his shot was straight at goalkeeper Craig Samson.
But the home side were stunned on 50 minutes when Westlake let fly from outside the box and his shot deflected past Craig Gordon giving the keeper no chance.
Celtic levelled the game eight minutes later when they worked the ball across the front of the box and Brown nipped in front of Sammy Clingan at the edge of the penalty area and both he and Commons swiped to steer a shot into the top corner, with the latter looking like making the connection.
There was no doubt about who netted the second though as Commons clipped the ball in from the goal-line for Griffiths to power his header into the net from close range.
Commons then worked a neat one-two with Stefan Johansen only to have Samson turn his drive behind, and from the resulting corner Jason Denayer fluffed a great opportunity.
Johansen missed a great chance shortly after that as he shinned the ball wide with the goal at his mercy after Samson had blocked a shot from Guidetti.
But Griffiths (80) secured the points with the next attack as Brown pulled the ball back to the striker and he blasted a curling shot off the inside of the post and into the opposite corner of the net.
Two minutes later Gordon produced a wonderful save to keep out a net-bound chip from Nathan Eccleston seconds after the Killie substitute came on the pitch.
Griffiths completed the scoring and his treble five minutes from time when he latched on to a long ball and rifled a low drive past Samson.