After the home side passed up on a series of chances to take the lead, Killie striker Conor Sammon scored a breakaway goal in the 54th minute to give the visitors a shock lead.
Celtic fought back and in the 84th minute Rogne headed in from a Ki Sung-yueng free-kick but it was the visitors who could have taken all three points with Sammon thwarted in injury time.
Victory would have put Neil Lennon's side top of the SPL on goal difference but the draw leave them two points behind defending champions Rangers, who now also have a game in hand on the Hoops while Kilmarnock remain in fifth.
Niall McGinn, James Forrest and Emilio Izaguirre returned to a reshuffled Celtic side, injured stand-in skipper Shaun Maloney, Jos Hooiveld and Charlie Mulgrew dropped out, while captain Scott Brown was back on the bench for the first time since recovering from a foot injury which has kept him out since October.
Kilmarnock were looking for their first win at Celtic Park in 55 years, but Mixu Paatelainen's unchanged side travelled to Glasgow as one of the form teams in Scotland having won five of the last seven SPL games.
The Parkhead side bombed forward from the start and in the fifth minute after some hesitation in the Killie defence McGinn had a chance from 12 yards, but drove straight at the visitors goalkeeper Cameron Bell.
In the 12th minute Bell fumbled a low McGinn cross to cause some panic in the Kilmarnock defence, and as they failed to clear the danger properly, the former Derry City winger was presented with another opportunity but slashed wide from 10 yards on the angle.
McGinn had another chance against Bell in the 20th minute when he was through on goal, but the keeper blocked for another corner which Killie survived.
In the 35th minute Celtic striker Gary Hooper was carried off on a stretcher after collapsing to the ground following what looked like an innocuous tussle with Killie defender Frazer Wright.
Hooper was replaced by Anthony Stokes but the home side appeared to lose a little momentum as the game edged towards the interval.
The pattern of the second half remained unchanged as Celtic used both wings to press for the opening goal while Killie's unforced errors helped keep them on the back foot.
However, in the 54th minute, in what was the first real attack of the game, Kilmarnock went ahead through their in-form striker Sammon.
After Celtic had spurned yet another opportunity the visitors broke quickly and midfielder Mehdi Taouil sent Sammon racing over the halfway line.
There was still a lot to do but the Irishman brushed off the attention of the back-tracking Izaguirre to burst into the box and drive an angled shot past Celtic goalkeeper Fraser Forster for his 12th SPL goal of the season.
As Celtic struggled to get back level, Brown replaced Joe Ledley but it was Kilmarnock who were breaking with confidence and looking to get a second goal.
As the nerves kicked in for the Celtic defence, Sammon had another chance from eight yards but was off-balance and could only toe-poke the ball at Forster.
As Celtic threw everything forward, striker Daryl Murphy blazed a shot over the crossbar in the 76th minute after the ball broke kindly to him eight yards out following Cha Du-ri's cross from the right.
Bell made a decent save from Murphy's angled drive but with five minutes remaining the Parkhead men drew level when defender Rogne headed in a deep free-kick from Ki.
The goal set up a storming finale but in the first minute of injury time, as the Ayrshire side broke, Sammon rounded Forster but his attempt to squeeze the ball in from a tight angle was thwarted by Daniel Majstorovic.