The Wales midfielder converted a James Forrest cross after 16 minutes from close range to extend the Hoops' domestic winning run to 17 games.
However, most of the drama came after the break.
Celtic defender Daniel Majstorovic, back in the side for the injured Thomas Rogne, was controversially shown a straight red card by referee Steven McLean on the hour mark for a 'last man' challenge on Jonny Hayes, a decision which had the home crowd in uproar.
Their anger dissipated a little when Caley defender Steve Williams was sent packing in the 77th minute for picking up the second of two yellow cards for a foul on Georgios Samaras.
In the end the nervy home support were happy to hear the final whistle although Majstorovic's red card will no doubt be the talking point for a few days. Celtic, with only one change to their line-up, went into the game one point ahead of Rangers following the Light Blues' 4-1 win at Dunfermline earlier in the day.
Ryan Esson returned to the Inverness goal at the expense of Jonathan Tuffey, while Williams, Claude Gnakpa and Shane Sutherland came in for Gregory Tade, Ross Tokely and David Proctor, the latter two starting on the bench along with Tuffey.
Caley had lost 2-0 at home to Celtic in the William Hill Scottish Cup fifth-round last week and once more they had a shaky start.
There were less than two minutes played when Hoops midfielder Victor Wanyama headed a Charlie Mulgrew corner just over the bar and it appeared that the visitors could be in for a long afternoon.
However, three minutes later a fine pass from Gnakpa found Nick Ross in acres of space just outside the Celtic box but the Caley midfielder failed to control the ball instantly and the opportunity was gone.
Caley's chances of a shock win, though, took a severe dent when Celtic forged ahead through Ledley.
Skipper Scott Brown sent the unmarked Forrest free down the right-hand side and his low cross to the back post was tapped in by the Wales international from a yard out.
Five minutes later Mulgrew forced Esson into a fine save just under the bar from a whipped-in free-kick but Celtic made nothing of the corner.
The Hoops fans were in celebratory mode but there were occasional flashes from Terry Butcher's men which suggested they could pose a threat.
In the 29th minute Caley midfielder Greg Tansey sent Hayes racing into the Celtic box with a pass which again cut open the Parkhead rearguard but he was crowded out before he could get his shot away.
Mulgrew came close in the 34th minute when he drilled a 25-yard shot past low past Esson's far post, after the Inverness defence had repelled a Forrest cross from the left.
Esson denied Celtic with three minutes of the first-half remaining when he did well to cut out a dangerous low cross from Forrest who had returned to the right-hand side after a spell on the left.
There was little really to get excited in the second half until, in a moment of drama, Majstorovic was shown a straight red card by referee McLean.
It appeared the Sweden defender, while originally not realising that Hayes was behind him as he watched Mulgrew's sliding pass back go on its way back to Fraser Forster, got a touch on the ball with a recovery tackle at the edge of the box.
The stand side assistant kept his flag down but McLean acted and his decision angered the home support and Lennon, who was spoken to by McLean after appearing to remonstrate with fourth official Charlie Richmond.
Ki Sung-yueng replaced Gary Hooper as Celtic reshuffled but the visitors sensed their might be something in the game for them.
The numbers were evened up by McLean when Williams, booked just after the break for a foul on Hooper, was sent off for a second yellow card offence on Samaras.
The complexion of the game changed again but Ross missed a decent chance for Caley when he screwed his angled-drive from 10 yards wide of the target.
At the other end, Samaras finished a mazy run with a low drive which tested Esson but in the end Celtic were happy to see the game out.