Hoops midfielder Ki Sung-yueng gave the home side the lead in the 17th minute when he finished off a fine move but the home side failed to build on that and Dons defender Ryan Jack levelled on the hour mark to set the nerves jangling among the home support.
However, in the 72nd minute, following a Ki free-kick, former Aberdeen defender Mulgrew drove high into the net from close range before Jack - booked for over-celebrating his equaliser by running to the travelling support - was sent off after picking up the second card for a foul on Gary Hooper.
Mulgrew, though, emerged again as the Hoops hero after grabbing an equaliser in the 3-3 draw at Kilmarnock last week.
Celtic have a game in hand over Rangers but Lennon will not want many more nervy afternoons like this.
All the recent statistics pointed to a comfortable home win for Lennon's side, which had striker Hooper back in after recovering from an ankle injury, to replace Cha Du-ri.
The Dons had lost the corresponding fixture last year 9-0 and were beaten in all five encounters between the two teams in three different competitions last season, conceding 21 goals and scoring just one.
Aberdeen, though, had won their last two SPL games and kept the same side for the third time with Rory Fallon and Scott Vernon leading the line, which - if nothing else - was a statement of intent from boss Craig Brown.
The Dons did not look intimidated in the early stages and indeed Kari Arnason's long-distance drive in the seventh minute, which went well wide, only alleviated the pressure on the Hoops defence.
Moments later Celtic goalkeeper Fraser Forster did well to cut out a dangerous cross from Dons wide-man Fraser Fyvie.
In the 13th minute Ki set up Beram Kayal 30 yards from goal but the midfielder's shot was wide.
However, Ki opened the scoring when he finished a fine move started by Anthony Stokes and Kayal by taking a Hooper pass and steering the ball past David Gonzalez and inside the far post from 10 yards, although it looked as if the visiting keeper might have done better.
Parkhead stopper Glenn Loovens, having picked up an injury, was immediately replaced by the man he had recently replaced, Daniel Majstorovic.
The rest of the half was untidy. Aberdeen responded to an extent by forcing a couple of corners but they came to nothing.
The visitors had lost their earlier belief, resorting to long balls to get them up the park but posed little threat to Forster.
The visitors started the second-half sharply but Ki should have doubled Celtic's lead in the 49th minute when Matthews' cross from the right came right across his path - but the midfielder drove his left-footed shot wide.
In the 55th minute, Joe Ledley - playing left-back again - fired in a cross which was cleared to the edge of the box - but Hooper's shot took a deflection and Gonzalez saved at his left-hand post.
Aberdeen levelled on the hour when full-back Ryan Jack weaved his way into the Celtic box and as the Parkhead defence hesitated, fired in a left-foot drive which squirmed its way in at the far post.
Moments later, Stokes fired a free-kick over from 25 yards - but the mood inside Celtic Park had changed.
Celtic piled on the pressure and Gonzalez had to tip a powerful drive from Matthews over the bar for a corner and the Dons were glad to escape two more scares within a minute when Hooper had chances.
However, Mulgrew restored Celtic's lead from close range after Ki's long free-kick fell to him six yards out, the former Dons player sweeping the ball high into the net.
The Dons were deflated and in the 80th minute, Jack was shown a second yellow by referee Stevie O'Reilly for a foul on Hooper..
Five minutes later Hoops substitute Paddy McCourt, on earlier for Victor Wanyama, twisted and turned inside the Dons box but failed to hit the target and in the end the home side were glad to hear the final whistle.
