Kris Commons scored from the spot in the 18th minute after Gorgie midfielder Jamie Hamill was controversially penalised for handball before Jason Holt equalised in the 56th with a deflected drive from the edge of the box.
Anthony Stokes put Celtic back in front after 65 minutes with an unstoppable volley from 12 yards before substitute Pukki, who replaced Commons, made it 3-1 five minutes from time to make a perfect start to his Hoops career following his move from Schalke 04.
The win had lifted Celtic level on points with Inverness but Caley later reclaimed sole leadership when Billy McKay hit his sixth goal in as many games to give them a 2-1 victory over Kilmarnock at Rugby Park.
McKay lobbed home after Kilmarnock were caught out by a ball over the top 10 minutes into the second half as Inverness restored their three-point lead.
Richie Foran had headed home from close range to give Inverness an early lead and they deserved the points despite Kilmarnock improving after a poor first half.
Although Killie started the second half strongly, only a freak ricochet off Barry Nicholson prevented Caley Thistle from securing a fifth clean sheet of the season.
McKay restored the lead five minutes later and Inverness defended stoutly to seal the points in front of only 3,063 fans as Killie continue to seek their first win under Allan Johnston.
The Easter Road outfit had fallen behind to Stevie May's 17th-minute opener but Paul Heffernan levelled in the 35th minute before Collins' second-half effort secured a hard-fought victory.
Hibernian, who won 2-1 at Kilmarnock in their last match on their travels, move up into the top six as a result of their success.
St Mirren old boy John Sutton returned to haunt his former club and heap further pressure on the under-fire Danny Lennon as he grabbed the only goal for Motherwell in a hard-fought encounter at St Mirren Park.
The goal came in the 38th minute when excellent play between Iain Vigurs and Zaine Francis-Angol on the left allowed the latter space to hang up an inviting cross to the back post, where Sutton rose highest to plant his third goal of the season past David Cornell.
Calvin Zola, Peter Pawlett and Josh Magennis took advantage of some poor Thistle defending, as Derek McInnes' younger players stepped up to the mark at Firhill.